canal.mq.flatMessage
是否为json格式
如果设置为false,对应MQ收到的消息为protobuf格式
需要通过CanalMessageDeserializer进行解码
canal.mq.flatMessage = true 生产到kafka的消息就是json的, 否则就是protobuf二进制的1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28$bin/kafka-console-consumer.sh --bootstrap-server 172.16.23.174:9092 --topic fanboshi.t1 --from-beginning
{"data":null,"database":"fanboshi","es":1583414454000,"id":2,"isDdl":true,"mysqlType":null,"old":null,"pkNames":null,"sql":"create table t1(id int not null auto_increment primary key,sname varchar(10))","sqlType":null,"table":"t1","ts":1583414454111,"type":"CREATE"}
{"data":[{"id":"1","sname":"fan"}],"database":"fanboshi","es":1583414520000,"id":3,"isDdl":false,"mysqlType":{"id":"int(11)","sname":"varchar(10)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":4,"sname":12},"table":"t1","ts":1583414520403,"type":"INSERT"}
{"data":[{"id":"1","sname":"fan"}],"database":"fanboshi","es":1583414520000,"id":1,"isDdl":false,"mysqlType":{"id":"int(11)","sname":"varchar(10)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":4,"sname":12},"table":"t1","ts":1583414603231,"type":"INSERT"}
{"data":[{"id":"2","sname":"bo"}],"database":"fanboshi","es":1583414621000,"id":2,"isDdl":false,"mysqlType":{"id":"int(11)","sname":"varchar(10)"},"old":null,"pkNames":["id"],"sql":"","sqlType":{"id":4,"sname":12},"table":"t1","ts":1583414621766,"type":"INSERT"}
*
mysql-bin.000002 *UTF-80֊.8fanboshiJt1P,Xb6
curtGtid*c30c6a02-4e32-11ea-84ec-fa163edcd14e:65460b
curtGtidSn64398b
curtGtidLct64397b
_-+_C-+++1+,c30c6a02-4e32-11ea-84ec-fa163edcd14e:1-65460Pbid (0B1Ri++(11)#
_+a+e (0Bfa+R
+a_cha_(10)
+y_-+-bi+.000002 *UTF-80֊.8fa+b-_hiJ+1P+Xb6
c+_+G+id*c30c6a02-4e32-11ea-84ec-fa163edcd14e:65461b
c+_+G+idS+64399b
c+_+G+idLc+64398b
_-+_C-+++1+,c30c6a02-4e32-11ea-84ec-fa163edcd14e:1-65461Pbid (0B2Ri++(11)"
_+a+e (0Bb-R
+a_cha_(10)
*
乱码部分就是protobuf的
protobuf性能要好很多. 1.1.5性能有很大提升. 即便canal.mq.flatMessage = true性能也比以前好了很多很多
canal.mq.flatMessage = true默认的格式没带GTID