Drainer不支持的DDL-1
版本3.0.5
1.rename table t1 to b_db.b_t1
已经做好TiDB->MySQL的同步, 开启了pump, drainer
问题复现
在TiDB创建fanboshi表
1 | root@10.152.x.150 17:53:27 [sysbench]> show tables; |
在下游MySQL查看, 同步没有问题
1 | root@localhost 18:59:35 [sysbench]> show tables; |
在TiDB执行
1 | root@10.152.x.150 18:57:59 [sysbench]> create database tidb_sysbench; |
Drainer报错退出
1 | [2019/12/02 18:59:14.979 +08:00] [INFO] [collector.go:284] ["get ddl job"] [job="ID:79, Type:rename table, State:synced, SchemaState:public, SchemaID:77, TableID:75, RowCount:0, ArgLen:0, start time: 2019-12-02 18:59:14.012 +0800 CST, Err:<nil>, ErrCount:0, SnapshotVersion:0"] |
问题修复
参考同步时出现上游数据库支持但是下游数据库执行会出错的 DDL,应该怎么办?)
根据drainer.log中的日志信息我们获取这个ddl的commit ts
为412951341698121738
在中控机配置文件或者直接在drainer所属服务器的配置文件中直接修改都可以, 我这里在中控机修改, 因为我只有一个drainer. 如果有多个drainer建议在drainer服务器修改配置文件
1.添加要忽略的commit-ts
1 | # ignore syncing the txn with specified commit ts to downstream |
注意这个应该填写integer, 如果填写为
"412951341698121738"
drainer会起不来, 而且使用systemctl start drainer-8249.service
无任何错误提示/DATA1/deploy/log/drainer_stderr.log
日志业务内容
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25 [root@knode10-152-1-166 deploy]# cat /etc/systemd/system/drainer-8249.service
[Unit]
Description=drainer-8249 service
After=syslog.target network.target remote-fs.target nss-lookup.target
[Service]
LimitNOFILE=1000000
#LimitCORE=infinity
LimitSTACK=10485760
User=tidb
ExecStart=/DATA1/deploy/scripts/run_drainer.sh
Restart=on-failure
RestartSec=15s
SendSIGKILL=no
[Install]
WantedBy=multi-user.target
[root@knode10-152-1-166 deploy]# ./bin/drainer --addr="10.152.x.166:8249" --pd-urls="http://10.152.x.152:2379,http://10.152.x.146:2379,http://10.152.x.147:2379" --data-dir="/DATA1/deploy/data.drainer" --log-file="/DATA1/deploy/log/drainer.log" --config=conf/drainer.toml --initial-commit-ts="412950687779913731" 2>> "/DATA1/deploy/log/drainer_stderr.log"
[2019/12/02 19:17:35.996 +08:00] [FATAL] [main.go:39] ["verifying flags failed, See 'drainer --help'."] [error="toml: cannot load TOML value of type string into a Go integer"] [errorVerbose="toml: cannot load TOML value of type string into a Go integer\ngithub.com/pingcap/errors.AddStack\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/pkg/mod/github.com/pingcap/errors@v0.11.4/errors.go:174\ngithub.com/pingcap/errors.Trace\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/pkg/mod/github.com/pingcap/errors@v0.11.4/juju_adaptor.go:15\ngithub.com/pingcap/tidb-binlog/pkg/util.StrictDecodeFile\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-binlog/pkg/util/util.go:188\ngithub.com/pingcap/tidb-binlog/drainer.(*Config).configFromFile\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-binlog/drainer/config.go:226\ngithub.com/pingcap/tidb-binlog/drainer.(*Config).Parse\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-binlog/drainer/config.go:169\nmain.main\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-binlog/cmd/drainer/main.go:38\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:1337"] [stack="github.com/pingcap/log.Fatal\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/pkg/mod/github.com/pingcap/log@v0.0.0-20190715063458-479153f07ebd/global.go:59\nmain.main\n\t/home/jenkins/agent/workspace/release_tidb_3.0/go/src/github.com/pingcap/tidb-binlog/cmd/drainer/main.go:39\nruntime.main\n\t/usr/local/go/src/runtime/proc.go:200"]
查看日志无内容
[root@knode10-152-1-166 deploy]# cat /DATA1/deploy/log/drainer_stderr.log
[root@knode10-152-1-166 deploy]#
2.在MySQL手动执行失败的DDL语句
1 | root@localhost 19:09:27 [sysbench]> rename table fanboshi to tidb_sysbench.tidb_fanboshi; |
3.启动drainer
1 | systemctl start drainer-8249.service |
5.验证同步
1 | [root@node10-133-1-59 fanboshi]# sh sysbench_tidb_run.sh |
TiDB
1 | root@10.152.x.150 19:20:43 [sysbench]> select count(*) from sbtest1; |
下游MySQL
1 | root@localhost 19:20:08 [sysbench]> select count(*) from sbtest1; |
同步恢复