1. 背景
TiCDC 作为 TiDB 生态系统中的关键组件，在数据复制和同步领域发挥着至关重要的作用。新版 TiCDC 在继承以往版本优势的基础上，进行了全面的优化和升级，进一步提升了性能、稳定性和功能的丰富性。
使用软件版本
过程中使用的软件版本
操作系统:CentOS Linux release 7.9.2009 (Core)
tidb:v8.1.2
mysql:5.7.34
mydumper:0.10.3-1
2. 安装
已经有了tidb的集群,本次将在原来的集群上增加cdc组件.
首先添加配置文件
[tidb@test5135 ~]$ cat scale-out.yml
cdc_servers:
- host: 192.168.5.135
gc-ttl: 86400
data_dir: /tidb-data/cdc-8300
- host: 192.168.5.184
gc-ttl: 86400
data_dir: /tidb-data/cdc-8300
- host: 192.168.5.185
gc-ttl: 86400
data_dir: /tidb-data/cdc-8300
使用tiup进行安装.
tiup cluster scale-out tidb-test scale-out.yml
安装完成后,出现提示信息
+ Refresh components conifgs
- Generate config pd -> 192.168.5.135:2379 ... Done
- Generate config pd -> 192.168.5.184:2379 ... Done
- Generate config pd -> 192.168.5.185:2379 ... Done
- Generate config tikv -> 192.168.5.135:20160 ... Done
- Generate config tikv -> 192.168.5.184:20160 ... Done
- Generate config tikv -> 192.168.5.185:20160 ... Done
- Generate config tidb -> 192.168.5.135:4000 ... Done
- Generate config tidb -> 192.168.5.184:4000 ... Done
- Generate config tidb -> 192.168.5.185:4000 ... Done
- Generate config cdc -> 192.168.5.135:8300 ... Done
- Generate config cdc -> 192.168.5.184:8300 ... Done
- Generate config cdc -> 192.168.5.185:8300 ... Done
- Generate config prometheus -> 192.168.5.135:9090 ... Done
- Generate config grafana -> 192.168.5.135:3000 ... Done
- Generate config alertmanager -> 192.168.5.135:9093 ... Done
+ Reload prometheus and grafana
- Reload prometheus -> 192.168.5.135:9090 ... Done
- Reload grafana -> 192.168.5.135:3000 ... Done
+ [ Serial ] - UpdateTopology: cluster=tidb-test
Scaled cluster `tidb-test` out successfully
查看集群的启动信息,cdc已经正常启动
[tidb@test5135 .tiup]$ tiup cluster display tidb-test
Cluster type: tidb
Cluster name: tidb-test
Cluster version: v8.1.2
Deploy user: tidb
SSH type: builtin
Dashboard URL: http://192.168.5.185:2379/dashboard
Grafana URL: http://192.168.5.135:3000
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir
-- ---- ---- ----- ------- ------ -------- ----------
192.168.5.135:9093 alertmanager 192.168.5.135 9093/9094 linux/x86_64 Up /tidb-data/alertmanager-9093 /tidb-deploy/alertmanager-9093
192.168.5.135:8300 cdc 192.168.5.135 8300 linux/x86_64 Up /tidb-data/cdc-8300 /tidb-deploy/cdc-8300
192.168.5.184:8300 cdc 192.168.5.184 8300 linux/x86_64 Up /tidb-data/cdc-8300 /tidb-deploy/cdc-8300
192.168.5.185:8300 cdc 192.168.5.185 8300 linux/x86_64 Up /tidb-data/cdc-8300 /tidb-deploy/cdc-8300
192.168.5.135:3000 grafana 192.168.5.135 3000 linux/x86_64 Up - /tidb-deploy/grafana-3000
192.168.5.135:2379 pd 192.168.5.135 2379/2380 linux/x86_64 Up|L /tidb-data/pd-2379 /tidb-deploy/pd-2379
192.168.5.184:2379 pd 192.168.5.184 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379
192.168.5.185:2379 pd 192.168.5.185 2379/2380 linux/x86_64 Up|UI /tidb-data/pd-2379 /tidb-deploy/pd-2379
192.168.5.135:9090 prometheus 192.168.5.135 9090/12020 linux/x86_64 Up /tidb-data/prometheus-8249 /tidb-deploy/prometheus-8249
192.168.5.135:4000 tidb 192.168.5.135 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
192.168.5.184:4000 tidb 192.168.5.184 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
192.168.5.185:4000 tidb 192.168.5.185 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
192.168.5.135:20160 tikv 192.168.5.135 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
192.168.5.184:20160 tikv 192.168.5.184 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
192.168.5.185:20160 tikv 192.168.5.185 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
Total nodes: 15
安装完成后,cdc命令不能直接使用,需要加载到环境变量中.
cdc的命令入口是cdc,保存在$deploy_dir/cdc-8300/bin下面.
将命令加入到环境变量export PATH=/home/tidb/.tiup/bin:/tidb-deploy/cdc-8300/bin:$PATH
重新加载环境变量source .bash_profile,已经可以正常使用.
[tidb@test5135 scripts]$ cdc version
Release Version: v8.1.2
Git Commit Hash: cd5e9e4ecdd7772a960b78cdbd2f1ebad7fd5cab
Git Branch: HEAD
UTC Build Time: 2024-12-19 12:13:09
Go Version: go version go1.21.13 linux/amd64
Failpoint Build: false
3. 启动全量迁移
3.1 关闭gc
因为在 Dumpling 退出后，GC 可能会被触发，从而导致增量变更迁移失败。
--执行如下命令关闭 GC：
mysql> SET GLOBAL tidb_gc_enable=FALSE;
Query OK, 0 rows affected (0.05 sec)
--查询 tidb_gc_enable 的取值，判断 GC 是否已关闭：
mysql> SELECT @@global.tidb_gc_enable;
+-------------------------+
| @@global.tidb_gc_enable |
+-------------------------+
| 0 |
+-------------------------+
1 row in set (0.01 sec)
3.2 tidb导出全量数据
使用dumpling从tidb导出全量数据,
[tidb@test5135 scripts]$tiup dumpling -u root -P 4000 -h 127.0.0.1 -p"EUi25N@8K1-7c+3^db" --filetype sql -t 8 -o ./dumpling_output -r 200000 -F256MiB
--以下为命令输出
A new version of dumpling is available: -> v8.1.2
To update this component: tiup update dumpling
To update all components: tiup update --all
The component `dumpling` version is not installed; downloading from repository.
Starting component dumpling: /home/tidb/.tiup/components/dumpling/v8.1.2/dumpling -u root -P 4000 -h 127.0.0.1 -p****** --filetype sql -t 8 -o ./dumpling_output -r 200000 -F256MiB
Release version: v8.1.2
Git commit hash: 80db67052c0b1d89ab5ea94d6efa576a2191432d
Git branch: HEAD
Build timestamp: 2024-12-24 06:39:16Z
Go version: go version go1.21.13 linux/amd64
...
[2025/02/26 15:08:30.394 +08:00] [INFO] [main.go:82] ["dump data successfully, dumpling will exit now"]
3.3 记录导出文件的TSO
在导出目录的metadata文件中记录导出的时间点信息,方便后面启动cdc增量迁移时使用.
[tidb@test5135 ~]$ cat dumpling_output/metadata
Started dump at: 2025-02-26 15:08:08
SHOW MASTER STATUS:
Log: tidb-binlog
Pos: 456275706010927125
GTID:
Finished dump at: 2025-02-26 15:08:30
4. mysql恢复全量数据
使用myloader将tidb导出的数据导入到mysql中,myloader使用0.10版本及以下版本,高版本读取元数据报错.
Release v0.10.3 · mydumper/mydumper · GitHub
启动mysql任务
[mysql@test5184 tidbback]$ myloader -u root -p "Cx1842#09Gjw0121" -S /data/mysql/data/mysql.sock -d /data/tidbback/dumpling_output/
5. 使用cdc部署增量任务
在上游集群中，执行以下命令创建从上游到下游mysql的同步链路：
cdc cli changefeed create \
--server=http://192.168.5.135:8300 \
--sink-uri="mysql://sync_data:As01FTsxqzHF0upeSkhmUMT@192.168.5.184:3306/" \
--changefeed-id="simple-replication-task" \
--start-ts="456275706010927125"
- --server:cdc的服务端
- --sink-uri:下游mysql的地址
- --changefeed-id:同步任务的ID,如果不写,将自动生成
- --start-ts:dumpling导出的时间戳
启动cdc任务,任务启动后,自动转入后台运行.
[tidb@test5135 dumpling_output]$ cdc cli changefeed create --server=http://192.168.5.135:8300 --sink-uri="mysql://sync_data:As01FTsxqzHF0upeSkhmUMT@192.168.5.184:3306/" --changefeed-id="simple-replication-task" --start-ts="456275706010927125"
Create changefeed successfully!
ID: simple-replication-task
[tidb@test5135 dumpling_output]$ ps -ef|grep cdc
tidb 1037 11576 0 15:48 pts/2 00:00:00 grep --color=auto cdc
tidb 6549 1 5 14:05 ? 00:05:23 bin/cdc server --addr 0.0.0.0:8300 --advertise-addr 192.168.5.135:8300 --pd http://192.168.5.135:2379,http://192.168.5.184:2379,http://192.168.5.185:2379 --data-dir=/tidb-data/cdc-8300 --gc-ttl 86400 --config conf/cdc.toml --log-file /tidb-deploy/cdc-8300/log/cdc.log
6. 测试
在tidb中创建一张表,并向表中添加数据,在mysql中查看数据.
--tidb
mysql> CREATE TABLE t1 (
-> id INT AUTO_INCREMENT PRIMARY KEY,
-> name VARCHAR(255),
-> created TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-> );
Query OK, 0 rows affected (0.17 sec)
mysql>
mysql>
mysql> insert into t1(name)values("aa");
Query OK, 1 row affected (0.01 sec)
mysql> select * from t1;
+----+------+---------------------+
| id | name | created |
+----+------+---------------------+
| 1 | aa | 2025-02-26 16:20:00 |
+----+------+---------------------+
1 row in set (0.01 sec)
--mysql
mysql> desc t1;
+---------+--------------+------+-----+-------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------+--------------+------+-----+-------------------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| name | varchar(255) | YES | | NULL | |
| created | timestamp | NO | | CURRENT_TIMESTAMP | |
+---------+--------------+------+-----+-------------------+----------------+
3 rows in set (0.00 sec)
mysql>
mysql> select * from t1;
+----+------+---------------------+
| id | name | created |
+----+------+---------------------+
| 1 | aa | 2025-02-26 16:20:00 |
+----+------+---------------------+
1 row in set (0.00 sec)
在借助 TiCDC 成功构建起上下游数据同步链路之后,原集群中写入的数据能够以极低的延迟快速同步至下游的mysql数据库.这一高效且稳定的同步效果,为后续的业务调整奠定了坚实基础.此时,我们便可以有条不紊地将读流量逐步迁移至新集群,充分发挥新集群的性能优势,进一步优化整体业务架构.
7. 问题
问题1
cdc启动时报错
[tidb@test5135 scripts]$ cdc cli changefeed create \
> --server=http://192.168.5.135:8300 \
> --sink-uri="mysql://sync_data:As01FTsxqzHF0upeSkhmUMT@192.168.5.184:3306/" \
> --changefeed-id="simple-replication-task"
Error: [CDC:ErrMySQLConnectionError]fail to open MySQL connection: Error 1298 (HY000): Unknown or incorrect time zone: 'Asia/Shanghai'
在mysql中导入时区后正常
[mysql@test5184 scripts]$ mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql -u root mysql -p
Enter password:
Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/leapseconds' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/tzdata.zi' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.