0
0
0
0
博客/.../

Tidb v8.5.4新特性 - 支持表级别数据打散功能

 lmdb  发表于  2025-12-25

平凯数据库在11月27日正式发布了 v.8.5.4版本,提到新特性“支持表级别数据打散功能”,虽然现在是实验特性不建议在生产库中实践,这个特性的引入可能解决日常运维中痛点问题 , 因 TiKV 节点上的数据分布不均,造成数据热块问题。

这里做尝鲜熟悉下新特性,仅此而已,也希望这特性在下个版本中正式发布。

支持表级别数据打散功能(实验特性)#63260 @bufferflies

PD 会自动调度数据,将整个集群的数据尽可能均匀地分布到所有 TiKV 节点上。然而,这种自动调度是基于集群全局的。在某些场景下,尽管整个集群的数据分布是均衡的,但某张表在各个 TiKV 节点上的数据分布仍然可能不均匀。

从 v8.5.4 开始,你可以通过 SHOW TABLE DISTRIBUTION 语句查看某张表在集群中所有 TiKV 节点上的数据分布情况。如果存在数据分布不均衡,可以通过 DISTRIBUTE TABLE 语句对该表进行数据打散(实验特性),以提升负载均衡性。

表级数据打散功能属于一次性执行任务,并设有超时时间限制。如果到达超时时间后,打散任务还未完成,则会自动退出。

这里提到了新查看命令 SHOW TABLE DISTRIBUTIONDISTRIBUTE TABLE,分别对表进行查看数据分别和打散数据,这与我们经常使用 SHOW TABLE REGIONS 语句显示 TiDB 中某个表的 Region 信息还是不同。

SHOW TABLE REGIONS 会返回如下列:

  • REGION_ID:Region 的 ID。
  • START_KEY:Region 的 Start key。
  • END_KEY:Region 的 End key。
  • LEADER_ID:Region 的 Leader ID。
  • LEADER_STORE_ID:Region leader 所在的 store (TiKV) ID。
  • PEERS:Region 所有副本的 ID。
  • SCATTERING:Region 是否正在调度中。1 表示正在调度。
  • WRITTEN_BYTES:估算的 Region 在 1 个心跳周期内的写入数据量大小,单位是 byte。
  • READ_BYTES:估算的 Region 在 1 个心跳周期内的读数据量大小,单位是 byte。
  • APPROXIMATE_SIZE(MB):估算的 Region 的数据量大小,单位是 MB。
  • APPROXIMATE_KEYS:估算的 Region 内 Key 的个数。
  • SCHEDULING_CONSTRAINTS:Region 所在的表或者分区所关联的放置策略的规则。
  • SCHEDULING_STATE:关联了放置策略的 Region 的当前调度状态。

SHOW TABLE DISTRIBUTION 会返回如下列:

  • PARTITION_NAME: STORE_ID: STORE_TYPE: REGION_LEADER_COUNT: REGION_PEER_COUNT: REGION_WRITE_BYTES: REGION_WRITE_KEYS: REGION_WRITE_QUERY: REGION_LEADER_READ_BYTES: REGION_LEADER_READ_KEYS: REGION_LEADER_READ_QUERY: REGION_PEER_READ_BYTES: REGION_PEER_READ_KEYS: REGION_PEER_READ_QUERY:

对新特性做简单的测试

MySQL [sakila]> 
MySQL [sakila]> CREATE TABLE tt (
    ->  id INT NOT NULL PRIMARY KEY auto_increment,
    ->  b INT NOT NULL,
    ->  pad1 VARBINARY(1024),
    ->  pad2 VARBINARY(1024),
    ->  pad3 VARBINARY(1024)
    -> )ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin 
    -> PARTITION BY RANGE (`id`)
    -> (PARTITION `p1` VALUES LESS THAN (10000),
    -> PARTITION `p2` VALUES LESS THAN (20000),
    -> PARTITION `p3` VALUES LESS THAN (30000),
    -> PARTITION `p4` VALUES LESS THAN (40000),
    -> PARTITION `p-max` VALUES LESS THAN (MAXVALUE));
Query OK, 0 rows affected (0.61 sec)

MySQL [sakila]> 
MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM dual;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 1 row affected (0.04 sec)
Records: 1  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 1 row affected (0.04 sec)
Records: 1  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 8 rows affected (0.01 sec)
Records: 8  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 1000 rows affected (0.02 sec)
Records: 1000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.14 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.16 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.23 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.18 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.35 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.32 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.46 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> INSERT INTO tt SELECT NULL, FLOOR(RAND()*1000), RANDOM_BYTES(1024), RANDOM_BYTES(1024), RANDOM_BYTES(1024) FROM tt a JOIN tt b JOIN tt c LIMIT 10000;
Query OK, 10000 rows affected (0.36 sec)
Records: 10000  Duplicates: 0  Warnings: 0

MySQL [sakila]> select count(*) from tt;
+----------+
| count(*) |
+----------+
|    81010 |
+----------+
1 row in set (0.11 sec)

MySQL [sakila]> SELECT SLEEP(5);
+----------+
| SLEEP(5) |
+----------+
|        0 |
+----------+
1 row in set (5.01 sec)

MySQL [sakila]> 
MySQL [sakila]> SHOW TABLE tt DISTRIBUTIONS;
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
| PARTITION_NAME | STORE_ID | STORE_TYPE | REGION_LEADER_COUNT | REGION_PEER_COUNT | REGION_WRITE_BYTES | REGION_WRITE_KEYS | REGION_WRITE_QUERY | REGION_LEADER_READ_BYTES | REGION_LEADER_READ_KEYS | REGION_LEADER_READ_QUERY | REGION_PEER_READ_BYTES | REGION_PEER_READ_KEYS | REGION_PEER_READ_QUERY |
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
| p1             |        1 | tikv       |                   1 |                 1 |              44195 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p1             |        2 | tikv       |                   0 |                 1 |              44195 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p1             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        1 | tikv       |                   1 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        2 | tikv       |                   0 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        1 | tikv       |                   1 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        2 | tikv       |                   0 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        2 | tikv       |                   0 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        1 | tikv       |                   1 |                 1 |              44193 |               108 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        1 | tikv       |                   1 |                 1 |              57695 |               616 |                  1 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        2 | tikv       |                   0 |                 1 |              57695 |               616 |                  1 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
15 rows in set (0.02 sec)

MySQL [sakila]> SHOW TABLE tt REGIONS;
+-----------+-----------+--------------------+-----------+-----------------+----------+------------+---------------+------------+----------------------+------------------+------------------------+------------------+
| REGION_ID | START_KEY | END_KEY            | LEADER_ID | LEADER_STORE_ID | PEERS    | SCATTERING | WRITTEN_BYTES | READ_BYTES | APPROXIMATE_SIZE(MB) | APPROXIMATE_KEYS | SCHEDULING_CONSTRAINTS | SCHEDULING_STATE |
+-----------+-----------+--------------------+-----------+-----------------+----------+------------+---------------+------------+----------------------+------------------+------------------------+------------------+
|       270 | t_192_    | t_193_             |       272 |               1 | 271, 272 |          0 |        566907 |          0 |                    1 |                0 |                        |                  |
|       273 | t_193_    | t_194_             |       275 |               1 | 274, 275 |          0 |        566604 |          0 |                    1 |                0 |                        |                  |
|       276 | t_194_    | t_195_             |       278 |               1 | 277, 278 |          0 |        566616 |          0 |                    1 |                0 |                        |                  |
|       279 | t_195_    | t_196_             |       281 |               1 | 280, 281 |          0 |        566616 |          0 |                    1 |                0 |                        |                  |
|        28 | t_196_    | t_281474976710648_ |        31 |               1 | 29, 31   |          0 |       3461718 |          0 |                    1 |                0 |                        |                  |
+-----------+-----------+--------------------+-----------+-----------------+----------+------------+---------------+------------+----------------------+------------------+------------------------+------------------+
5 rows in set (0.04 sec)

MySQL [sakila]>

对表的 Region 进行重新打散和调度,以实现表级别的均衡分布, 语法很简单,这里可以指定ENGINE 是tikv 或者tiflash。

MySQL [sakila]> DISTRIBUTE TABLE tt RULE = "leader-scatter" ENGINE = "tikv" TIMEOUT = "1h";
+--------+
| JOB_ID |
+--------+
|      0 |
+--------+
1 row in set (0.06 sec)
MySQL [sakila]> admin show ddl jobs;
+--------+---------+----------------------------+--------------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+----------+
| JOB_ID | DB_NAME | TABLE_NAME                 | JOB_TYPE     | SCHEMA_STATE | SCHEMA_ID | TABLE_ID | ROW_COUNT | CREATE_TIME                | START_TIME                 | END_TIME                   | STATE  | COMMENTS |
+--------+---------+----------------------------+--------------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+----------+
|    197 | sakila  | tt                         | create table | public       |       145 |      191 |         0 | 2025-12-24 22:31:58.749000 | 2025-12-24 22:31:58.799000 | 2025-12-24 22:31:58.899000 | synced |          |
|    190 | sakila  | t                          | create table | public       |       145 |      187 |         0 | 2025-12-24 22:29:23.454000 | 2025-12-24 22:29:23.850000 | 2025-12-24 22:29:24.149000 | synced |          |
|    186 | sakila  | t1                         | create table | public       |       145 |      185 |         0 | 2025-12-24 22:02:32.149000 | 2025-12-24 22:02:32.149000 | 2025-12-24 22:02:32.299000 | synced |          |
|    184 | sakila  | actor_info                 | create view  | public       |       145 |      183 |         0 | 2025-12-24 21:55:04.699000 | 2025-12-24 21:55:04.699000 | 2025-12-24 21:55:04.749000 | synced |          |
|    182 | sakila  | sales_by_film_category     | create view  | public       |       145 |      181 |         0 | 2025-12-24 21:55:04.149000 | 2025-12-24 21:55:04.149000 | 2025-12-24 21:55:04.249000 | synced |          |
|    180 | sakila  | nicer_but_slower_film_list | create view  | public       |       145 |      179 |         0 | 2025-12-24 21:55:03.600000 | 2025-12-24 21:55:03.649000 | 2025-12-24 21:55:03.699000 | synced |          |
|    178 | sakila  | film_list                  | create view  | public       |       145 |      177 |         0 | 2025-12-24 21:55:03.049000 | 2025-12-24 21:55:03.101000 | 2025-12-24 21:55:03.149000 | synced |          |
|    176 | sakila  | store                      | create table | public       |       145 |      175 |         0 | 2025-12-24 21:55:02.549000 | 2025-12-24 21:55:02.549000 | 2025-12-24 21:55:02.699000 | synced |          |
|    174 | sakila  | staff                      | create table | public       |       145 |      173 |         0 | 2025-12-24 21:55:01.999000 | 2025-12-24 21:55:02.049000 | 2025-12-24 21:55:02.199000 | synced |          |
|    172 | sakila  | rental                     | create table | public       |       145 |      171 |         0 | 2025-12-24 21:55:01.451000 | 2025-12-24 21:55:01.499000 | 2025-12-24 21:55:01.649000 | synced |          |
+--------+---------+----------------------------+--------------+--------------+-----------+----------+-----------+----------------------------+----------------------------+----------------------------+--------+----------+
10 rows in set (0.02 sec)

MySQL [sakila]> SHOW TABLE tt DISTRIBUTIONS;
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
| PARTITION_NAME | STORE_ID | STORE_TYPE | REGION_LEADER_COUNT | REGION_PEER_COUNT | REGION_WRITE_BYTES | REGION_WRITE_KEYS | REGION_WRITE_QUERY | REGION_LEADER_READ_BYTES | REGION_LEADER_READ_KEYS | REGION_LEADER_READ_QUERY | REGION_PEER_READ_BYTES | REGION_PEER_READ_KEYS | REGION_PEER_READ_QUERY |
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
| p1             |        1 | tikv       |                   1 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p1             |        2 | tikv       |                   0 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p1             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        1 | tikv       |                   1 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        2 | tikv       |                   0 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p2             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        1 | tikv       |                   1 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        2 | tikv       |                   0 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p3             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        1 | tikv       |                   1 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        2 | tikv       |                   0 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p4             |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        1 | tikv       |                   1 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        2 | tikv       |                   0 |                 1 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
| p-max          |        7 | tikv       |                   0 |                 0 |                  0 |                 0 |                  0 |                        0 |                       0 |                        0 |                      0 |                     0 |                      0 |
+----------------+----------+------------+---------------------+-------------------+--------------------+-------------------+--------------------+--------------------------+-------------------------+--------------------------+------------------------+-----------------------+------------------------+
15 rows in set (0.01 sec)

MySQL [sakila]> 

注意事项

DISTRIBUTE TABLE 语句在重新调度表中的 Region 时,可能会受到 PD 热点调度器的影响。调度完成后,随着时间推移,表的 Region 分布可能再次失衡。

0
0
0
0

版权声明:本文为 TiDB 社区用户原创文章,遵循 CC BY-NC-SA 4.0 版权协议,转载请附上原文出处链接和本声明。

评论
暂无评论