一、服务器列表
| Hostname | Ipv4 | Ipv6 |
|---|---|---|
| test1 | 10.55.155.129 | 9999::aa |
| test2 | 10.55.155.130 | 9999::bb |
| test3 | 10.55.155.131 | 9999::cc |
二、手动配置ipv6地址
注意：fe80开头的ipv6地址集群不支持
test1:
# vim /etc/sysconfig/network-scripts/ifcfg-bond0
BONDING_OPTS="miimon=100 mode=active-backup"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=bond0
UUID=b7ab1b3b-97dc-419c-a3cd-cafbe2886396
DEVICE=bond0
ONBOOT=yes
IPADDR=10.55.155.129
PREFIX=24
GATEWAY=10.55.155.254
IPV6ADDR=9999::aa/96
TPV6_DEFAILTGW=9999::1
test2：
# vim /etc/sysconfig/network-scripts/ifcfg-bond0
BONDING_OPTS="miimon=100 mode=active-backup"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=bond0
UUID=38b0c2a4-f3b5-4085-9624-c66b43cc9346
DEVICE=bond0
ONBOOT=yes
IPADDR=10.55.155.130
PREFIX=24
GATEWAY=10.55.155.254
IPV6ADDR=9999::bb/96
TPV6_DEFAILTGW=9999::1
test3：
# vim /etc/sysconfig/network-scripts/ifcfg-bond0
BONDING_OPTS="miimon=100 mode=active-backup"
TYPE=Bond
BONDING_MASTER=yes
PROXY_METHOD=none
BROWSER_ONLY=no
BOOTPROTO=none
DEFROUTE=yes
IPV4_FAILURE_FATAL=no
IPV6INIT=yes
IPV6_AUTOCONF=no
IPV6_DEFROUTE=yes
IPV6_FAILURE_FATAL=no
IPV6_ADDR_GEN_MODE=stable-privacy
NAME=bond0
UUID=6d4451e5-ccb5-4407-8a50-37d6d18121f9
DEVICE=bond0
ONBOOT=yes
IPADDR=10.55.155.131
PREFIX=24
GATEWAY=10.55.155.254
IPV6ADDR=9999::cc/96
TPV6_DEFAILTGW=9999::1
依次重启3台服务的network服务。
systemctl restart network
三、准备yaml文件
$ cat topology_ipv6.yaml
global:
user: "tidb"
ssh_port: 22
deploy_dir: "/tidb-deploy"
data_dir: "/tidb-data"
server_configs:
tidb:
performance.max-procs: 32 #numa的cpu总数（64）
token-limit: 5000
tmp-storage-path: /data/tmp_storage_path
new_collations_enabled_on_first_bootstrap: false
enable-telemetry: false
performance.txn-total-size-limit: 1073741824
tikv-client.grpc-connection-count: 16
performance.stmt-count-limit: 10000
tikv:
storage.block-cache.capacity: "1GB" #MEM_TOTAL * 0.5/N(512G服务器4实例)
raftstore.apply-pool-size: 3
raftstore.store-pool-size: 3
readpool.unified.max-thread-count: 4
log.file.max-backups: 1000
log.file.max-days: 60
log-backup.temp-path: /data/tidb-test/log-backup-temp
server.grpc-compression-type: "gzip"
server.grpc-raft-conn-num: 8
coprocessor.split-region-on-table: true
server.grpc-keepalive-time: "120s"
server.grpc-keepalive-timeout: "120s"
storage.flow-control.memtables-threshold: 10
pd:
replication.max-replicas: 3
schedule.max-store-down-time: "1h0m0s"
pd_servers:
- host: 9999::aa
- host: 9999::bb
- host: 9999::cc
tidb_servers:
- host: 9999::aa
- host: 9999::bb
- host: 9999::cc
tikv_servers:
- host: 9999::aa
- host: 9999::bb
- host: 9999::cc
monitoring_servers:
- host: 9999::aa
grafana_servers:
- host: 9999::aa
alertmanager_servers:
- host: 9999::aa
四、安装集群
安装前置步骤省略，具体参考官档本地安装集群部分。https://docs.pingcap.com/zh/tidb/v7.5/production-deployment-using-tiup
$ tiup cluster deploy v7testpool_ipv6 v7.5.3 topology_ipv6.yaml
+ Detect CPU Arch Name
- Detecting node 9999::aa Arch info ... Done
- Detecting node 9999::bb Arch info ... Done
- Detecting node 9999::cc Arch info ... Done
+ Detect CPU OS Name
- Detecting node 9999::aa OS info ... Done
- Detecting node 9999::bb OS info ... Done
- Detecting node 9999::cc OS info ... Done
Please confirm your topology:
Cluster type: tidb
Cluster name: v7testpool_ipv6
Cluster version: v7.5.3
Role Host Ports OS/Arch Directories
---- ---- ----- ------- -----------
pd 9999::aa 2379/2380 linux/x86_64 /tidb-deploy/pd-2379,/tidb-data/pd-2379
pd 9999::bb 2379/2380 linux/x86_64 /tidb-deploy/pd-2379,/tidb-data/pd-2379
pd 9999::cc 2379/2380 linux/x86_64 /tidb-deploy/pd-2379,/tidb-data/pd-2379
tikv 9999::aa 20160/20180 linux/x86_64 /tidb-deploy/tikv-20160,/tidb-data/tikv-20160
tikv 9999::bb 20160/20180 linux/x86_64 /tidb-deploy/tikv-20160,/tidb-data/tikv-20160
tikv 9999::cc 20160/20180 linux/x86_64 /tidb-deploy/tikv-20160,/tidb-data/tikv-20160
tidb 9999::aa 4000/10080 linux/x86_64 /tidb-deploy/tidb-4000
tidb 9999::bb 4000/10080 linux/x86_64 /tidb-deploy/tidb-4000
tidb 9999::cc 4000/10080 linux/x86_64 /tidb-deploy/tidb-4000
prometheus 9999::aa 9090/12020 linux/x86_64 /tidb-deploy/prometheus-9090,/tidb-data/prometheus-9090
grafana 9999::aa 3000 linux/x86_64 /tidb-deploy/grafana-3000
alertmanager 9999::aa 9093/9094 linux/x86_64 /tidb-deploy/alertmanager-9093,/tidb-data/alertmanager-9093
Attention:
1. If the topology is not what you expected, check your yaml file.
2. Please confirm there is no port/directory conflicts in same host.
Do you want to continue? [y/N]: (default=N) y
+ Generate SSH keys ... Done
+ Download TiDB components
- Download pd:v7.5.3 (linux/amd64) ... Done
- Download tikv:v7.5.3 (linux/amd64) ... Done
- Download tidb:v7.5.3 (linux/amd64) ... Done
- Download prometheus:v7.5.3 (linux/amd64) ... Done
- Download grafana:v7.5.3 (linux/amd64) ... Done
- Download alertmanager: (linux/amd64) ... Done
- Download node_exporter: (linux/amd64) ... Done
- Download blackbox_exporter: (linux/amd64) ... Done
+ Initialize target host environments
- Prepare 9999::aa:22 ... Done
- Prepare 9999::bb:22 ... Done
- Prepare 9999::cc:22 ... Done
+ Deploy TiDB instance
- Copy pd -> 9999::aa ... Done
- Copy pd -> 9999::bb ... Done
- Copy pd -> 9999::cc ... Done
- Copy tikv -> 9999::aa ... Done
- Copy tikv -> 9999::bb ... Done
- Copy tikv -> 9999::cc ... Done
- Copy tidb -> 9999::aa ... Done
- Copy tidb -> 9999::bb ... Done
- Copy tidb -> 9999::cc ... Done
- Copy prometheus -> 9999::aa ... Done
- Copy grafana -> 9999::aa ... Done
- Copy alertmanager -> 9999::aa ... Done
- Deploy node_exporter -> 9999::cc ... Done
- Deploy node_exporter -> 9999::aa ... Done
- Deploy node_exporter -> 9999::bb ... Done
- Deploy blackbox_exporter -> 9999::aa ... Done
- Deploy blackbox_exporter -> 9999::bb ... Done
- Deploy blackbox_exporter -> 9999::cc ... Done
+ Copy certificate to remote host
+ Init instance configs
- Generate config pd -> [9999::aa]:2379 ... Done
- Generate config pd -> [9999::bb]:2379 ... Done
- Generate config pd -> [9999::cc]:2379 ... Done
- Generate config tikv -> [9999::aa]:20160 ... Done
- Generate config tikv -> [9999::bb]:20160 ... Done
- Generate config tikv -> [9999::cc]:20160 ... Done
- Generate config tidb -> [9999::aa]:4000 ... Done
- Generate config tidb -> [9999::bb]:4000 ... Done
- Generate config tidb -> [9999::cc]:4000 ... Done
- Generate config prometheus -> [9999::aa]:9090 ... Done
- Generate config grafana -> [9999::aa]:3000 ... Done
- Generate config alertmanager -> [9999::aa]:9093 ... Done
+ Init monitor configs
- Generate config node_exporter -> 9999::bb ... Done
- Generate config node_exporter -> 9999::cc ... Done
- Generate config node_exporter -> 9999::aa ... Done
- Generate config blackbox_exporter -> 9999::cc ... Done
- Generate config blackbox_exporter -> 9999::aa ... Done
- Generate config blackbox_exporter -> 9999::bb ... Done
Enabling component pd
Enabling instance [9999::cc]:2379
Enabling instance [9999::aa]:2379
Enabling instance [9999::bb]:2379
Enable instance [9999::bb]:2379 success
Enable instance [9999::cc]:2379 success
Enable instance [9999::aa]:2379 success
Enabling component tikv
Enabling instance [9999::cc]:20160
Enabling instance [9999::aa]:20160
Enabling instance [9999::bb]:20160
Enable instance [9999::cc]:20160 success
Enable instance [9999::bb]:20160 success
Enable instance [9999::aa]:20160 success
Enabling component tidb
Enabling instance [9999::bb]:4000
Enabling instance [9999::cc]:4000
Enabling instance [9999::aa]:4000
Enable instance [9999::bb]:4000 success
Enable instance [9999::cc]:4000 success
Enable instance [9999::aa]:4000 success
Enabling component prometheus
Enabling instance [9999::aa]:9090
Enable instance [9999::aa]:9090 success
Enabling component grafana
Enabling instance [9999::aa]:3000
Enable instance [9999::aa]:3000 success
Enabling component alertmanager
Enabling instance [9999::aa]:9093
Enable instance [9999::aa]:9093 success
Enabling component node_exporter
Enabling instance 9999::cc
Enabling instance 9999::aa
Enabling instance 9999::bb
Enable 9999::bb success
Enable 9999::cc success
Enable 9999::aa success
Enabling component blackbox_exporter
Enabling instance 9999::cc
Enabling instance 9999::bb
Enabling instance 9999::aa
Enable 9999::cc success
Enable 9999::bb success
Enable 9999::aa success
Cluster `v7testpool_ipv6` deployed successfully, you can start it with command: `tiup cluster start v7testpool_ipv6 --init`
五、启动集群
$ tiup cluster start v7testpool_ipv6
Starting cluster v7testpool_ipv6...
+ [ Serial ] - SSHKeySet: privateKey=/home/tidb/.tiup/storage/cluster/clusters/v7testpool_ipv6/ssh/id_rsa, publicKey=/home/tidb/.tiup/storage/cluster/clusters/v7testpool_ipv6/ssh/id_rsa.pub
+ [Parallel] - UserSSH: user=tidb, host=9999::bb
+ [Parallel] - UserSSH: user=tidb, host=9999::cc
+ [Parallel] - UserSSH: user=tidb, host=9999::bb
+ [Parallel] - UserSSH: user=tidb, host=9999::cc
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::bb
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::aa
+ [Parallel] - UserSSH: user=tidb, host=9999::cc
+ [ Serial ] - StartCluster
Starting component pd
Starting instance [9999::cc]:2379
Starting instance [9999::bb]:2379
Starting instance [9999::aa]:2379
Start instance [9999::bb]:2379 success
Start instance [9999::cc]:2379 success
Start instance [9999::aa]:2379 success
Starting component tikv
Starting instance [9999::cc]:20160
Starting instance [9999::aa]:20160
Starting instance [9999::bb]:20160
Start instance [9999::bb]:20160 success
Start instance [9999::cc]:20160 success
Start instance [9999::aa]:20160 success
Starting component tidb
Starting instance [9999::cc]:4000
Starting instance [9999::bb]:4000
Starting instance [9999::aa]:4000
Start instance [9999::bb]:4000 success
Start instance [9999::cc]:4000 success
Start instance [9999::aa]:4000 success
Starting component prometheus
Starting instance [9999::aa]:9090
Start instance [9999::aa]:9090 success
Starting component grafana
Starting instance [9999::aa]:3000
Start instance [9999::aa]:3000 success
Starting component alertmanager
Starting instance [9999::aa]:9093
Start instance [9999::aa]:9093 success
Starting component node_exporter
Starting instance 9999::cc
Starting instance 9999::aa
Starting instance 9999::bb
Start 9999::bb success
Start 9999::cc success
Start 9999::aa success
Starting component blackbox_exporter
Starting instance 9999::cc
Starting instance 9999::aa
Starting instance 9999::bb
Start 9999::cc success
Start 9999::bb success
Start 9999::aa success
+ [ Serial ] - UpdateTopology: cluster=v7testpool_ipv6
Started cluster `v7testpool_ipv6` successfully
六、集群状态
$ tiup cluster display v7testpool_ipv6
Cluster type: tidb
Cluster name: v7testpool_ipv6
Cluster version: v7.5.3
Deploy user: tidb
SSH type: builtin
Dashboard URL: http://[9999::cc]:2379/dashboard
Grafana URL: http://[9999::aa]:3000
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir
-- ---- ---- ----- ------- ------ -------- ----------
[9999::aa]:9093 alertmanager 9999::aa 9093/9094 linux/x86_64 Up /tidb-data/alertmanager-9093 /tidb-deploy/alertmanager-9093
[9999::aa]:3000 grafana 9999::aa 3000 linux/x86_64 Up - /tidb-deploy/grafana-3000
[9999::aa]:2379 pd 9999::aa 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379
[9999::bb]:2379 pd 9999::bb 2379/2380 linux/x86_64 Up /tidb-data/pd-2379 /tidb-deploy/pd-2379
[9999::cc]:2379 pd 9999::cc 2379/2380 linux/x86_64 Up|L|UI /tidb-data/pd-2379 /tidb-deploy/pd-2379
[9999::aa]:9090 prometheus 9999::aa 9090/12020 linux/x86_64 Up /tidb-data/prometheus-9090 /tidb-deploy/prometheus-9090
[9999::aa]:4000 tidb 9999::aa 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
[9999::bb]:4000 tidb 9999::bb 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
[9999::cc]:4000 tidb 9999::cc 4000/10080 linux/x86_64 Up - /tidb-deploy/tidb-4000
[9999::aa]:20160 tikv 9999::aa 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
[9999::bb]:20160 tikv 9999::bb 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
[9999::cc]:20160 tikv 9999::cc 20160/20180 linux/x86_64 Up /tidb-data/tikv-20160 /tidb-deploy/tikv-20160
Total nodes: 12