一.背景
针对 LTS 版本，PingCAP 会提供最多至 3 年时间的常规版本更新，以解决版本运行过程中遇到的问题，以及安全相关的漏洞修复。而对于已经结束维护周期的版本，官方不会提供支持。那么如果想要跟紧 TiDB 的脚步，可能最多 3 年就要做一次大版本的升级，下面是目前最新的版本支持时间表。
| Version | Release Date | Maintenance Support ends | Extended Support ends (EOL) |
|---|---|---|---|
| v7.1 | 2023-05-31 | 2026-05-31 | 2027-05-31 |
| v6.5 | 2022-12-29 | 2025-12-29 | 2026-12-29 |
| v6.1 | 2022-06-13 | 2025-06-13 | 2026-06-13 |
| v5.4 | 2022-02-15 | 2025-02-15 | 2026-02-15 |
| v5.3 | 2021-11-30 | 2023-11-30 | 2024-11-30 |
| v5.2 | 2021-08-27 | 2023-08-27 | 2024-08-27 |
| v5.1 | 2021-06-24 | 2023-06-24 | 2024-06-24 |
| v5.0 | 2021-04-07 | 2023-04-07 | 2024-04-07 |
| v4.0 | 2020-05-28 | 2023-04-02 | 2024-04-02 |
| v3.0 | 2019-06-28 | 2022-09-25 | 2023-09-25 |
TiDB 虽然有使用 tiup 进行原地升级的方式，但是目前暂不支持版本降级或升级后回退，还要考虑停机时间等问题，风险还是比较高的。而且如果版本跨度较大可能要进行多次升级才能到目标版本，操作也比较复杂。因此，条件允许的话还是建议尽量使用新旧集群之间做实时同步然后割接的方案来达到升级的目的。TiDB 作为分布式数据库有 tidb，pd 和 tikv 等多个组件，每个组件都有自己的参数，每次版本的升级可能都会有参数的变化，比如新增或者废弃的参数，或者参数的默认值可能会发生变化。如果不进行测试直接进行升级可能会由于参数变动发生预期之外的影响。
二.什么是 TiDBA
TiDBA 可以对比 2 个集群之间的参数，来识别新旧版本或者不同参数的变化。
从 toolkit 离线包中可以看到有几个我们平时不常用的压缩包，dba-v1.0.4-linux-amd64.tar.gz 就是其中之一，解压后无需编译可以直接得到 tidba 这个可执行文件，使用非常方便。
[root@cctest tidb-community-toolkit-v7.1.0-linux-amd64]# ls
1535.dmctl.json 3729.prometheus.json dba-v1.0.4-linux-amd64.tar.gz pump-v7.1.0-linux-amd64.tar.gz
1542.dm-worker.json 3752.cdc.json dmctl-v7.1.0-linux-amd64.tar.gz reparo
1546.dm-master.json 3.package.json dm-master-v7.1.0-linux-amd64.tar.gz root.json
17.tispark.json 3.PCC.json dm-v1.12.2-linux-amd64.tar.gz server-v1.12.2-linux-amd64.tar.gz
1.dba.json 487.bench.json dm-worker-v7.1.0-linux-amd64.tar.gz snapshot.json
1.index.json 5.alertmanager.json drainer-v7.1.0-linux-amd64.tar.gz spark-v2.4.3-any-any.tar.gz
1.root.json 7.blackbox_exporter.json dumpling-v7.1.0-linux-amd64.tar.gz sync_diff_inspector
239.dm.json 7.node_exporter.json errdoc-v4.0.7-linux-amd64.tar.gz tidba
241.server.json 80.tikv-importer.json etcdctl tidb-lightning-ctl
281.tiup.json 9.errdoc.json grafana-v7.1.0-linux-amd64.tar.gz tidb-lightning-v7.1.0-linux-amd64.tar.gz
2.spark.json alertmanager-v0.17.0-linux-amd64.tar.gz keys tikv-importer-v4.0.2-linux-amd64.tar.gz
3541.drainer.json arbiter local_install.sh timestamp.json
3562.pump.json bench-v1.12.0-linux-amd64.tar.gz mydumper tispark-v2.4.1-any-any.tar.gz
3569.pd-recover.json bin node_exporter-v1.3.1-linux-amd64.tar.gz tiup-linux-amd64.tar.gz
3586.dumpling.json binlogctl package-v0.0.9-linux-amd64.tar.gz tiup-v1.12.2-linux-amd64.tar.gz
3657.tidb-lightning.json blackbox_exporter-v0.21.1-linux-amd64.tar.gz PCC-1.0.1-linux-amd64.tar.gz
3660.br.json br-v7.1.0-linux-amd64.tar.gz pd-recover-v7.1.0-linux-amd64.tar.gz
3678.grafana.json cdc-v7.1.0-linux-amd64.tar.gz prometheus-v7.1.0-linux-amd64.tar.gz
三.环境准备
本次测试仅用于参数对比，因此采用最简单省时的 tiup playground 来创建集群。
准备 2 套集群，源端为 v6.1.6
[root@cctest ~]# tiup playground v6.1.6
tiup is checking updates for component playground ...
Starting component `playground`: /root/.tiup/components/playground/v1.12.3/tiup-playground v6.1.6
Start pd instance:v6.1.6
Start tikv instance:v6.1.6
Start tidb instance:v6.1.6
Waiting for tidb instances ready
127.0.0.1:42240 ... Done
Start tiflash instance:v6.1.6
Waiting for tiflash instances ready
127.0.0.1:3930 ... Done
🎉 TiDB Playground Cluster is started, enjoy!
Connect TiDB: mysql --comments --host 127.0.0.1 --port 42240 -u root
TiDB Dashboard: http://127.0.0.1:38754/dashboard
Grafana: http://127.0.0.1:3000
目标端为 7.1.0
[root@cctest ~]# tiup playground v7.1.0
tiup is checking updates for component playground ...
Starting component playground: /root/.tiup/components/playground/v1.12.3/tiup-playground v7.1.0
Start pd instance:v7.1.0
Start tikv instance:v7.1.0
Start tidb instance:v7.1.0
Waiting for tidb instances ready
127.0.0.1:4000 ... Done
Start tiflash instance:v7.1.0
Waiting for tiflash instances ready
127.0.0.1:41072 ... Done
🎉 TiDB Playground Cluster is started, enjoy!
Connect TiDB: mysql --comments --host 127.0.0.1 --port 4000 -u root
TiDB Dashboard: http://127.0.0.1:2379/dashboard
Grafana: http://127.0.0.1:43315
四.参数对比测试
1.tidb 系统变量对比
tidba diff tidb --diff-type variable --base-tidb-addr 127.0.0.1:42240 --base-tidb-user root --new-tidb-addr 127.0.0.1:4000 --new-tidb-user root
[root@cctest tidb-community-toolkit-v7.1.0-linux-amd64]# ./tidba diff tidb --diff-type variable --base-tidb-addr 127.0.0.1:42240 --base-tidb-user root --new-tidb-addr 127.0.0.1:4000 --new-tidb-user root
--- 127.0.0.1:42240
+++ 127.0.0.1:4000
{
"tidb_allow_batch_cop": "1",
"tidb_allow_fallback_to_tikv": "",
- "tidb_allow_function_for_expression_index": "lower, md5, reverse, tidb_shard, upper, vitess_hash",
+ "tidb_allow_function_for_expression_index": "json_array, json_array_append, json_array_insert, json_contains, json_contains_path, json_depth, json_extract, json_insert, json_keys, json_length, json_merge_patch, json_merge_preserve, json_object, json_pretty, json_quote, json_remove, json_replace, json_search, json_set, json_storage_size, json_type, json_unquote, json_valid, lower, md5, reverse, tidb_shard, upper, vitess_hash",
"tidb_allow_mpp": "ON",
"tidb_allow_remove_auto_inc": "OFF",
"tidb_analyze_version": "2",
"tidb_auto_analyze_end_time": "23:59 +0000",
"tidb_auto_analyze_ratio": "0.5",
"tidb_auto_analyze_start_time": "00:00 +0000",
"tidb_backoff_lock_fast": "10",
"tidb_backoff_weight": "2",
"tidb_batch_commit": "OFF",
"tidb_batch_delete": "OFF",
"tidb_batch_insert": "OFF",
"tidb_batch_pending_tiflash_count": "4000",
"tidb_broadcast_join_threshold_count": "10240",
"tidb_broadcast_join_threshold_size": "104857600",
"tidb_build_stats_concurrency": "4",
"tidb_capture_plan_baselines": "OFF",
"tidb_check_mb4_value_in_utf8": "ON",
"tidb_checksum_table_concurrency": "4",
"tidb_committer_concurrency": "128",
"tidb_constraint_check_in_place": "OFF",
- "tidb_current_ts": "442484779120328706",
+ "tidb_current_ts": "442484779124785154",
"tidb_ddl_error_count_limit": "512",
"tidb_ddl_reorg_batch_size": "256",
"tidb_ddl_reorg_priority": "PRIORITY_LOW",
"tidb_ddl_reorg_worker_cnt": "4",
"tidb_disable_txn_auto_retry": "ON",
"tidb_distsql_scan_concurrency": "15",
"tidb_dml_batch_size": "0",
"tidb_enable_1pc": "ON",
- "tidb_enable_amend_pessimistic_txn": "OFF",
"tidb_enable_analyze_snapshot": "OFF",
"tidb_enable_async_commit": "ON",
"tidb_enable_auto_analyze": "ON",
"tidb_enable_auto_increment_in_generated": "OFF",
"tidb_enable_batch_dml": "OFF",
"tidb_enable_cascades_planner": "OFF",
- "tidb_enable_change_multi_schema": "OFF",
- "tidb_enable_chunk_rpc": "1",
+ "tidb_enable_chunk_rpc": "ON",
- "tidb_enable_clustered_index": "INT_ONLY",
+ "tidb_enable_clustered_index": "ON",
"tidb_enable_collect_execution_info": "ON",
"tidb_enable_column_tracking": "OFF",
"tidb_enable_enhanced_security": "OFF",
- "tidb_enable_exchange_partition": "OFF",
+ "tidb_enable_exchange_partition": "ON",
"tidb_enable_extended_stats": "OFF",
"tidb_enable_fast_analyze": "OFF",
- "tidb_enable_historical_stats": "OFF",
+ "tidb_enable_historical_stats": "ON",
"tidb_enable_index_merge": "ON",
"tidb_enable_index_merge_join": "OFF",
"tidb_enable_inl_join_inner_multi_pattern": "OFF",
"tidb_enable_legacy_instance_scope": "ON",
"tidb_enable_list_partition": "ON",
"tidb_enable_local_txn": "OFF",
"tidb_enable_mutation_checker": "ON",
- "tidb_enable_new_cost_interface": "OFF",
+ "tidb_enable_new_cost_interface": "ON",
"tidb_enable_new_only_full_group_by_check": "OFF",
"tidb_enable_noop_functions": "OFF",
"tidb_enable_ordered_result_mode": "OFF",
- "tidb_enable_outer_join_reorder": "OFF",
+ "tidb_enable_outer_join_reorder": "ON",
- "tidb_enable_paging": "OFF",
+ "tidb_enable_paging": "ON",
"tidb_enable_parallel_apply": "OFF",
"tidb_enable_pipelined_window_function": "ON",
- "tidb_enable_point_get_cache": "OFF",
"tidb_enable_prepared_plan_cache": "ON",
- "tidb_enable_pseudo_for_outdated_stats": "ON",
+ "tidb_enable_pseudo_for_outdated_stats": "OFF",
- "tidb_enable_rate_limit_action": "ON",
+ "tidb_enable_rate_limit_action": "OFF",
"tidb_enable_slow_log": "ON",
"tidb_enable_stmt_summary": "ON",
"tidb_enable_strict_double_type_check": "ON",
"tidb_enable_table_partition": "ON",
"tidb_enable_telemetry": "OFF",
"tidb_enable_top_sql": "OFF",
"tidb_enable_tso_follower_proxy": "OFF",
"tidb_enable_vectorized_expression": "ON",
"tidb_enable_window_function": "ON",
"tidb_enforce_mpp": "OFF",
"tidb_evolve_plan_baselines": "OFF",
"tidb_evolve_plan_task_end_time": "23:59 +0000",
"tidb_evolve_plan_task_max_time": "600",
"tidb_evolve_plan_task_start_time": "00:00 +0000",
"tidb_executor_concurrency": "5",
"tidb_expensive_query_time_threshold": "60",
"tidb_force_priority": "NO_PRIORITY",
"tidb_gc_concurrency": "-1",
"tidb_gc_enable": "ON",
"tidb_gc_life_time": "10m0s",
"tidb_gc_max_wait_time": "86400",
"tidb_gc_run_interval": "10m0s",
"tidb_gc_scan_lock_mode": "LEGACY",
"tidb_general_log": "OFF",
"tidb_guarantee_linearizability": "ON",
"tidb_hash_exchange_with_new_collation": "ON",
"tidb_hash_join_concurrency": "-1",
"tidb_hashagg_final_concurrency": "-1",
"tidb_hashagg_partial_concurrency": "-1",
"tidb_ignore_prepared_cache_close_stmt": "OFF",
"tidb_index_join_batch_size": "25000",
"tidb_index_lookup_concurrency": "-1",
"tidb_index_lookup_join_concurrency": "-1",
"tidb_index_lookup_size": "20000",
"tidb_index_serial_scan_concurrency": "1",
"tidb_init_chunk_size": "32",
"tidb_isolation_read_engines": "tikv,tiflash,tidb",
"tidb_last_ddl_info": "{"query":"","seq_num":0}",
- "tidb_last_query_info": "{"txn_scope":"global","start_ts":442484779120328706,"for_update_ts":442484779120328706}",
+ "tidb_last_query_info": "{"txn_scope":"global","start_ts":442484779124785154,"for_update_ts":442484779124785154}",
"tidb_last_txn_info": "",
"tidb_log_file_max_days": "0",
"tidb_low_resolution_tso": "OFF",
"tidb_max_auto_analyze_time": "43200",
"tidb_max_chunk_size": "1024",
"tidb_max_delta_schema_count": "1024",
"tidb_max_tiflash_threads": "-1",
"tidb_mem_oom_action": "CANCEL",
"tidb_mem_quota_analyze": "-1",
"tidb_mem_quota_apply_cache": "33554432",
"tidb_mem_quota_binding_cache": "67108864",
"tidb_mem_quota_query": "1073741824",
- "tidb_memory_usage_alarm_ratio": "0.8",
+ "tidb_memory_usage_alarm_ratio": "0.7",
"tidb_merge_join_concurrency": "1",
"tidb_metric_query_range_duration": "60",
"tidb_metric_query_step": "60",
"tidb_mpp_store_fail_ttl": "60s",
"tidb_multi_statement_mode": "OFF",
"tidb_nontransactional_ignore_error": "OFF",
"tidb_opt_agg_push_down": "OFF",
"tidb_opt_broadcast_cartesian_join": "1",
"tidb_opt_concurrency_factor": "3",
"tidb_opt_copcpu_factor": "3",
"tidb_opt_correlation_exp_factor": "1",
"tidb_opt_correlation_threshold": "0.9",
"tidb_opt_cpu_factor": "3",
"tidb_opt_desc_factor": "3",
"tidb_opt_disk_factor": "1.5",
"tidb_opt_distinct_agg_push_down": "OFF",
"tidb_opt_enable_correlation_adjustment": "ON",
"tidb_opt_insubq_to_join_and_agg": "ON",
"tidb_opt_join_reorder_threshold": "0",
"tidb_opt_limit_push_down_threshold": "100",
"tidb_opt_memory_factor": "0.001",
"tidb_opt_mpp_outer_join_fixed_build_side": "OFF",
"tidb_opt_network_factor": "1",
"tidb_opt_prefer_range_scan": "OFF",
"tidb_opt_projection_push_down": "OFF",
"tidb_opt_scan_factor": "1.5",
"tidb_opt_seek_factor": "20",
"tidb_opt_tiflash_concurrency_factor": "24",
"tidb_opt_write_row_id": "OFF",
"tidb_optimizer_selectivity_level": "0",
- "tidb_partition_prune_mode": "static",
+ "tidb_partition_prune_mode": "dynamic",
"tidb_persist_analyze_options": "ON",
"tidb_placement_mode": "STRICT",
"tidb_pprof_sql_cpu": "0",
"tidb_prepared_plan_cache_memory_guard_ratio": "0.1",
"tidb_prepared_plan_cache_size": "100",
"tidb_projection_concurrency": "-1",
"tidb_query_log_max_len": "4096",
"tidb_rc_read_check_ts": "OFF",
"tidb_read_consistency": "strict",
"tidb_read_staleness": "0",
"tidb_record_plan_in_slow_log": "ON",
"tidb_redact_log": "OFF",
"tidb_regard_null_as_point": "ON",
"tidb_remove_orderby_in_subquery": "OFF",
"tidb_replica_read": "leader",
"tidb_restricted_read_only": "OFF",
"tidb_retry_limit": "10",
"tidb_row_format_version": "2",
"tidb_scatter_region": "OFF",
"tidb_shard_allocate_step": "9223372036854775807",
"tidb_skip_ascii_check": "OFF",
"tidb_skip_isolation_level_check": "OFF",
"tidb_skip_utf8_check": "OFF",
"tidb_slow_log_threshold": "300",
"tidb_slow_query_file": "tidb-slow.log",
"tidb_snapshot": "",
"tidb_stats_cache_mem_quota": "0",
- "tidb_stats_load_pseudo_timeout": "false",
+ "tidb_stats_load_pseudo_timeout": "ON",
- "tidb_stats_load_sync_wait": "0",
+ "tidb_stats_load_sync_wait": "100",
"tidb_stmt_summary_history_size": "24",
"tidb_stmt_summary_internal_query": "OFF",
"tidb_stmt_summary_max_sql_length": "4096",
"tidb_stmt_summary_max_stmt_count": "3000",
"tidb_stmt_summary_refresh_interval": "1800",
"tidb_store_limit": "0",
"tidb_streamagg_concurrency": "1",
"tidb_super_read_only": "OFF",
"tidb_sysdate_is_now": "OFF",
"tidb_table_cache_lease": "3",
"tidb_tmp_table_max_size": "67108864",
"tidb_top_sql_max_meta_count": "5000",
"tidb_top_sql_max_time_series_count": "100",
"tidb_track_aggregate_memory_usage": "ON",
"tidb_tso_client_batch_max_wait_time": "0",
"tidb_txn_assertion_level": "FAST",
"tidb_txn_commit_batch_size": "16384",
"tidb_txn_mode": "pessimistic",
"tidb_use_plan_baselines": "ON",
"tidb_wait_split_region_finish": "ON",
"tidb_wait_split_region_timeout": "300",
"tidb_window_concurrency": "-1"
+ "tidb_adaptive_closest_read_threshold": "4096"
+ "tidb_analyze_partition_concurrency": "1"
+ "tidb_auto_analyze_partition_batch_size": "1"
+ "tidb_auto_build_stats_concurrency": "1"
+ "tidb_cdc_write_source": "0"
+ "tidb_constraint_check_in_place_pessimistic": "ON"
+ "tidb_cost_model_version": "2"
+ "tidb_ddl_disk_quota": "107374182400"
+ "tidb_ddl_enable_fast_reorg": "ON"
+ "tidb_ddl_flashback_concurrency": "64"
+ "tidb_default_string_match_selectivity": "0.8"
+ "tidb_enable_ddl": "ON"
+ "tidb_enable_dist_task": "OFF"
+ "tidb_enable_external_ts_read": "OFF"
+ "tidb_enable_foreign_key": "ON"
+ "tidb_enable_gc_aware_memory_track": "OFF"
+ "tidb_enable_gogc_tuner": "ON"
+ "tidb_enable_historical_stats_for_capture": "OFF"
+ "tidb_enable_metadata_lock": "ON"
+ "tidb_enable_non_prepared_plan_cache": "OFF"
+ "tidb_enable_non_prepared_plan_cache_for_dml": "OFF"
+ "tidb_enable_noop_variables": "ON"
+ "tidb_enable_null_aware_anti_join": "ON"
+ "tidb_enable_plan_cache_for_param_limit": "ON"
+ "tidb_enable_plan_cache_for_subquery": "ON"
+ "tidb_enable_plan_replayer_capture": "ON"
+ "tidb_enable_plan_replayer_continuous_capture": "OFF"
+ "tidb_enable_prepared_plan_cache_memory_monitor": "ON"
+ "tidb_enable_resource_control": "ON"
+ "tidb_enable_reuse_chunk": "ON"
+ "tidb_enable_row_level_checksum": "OFF"
+ "tidb_enable_tiflash_read_for_write_stmt": "ON"
+ "tidb_enable_tmp_storage_on_oom": "ON"
+ "tidb_enable_unsafe_substitute": "OFF"
+ "tidb_external_ts": "0"
+ "tidb_generate_binary_plan": "ON"
+ "tidb_gogc_tuner_threshold": "0.6"
+ "tidb_historical_stats_duration": "168h0m0s"
+ "tidb_index_join_double_read_penalty_cost_rate": "0"
+ "tidb_index_merge_intersection_concurrency": "-1"
+ "tidb_last_plan_replayer_token": ""
+ "tidb_load_based_replica_read_threshold": "1s"
+ "tidb_max_bytes_before_tiflash_external_group_by": "-1"
+ "tidb_max_bytes_before_tiflash_external_join": "-1"
+ "tidb_max_bytes_before_tiflash_external_sort": "-1"
+ "tidb_max_paging_size": "50000"
+ "tidb_memory_debug_mode_alarm_ratio": "0"
+ "tidb_memory_debug_mode_min_heap_inuse": "0"
+ "tidb_memory_usage_alarm_keep_record_num": "5"
+ "tidb_merge_partition_stats_concurrency": "1"
+ "tidb_min_paging_size": "128"
+ "tidb_non_prepared_plan_cache_size": "100"
+ "tidb_opt_advanced_join_hint": "ON"
+ "tidb_opt_derive_topn": "OFF"
+ "tidb_opt_enable_late_materialization": "ON"
+ "tidb_opt_enable_three_stage_multi_distinct_agg": "OFF"
+ "tidb_opt_fix_control": ""
+ "tidb_opt_force_inline_cte": "OFF"
+ "tidb_opt_ordering_index_selectivity_threshold": "0"
+ "tidb_opt_prefix_index_single_scan": "ON"
+ "tidb_opt_range_max_size": "67108864"
+ "tidb_opt_skew_distinct_agg": "OFF"
+ "tidb_opt_three_stage_distinct_agg": "ON"
+ "tidb_pessimistic_txn_fair_locking": "ON"
+ "tidb_plan_cache_invalidation_on_fresh_stats": "ON"
+ "tidb_plan_cache_max_plan_size": "2097152"
+ "tidb_prefer_broadcast_join_by_exchange_data_size": "OFF"
+ "tidb_rc_write_check_ts": "OFF"
+ "tidb_server_memory_limit": "80%"
+ "tidb_server_memory_limit_gc_trigger": "0.7"
+ "tidb_server_memory_limit_sess_min_size": "134217728"
+ "tidb_session_plan_cache_size": "100"
+ "tidb_simplified_metrics": "OFF"
+ "tidb_slow_txn_log_threshold": "0"
+ "tidb_source_id": "1"
+ "tidb_stmt_summary_enable_persistent": "OFF"
+ "tidb_stmt_summary_file_max_backups": "0"
+ "tidb_stmt_summary_file_max_days": "3"
+ "tidb_stmt_summary_file_max_size": "64"
+ "tidb_stmt_summary_filename": "tidb-statements.log"
+ "tidb_store_batch_size": "4"
+ "tidb_sysproc_scan_concurrency": "1"
+ "tidb_ttl_delete_batch_size": "100"
+ "tidb_ttl_delete_rate_limit": "0"
+ "tidb_ttl_delete_worker_count": "4"
+ "tidb_ttl_job_enable": "ON"
+ "tidb_ttl_job_schedule_window_end_time": "23:59 +0000"
+ "tidb_ttl_job_schedule_window_start_time": "00:00 +0000"
+ "tidb_ttl_running_tasks": "-1"
+ "tidb_ttl_scan_batch_size": "500"
+ "tidb_ttl_scan_worker_count": "4"
}
2.tidb config 对比
tidba diff tidb --diff-type config --base-tidb-addr 127.0.0.1:42240 --base-tidb-user root --new-tidb-addr 127.0.0.1:4000 --new-tidb-user root
[root@cctest tidb-community-toolkit-v7.1.0-linux-amd64]# ./tidba diff tidb --diff-type config --base-tidb-addr 127.0.0.1:42240 --base-tidb-user root --new-tidb-addr 127.0.0.1:4000 --new-tidb-user root
--- 127.0.0.1:42240
+++ 127.0.0.1:4000
{
"advertise-address": "127.0.0.1",
"alter-primary-key": false,
"ballast-object-size": 0,
"binlog": {
"binlog-socket": "",
"enable": false,
"ignore-error": false,
"strategy": "range",
"write-timeout": "15s"
},
- "check-mb4-value-in-utf8": true,
"compatible-kill-query": false,
"cors": "",
"delay-clean-table-lock": 0,
"deprecate-integer-display-length": false,
- "enable-batch-dml": false,
- "enable-collect-execution-info": true,
"enable-enum-length-limit": true,
"enable-forwarding": false,
"enable-global-index": false,
"enable-global-kill": true,
"enable-table-lock": false,
"enable-tcp4-only": false,
"enable-telemetry": false,
"experimental": {
"allow-expression-index": false
},
"graceful-wait-before-shutdown": 0,
"host": "127.0.0.1",
"index-limit": 64,
"instance": {
"ddl_slow_threshold": 300,
"plugin_dir": "/data/deploy/plugin",
"plugin_load": "",
"tidb_check_mb4_value_in_utf8": true,
"tidb_enable_collect_execution_info": true,
"tidb_enable_slow_log": true,
"tidb_expensive_query_time_threshold": 60,
"tidb_force_priority": "NO_PRIORITY",
"tidb_general_log": false,
- "tidb_memory_usage_alarm_ratio": 0.8,
"tidb_pprof_sql_cpu": false,
"tidb_record_plan_in_slow_log": 1,
"tidb_slow_log_threshold": 300
+ "max_connections": 0
+ "tidb_enable_ddl": true
+ "tidb_rc_read_check_ts": false
+ "tidb_stmt_summary_enable_persistent": false
+ "tidb_stmt_summary_file_max_backups": 0
+ "tidb_stmt_summary_file_max_days": 3
+ "tidb_stmt_summary_file_max_size": 64
+ "tidb_stmt_summary_filename": "tidb-statements.log"
},
"isolation-read": {
"engines": [
: "tikv",
: "tiflash",
: "tidb"
]
},
"labels": {
},
"lease": "45s",
"log": {
"disable-error-stack": null,
"disable-timestamp": null,
"enable-error-stack": null,
- "enable-slow-log": true,
"enable-timestamp": null,
- "expensive-threshold": 10000,
"file": {
- "filename": "/root/.tiup/data/TiSzfi6/tidb-0/tidb.log",
+ "filename": "/root/.tiup/data/TiSzmGl/tidb-0/tidb.log",
"max-backups": 0,
"max-days": 0,
"max-size": 300
},
"format": "text",
"level": "info",
- "query-log-max-len": 4096,
- "record-plan-in-slow-log": 1,
"slow-query-file": "tidb-slow.log",
- "slow-threshold": 300
+ "timeout": 0
},
"max-ballast-object-size": 0,
"max-index-length": 3072,
- "max-server-connections": 0,
- "mem-quota-query": 1.073741824e+09,
"new_collations_enabled_on_first_bootstrap": true,
- "oom-action": "cancel",
- "oom-use-tmp-storage": true,
"opentracing": {
"enable": false,
"reporter": {
"buffer-flush-interval": 0,
"local-agent-host-port": "",
"log-spans": false,
"queue-size": 0
},
"rpc-metrics": false,
"sampler": {
"max-operations": 0,
"param": 1,
"sampling-refresh-interval": 0,
"sampling-server-url": "",
"type": "const"
}
},
- "path": "127.0.0.1:38754",
+ "path": "127.0.0.1:2379",
"pd-client": {
"pd-server-timeout": 3
},
"performance": {
"bind-info-lease": "3s",
- "committer-concurrency": 128,
"cross-join": true,
"distinct-agg-push-down": false,
"enable-stats-cache-mem-quota": false,
"enforce-mpp": false,
- "feedback-probability": 0,
- "force-priority": "NO_PRIORITY",
"gogc": 100,
- "max-memory": 0,
"max-procs": 0,
"max-txn-ttl": 3.6e+06,
- "memory-usage-alarm-ratio": 0.8,
"plan-replayer-gc-lease": "10m",
"projection-push-down": false,
"pseudo-estimate-ratio": 0.8,
- "query-feedback-limit": 512,
- "run-auto-analyze": true,
"server-memory-quota": 0,
"stats-lease": "3s",
"stats-load-concurrency": 5,
"stats-load-queue-size": 1000,
"stmt-count-limit": 5000,
"tcp-keep-alive": true,
"tcp-no-delay": true,
"txn-entry-size-limit": 6.291456e+06,
"txn-total-size-limit": 1.048576e+08
+ "analyze-partition-concurrency-quota": 16
+ "enable-load-fmsketch": false
+ "force-init-stats": false
+ "lite-init-stats": false
+ "plan-replayer-dump-worker-concurrency": 1
},
"pessimistic-txn": {
"deadlock-history-capacity": 10,
"deadlock-history-collect-retryable": false,
"max-retry-count": 256,
"pessimistic-auto-commit": false
+ "constraint-check-in-place-pessimistic": true
},
"plugin": {
- "dir": "/data/deploy/plugin",
- "load": ""
},
- "port": 42240,
+ "port": 4000,
"prepared-plan-cache": {
- "capacity": 100,
- "enabled": true,
- "memory-guard-ratio": 0.1
},
"proxy-protocol": {
"header-timeout": 5,
"networks": ""
+ "fallbackable": false
},
"repair-mode": false,
"repair-table-list": [
],
- "run-ddl": true,
"security": {
"auto-tls": true,
"cluster-ssl-ca": "",
"cluster-ssl-cert": "",
"cluster-ssl-key": "",
"cluster-verify-cn": null,
"enable-sem": false,
"rsa-key-size": 4096,
"secure-bootstrap": false,
"skip-grant-table": false,
"spilled-file-encryption-method": "plaintext",
"ssl-ca": "",
"ssl-cert": "",
"ssl-key": "",
"tls-version": ""
+ "auth-token-jwks": ""
+ "auth-token-refresh-interval": "1h0m0s"
+ "disconnect-on-expired-password": true
+ "session-token-signing-cert": ""
+ "session-token-signing-key": ""
},
"server-version": "",
"skip-register-to-dashboard": false,
- "socket": "/tmp/tidb-42240.sock",
+ "socket": "/tmp/tidb-4000.sock",
"split-region-max-num": 1000,
"split-table": true,
"status": {
"grpc-concurrent-streams": 1024,
"grpc-initial-window-size": 2.097152e+06,
"grpc-keepalive-time": 10,
"grpc-keepalive-timeout": 3,
"grpc-max-send-msg-size": 2.147483647e+09,
"metrics-addr": "",
"metrics-interval": 15,
"record-db-qps": false,
"report-status": true,
"status-host": "0.0.0.0",
- "status-port": 42773
+ "status-port": 10080
+ "record-db-label": false
},
"store": "tikv",
"stores-refresh-interval": 60,
"table-column-count-limit": 1017,
"tidb-edition": "",
"tidb-release-version": "",
"tikv-client": {
"async-commit": {
"allowed-clock-drift": 5e+08,
"keys-limit": 256,
"safe-window": 2e+09,
"total-key-size-limit": 4096
},
"batch-wait-size": 8,
"commit-timeout": "41s",
"copr-cache": {
"capacity-mb": 1000
},
"enable-chunk-rpc": true,
"grpc-compression-type": "none",
"grpc-connection-count": 4,
"grpc-keepalive-time": 10,
"grpc-keepalive-timeout": 3,
"max-batch-size": 128,
"max-batch-wait-time": 0,
"overload-threshold": 200,
"region-cache-ttl": 600,
"resolve-lock-lite-threshold": 16,
"store-limit": 0,
"store-liveness-timeout": "1s",
"ttl-refreshed-txn-size": 3.3554432e+07
},
- "tmp-storage-path": "/tmp/0_tidb/MTI3LjAuMC4xOjQyMjQwLzAuMC4wLjA6NDI3NzM=/tmp-storage",
+ "tmp-storage-path": "/tmp/0_tidb/MTI3LjAuMC4xOjQwMDAvMC4wLjAuMDoxMDA4MA==/tmp-storage",
"tmp-storage-quota": -1,
"token-limit": 1000,
"top-sql": {
"receiver-address": ""
},
"treat-old-version-utf8-as-utf8mb4": true,
"version-comment": ""
+ "autoscaler-addr": "tiflash-autoscale-lb.tiflash-autoscale.svc.cluster.local:8081"
+ "autoscaler-cluster-id": ""
+ "autoscaler-type": "aws"
+ "disaggregated-tiflash": false
+ "initialize-sql-file": ""
+ "is-tiflashcompute-fixed-pool": false
+ "keyspace-name": ""
+ "temp-dir": "/tmp/tidb"
+ "tidb-enable-exit-check": false
+ "tidb-max-reuse-chunk": 64
+ "tidb-max-reuse-column": 256
+ "transaction-summary": {
+ "transaction-id-digest-min-duration": 2.147483647e+09,
+ "transaction-summary-capacity": 500
+ }
+ "use-autoscaler": false
}
3.pd 参数对比
tidba diff pd --base-pd-addr 127.0.0.1:38754 --new-pd-addr 127.0.0.1:2379
[root@cctest tidb-community-toolkit-v7.1.0-linux-amd64]# ./tidba diff pd --base-pd-addr 127.0.0.1:38754 --new-pd-addr 127.0.0.1:2379
--- 127.0.0.1:38754
+++ 127.0.0.1:2379
{
"DisableStrictReconfigCheck": false,
"ElectionInterval": "3s",
"HeartbeatStreamBindInterval": "1m0s",
"LeaderPriorityCheckInterval": "1m0s",
"PreVote": true,
"TickInterval": "500ms",
"WarningMsgs": null,
- "advertise-client-urls": "http://127.0.0.1:38754",
+ "advertise-client-urls": "http://127.0.0.1:2379",
- "advertise-peer-urls": "http://127.0.0.1:35297",
+ "advertise-peer-urls": "http://127.0.0.1:2380",
"auto-compaction-mode": "periodic",
"auto-compaction-retention-v2": "1h",
- "client-urls": "http://127.0.0.1:38754",
+ "client-urls": "http://127.0.0.1:2379",
- "cluster-version": "6.1.6",
+ "cluster-version": "7.1.0",
"dashboard": {
"enable-experimental": false,
"enable-telemetry": false,
"internal-proxy": false,
"public-path-prefix": "",
"tidb-cacert-path": "",
"tidb-cert-path": "",
"tidb-key-path": ""
},
- "data-dir": "/root/.tiup/data/TiSzfi6/pd-0/data",
+ "data-dir": "/root/.tiup/data/TiSzmGl/pd-0/data",
"enable-grpc-gateway": true,
"enable-local-tso": false,
"force-new-cluster": false,
- "initial-cluster": "pd-0=http://127.0.0.1:35297",
+ "initial-cluster": "pd-0=http://127.0.0.1:2380",
"initial-cluster-state": "new",
"initial-cluster-token": "pd-cluster",
"join": "",
"label-property": {
},
"labels": {
},
"lease": 3,
"log": {
"development": false,
"disable-caller": false,
"disable-error-verbose": true,
"disable-stacktrace": false,
"disable-timestamp": false,
"file": {
- "filename": "/root/.tiup/data/TiSzfi6/pd-0/pd.log",
+ "filename": "/root/.tiup/data/TiSzmGl/pd-0/pd.log",
"max-backups": 0,
"max-days": 0,
- "max-size": 300
+ "max-size": 0
},
"format": "text",
"level": "info",
"sampling": null
+ "error-output-path": ""
},
"max-request-bytes": 1.572864e+08,
"metric": {
"address": "",
"interval": "15s",
"job": "pd-0"
},
"name": "pd-0",
"pd-server": {
- "dashboard-address": "http://127.0.0.1:38754",
+ "dashboard-address": "http://127.0.0.1:2379",
"flow-round-by-digit": 3,
"key-type": "table",
"max-gap-reset-ts": "24h0m0s",
"metric-storage": "",
- "min-resolved-ts-persistence-interval": "0s",
+ "min-resolved-ts-persistence-interval": "1s",
"runtime-services": "",
"use-region-storage": "true"
+ "enable-gogc-tuner": "false"
+ "gc-tuner-threshold": 0.6
+ "server-memory-limit": 0
+ "server-memory-limit-gc-trigger": 0.7
+ "trace-region-flow": "true"
},
- "peer-urls": "http://127.0.0.1:35297",
+ "peer-urls": "http://127.0.0.1:2380",
"quota-backend-bytes": "8GiB",
"replication": {
"enable-placement-rules": "true",
"enable-placement-rules-cache": "false",
"isolation-level": "",
"location-labels": "",
- "max-replicas": 1,
+ "max-replicas": 3,
"strictly-match-label": "false"
},
"replication-mode": {
"dr-auto-sync": {
"dr": "",
"dr-replicas": 0,
"label-key": "",
"pause-region-split": "false",
"primary": "",
"primary-replicas": 0,
- "wait-async-timeout": "2m0s",
"wait-store-timeout": "1m0s",
- "wait-sync-timeout": "1m0s"
},
"replication-mode": "majority"
},
"schedule": {
"enable-cross-table-merge": "true",
"enable-debug-metrics": "false",
"enable-joint-consensus": "true",
"enable-location-replacement": "true",
"enable-make-up-replica": "true",
"enable-one-way-merge": "false",
"enable-remove-down-replica": "true",
"enable-remove-extra-replica": "true",
"enable-replace-offline-replica": "true",
"high-space-ratio": 0.7,
"hot-region-cache-hits-threshold": 3,
"hot-region-schedule-limit": 4,
"hot-regions-reserved-days": 7,
"hot-regions-write-interval": "10m0s",
"leader-schedule-limit": 4,
"leader-schedule-policy": "count",
- "low-space-ratio": 0.99,
+ "low-space-ratio": 0.8,
"max-merge-region-keys": 200000,
"max-merge-region-size": 20,
"max-pending-peer-count": 64,
"max-snapshot-count": 64,
"max-store-down-time": "30m0s",
"max-store-preparing-time": "48h0m0s",
"merge-schedule-limit": 8,
"patrol-region-interval": "10ms",
"region-schedule-limit": 2048,
"region-score-formula-version": "v2",
"replica-schedule-limit": 64,
"scheduler-max-waiting-operator": 5,
"schedulers-payload": {
- "balance-hot-region-scheduler": null,
+ "balance-hot-region-scheduler": {
+ "byte-rate-rank-step-ratio": 0.05,
+ "count-rank-step-ratio": 0.01,
+ "dst-tolerance-ratio": 1.05,
+ "enable-for-tiflash": "true",
+ "forbid-rw-type": "none",
+ "great-dec-ratio": 0.95,
+ "key-rate-rank-step-ratio": 0.05,
+ "max-peer-number": 1000,
+ "max-zombie-rounds": 3,
+ "min-hot-byte-rate": 100,
+ "min-hot-key-rate": 10,
+ "min-hot-query-rate": 10,
+ "minor-dec-ratio": 0.99,
+ "query-rate-rank-step-ratio": 0.05,
+ "rank-formula-version": "v2",
+ "read-priorities": [
+ : "query",
+ : "byte"
+ ],
+ "src-tolerance-ratio": 1.05,
+ "strict-picking-store": "true",
+ "write-leader-priorities": [
+ : "query",
+ : "byte"
+ ],
+ "write-peer-priorities": [
+ : "byte",
+ : "key"
+ ]
+ },
"balance-leader-scheduler": {
"batch": 4,
"ranges": [
: {
"end-key": "",
"start-key": ""
}
]
},
"balance-region-scheduler": {
"name": "balance-region-scheduler",
"ranges": [
: {
"end-key": "",
"start-key": ""
}
]
},
- "split-bucket-scheduler": null
+ "balance-witness-scheduler": {
+ "batch": 4,
+ "ranges": [
+ : {
+ "end-key": "",
+ "start-key": ""
+ }
+ ]
+ }
+ "transfer-witness-leader-scheduler": null
},
"schedulers-v2": [
0: {
"args": null,
"args-payload": "",
"disable": false,
"type": "balance-region"
},
1: {
"args": null,
"args-payload": "",
"disable": false,
"type": "balance-leader"
},
+ 2: {
+ "args": null,
+ "args-payload": "",
+ "disable": false,
+ "type": "balance-witness"
+ },
3: {
"args": null,
"args-payload": "",
"disable": false,
"type": "split-bucket"
},
],
"split-merge-interval": "1h0m0s",
"store-limit": {
"1": {
"add-peer": 15,
"remove-peer": 15
},
- "68": {
- "add-peer": 30,
- "remove-peer": 30
- }
},
"store-limit-mode": "manual",
"tolerant-size-ratio": 0
+ "enable-diagnostic": "true"
+ "enable-tikv-split-region": "true"
+ "enable-witness": "false"
+ "slow-store-evicting-affected-store-ratio-threshold": 0.3
+ "store-limit-version": "v1"
+ "swtich-witness-interval": "1h0m0s"
+ "witness-schedule-limit": 4
},
"security": {
"SSLCABytes": null,
"SSLCertBytes": null,
"SSLKEYBytes": null,
"cacert-path": "",
"cert-allowed-cn": null,
"cert-path": "",
"encryption": {
"data-encryption-method": "plaintext",
"data-key-rotation-period": "168h0m0s",
"master-key": {
"endpoint": "",
"key-id": "",
"path": "",
"region": "",
"type": "plaintext"
}
},
"key-path": "",
"redact-info-log": false
},
"tso-save-interval": "3s",
"tso-update-physical-interval": "50ms"
+ "controller": {
+ "degraded-mode-wait-duration": "0s",
+ "request-unit": {
+ "read-base-cost": 0.25,
+ "read-cost-per-byte": 1.52587890625e-05,
+ "read-cpu-ms-cost": 0.3333333333333333,
+ "write-base-cost": 1,
+ "write-cost-per-byte": 0.0009765625
+ }
+ }
+ "keyspace": {
+ "pre-alloc": null
+ }
}
4.tikv 参数对比
tidba diff tikv --base-tikv-addr 127.0.0.1:20180 --new-tikv-addr 127.0.0.1:35014
[root@cctest tidb-community-toolkit-v7.1.0-linux-amd64]# ./tidba diff tikv --base-tikv-addr 127.0.0.1:20180 --new-tikv-addr 127.0.0.1:35014
--- 127.0.0.1:20180
+++ 127.0.0.1:35014
{
"abort-on-panic": false,
"backup": {
"auto-tune-refresh-interval": "1m",
"auto-tune-remain-threads": 3,
"batch-size": 8,
"enable-auto-tune": true,
"hadoop": {
"home": "",
"linux-user": ""
},
"io-thread-size": 2,
"num-threads": 8,
"s3-multi-part-size": "5MiB",
"sst-max-size": "144MiB"
},
"causal-ts": {
"renew-batch-min-size": 100,
"renew-interval": "100ms"
+ "alloc-ahead-buffer": "3s"
+ "renew-batch-max-size": 8192
},
"cdc": {
"hibernate-regions-compatible": true,
"incremental-scan-concurrency": 6,
"incremental-scan-speed-limit": "128MiB",
"incremental-scan-threads": 4,
"incremental-scan-ts-filter-ratio": 0.2,
"min-ts-interval": "1s",
"old-value-cache-memory-quota": "512MiB",
"sink-memory-quota": "512MiB"
+ "tso-worker-threads": 1
},
"coprocessor": {
"batch-split-limit": 10,
"consistency-check-method": "mvcc",
"enable-region-bucket": false,
"prefer-approximate-bucket": true,
"region-bucket-merge-size-ratio": 0.33,
"region-bucket-size": "96MiB",
"region-max-keys": 1.44e+06,
"region-max-size": "144MiB",
"region-size-threshold-for-approximate": "1440MiB",
"region-split-keys": 960000,
"region-split-size": "96MiB",
"split-region-on-table": false
},
"coprocessor-v2": {
"coprocessor-plugin-directory": null
},
"gc": {
"batch-keys": 512,
"compaction-filter-skip-version-check": false,
"enable-compaction-filter": true,
"max-write-bytes-per-sec": "0KiB",
"ratio-threshold": 1.1
},
"import": {
"import-mode-timeout": "10m",
"num-threads": 8,
"stream-channel-window": 128
+ "memory-use-ratio": 0.3
},
"log": {
"enable-timestamp": true,
"file": {
- "filename": "/root/.tiup/data/TiSzfi6/tikv-0/tikv.log",
+ "filename": "/root/.tiup/data/TiSzmGl/tikv-0/tikv.log",
"max-backups": 0,
"max-days": 0,
"max-size": 300
},
"format": "text",
"level": "info"
},
"log-backup": {
- "enable": false,
+ "enable": true,
"initial-scan-pending-memory-quota": "512MiB",
- "io-threads": 2,
- "max-flush-interval": "5m",
+ "max-flush-interval": "3m",
"num-threads": 8,
- "temp-file-size-limit-per-task": "128MiB",
- "temp-path": "/root/.tiup/data/TiSzfi6/tikv-0/data/log-backup-tmp"
+ "temp-path": "/root/.tiup/data/TiSzmGl/tikv-0/data/log-backup-temp"
+ "file-size-limit": "256MiB"
+ "initial-scan-rate-limit": "60MiB"
+ "min-ts-interval": "10s"
},
"log-file": "",
"log-format": "text",
"log-level": "info",
"log-rotation-size": "300MiB",
"log-rotation-timespan": "0s",
"memory-usage-high-water": 0.9,
- "memory-usage-limit": "24155MiB",
+ "memory-usage-limit": "23590MiB",
"pd": {
"enable-forwarding": false,
"endpoints": [
- 0: "http://127.0.0.1:38754"
+ 0: "http://127.0.0.1:2379"
],
"retry-interval": "300ms",
"retry-log-every": 10,
"retry-max-count": 9.223372036854776e+18,
"update-interval": "10m"
},
"pessimistic-txn": {
"in-memory": true,
"pipelined": true,
"wait-for-lock-timeout": "1s",
"wake-up-delay-duration": "20ms"
},
"quota": {
"background-cpu-time": 0,
"background-read-bandwidth": "0KiB",
"background-write-bandwidth": "0KiB",
"enable-auto-tune": false,
"foreground-cpu-time": 0,
"foreground-read-bandwidth": "0KiB",
"foreground-write-bandwidth": "0KiB",
"max-delay-duration": "500ms"
},
"raft-engine": {
"batch-compression-threshold": "8KiB",
- "bytes-per-sync": "4MiB",
+ "bytes-per-sync": null,
- "dir": "/root/.tiup/data/TiSzfi6/tikv-0/data/raft-engine",
+ "dir": "/root/.tiup/data/TiSzmGl/tikv-0/data/raft-engine",
"enable": true,
- "memory-limit": 5.181667584e+09,
+ "memory-limit": 5.060222361e+09,
"purge-rewrite-garbage-ratio": 0.6,
"purge-rewrite-threshold": "1GiB",
"purge-threshold": "10GiB",
"recovery-mode": "tolerate-corrupted-tail-records",
"recovery-read-block-size": "16KiB",
"recovery-threads": 4,
"target-file-size": "128MiB"
+ "enable-log-recycle": true
+ "format-version": 2
+ "prefill-for-recycle": false
+ "prefill-limit": null
+ "spill-dir": null
},
"raftdb": {
"allow-concurrent-memtable-write": true,
"bytes-per-sync": "1MiB",
"compaction-readahead-size": "0KiB",
"create-if-missing": true,
"defaultcf": {
"block-based-bloom-filter": false,
- "block-cache-size": "658MiB",
+ "block-cache-size": "643MiB",
"block-size": "64KiB",
"bloom-filter-bits-per-key": 10,
"bottommost-level-compression": "disable",
"bottommost-zstd-compression-dict-size": 0,
"bottommost-zstd-compression-sample-size": 0,
"cache-index-and-filter-blocks": true,
"compaction-guard-max-output-file-size": "128MiB",
"compaction-guard-min-output-file-size": "8MiB",
"compaction-pri": 0,
"compaction-style": 0,
"compression-per-level": [
: "no",
: "no",
: "lz4",
: "lz4",
: "lz4",
: "zstd",
: "zstd"
],
"disable-auto-compactions": false,
"disable-block-cache": false,
"disable-write-stall": false,
"dynamic-level-bytes": true,
- "enable-compaction-guard": false,
+ "enable-compaction-guard": null,
"enable-doubly-skiplist": true,
"force-consistency-checks": false,
"hard-pending-compaction-bytes-limit": "1TiB",
"level0-file-num-compaction-trigger": 4,
"level0-slowdown-writes-trigger": 20,
"level0-stop-writes-trigger": 20,
"max-bytes-for-level-base": "512MiB",
"max-bytes-for-level-multiplier": 10,
"max-compaction-bytes": "2GiB",
"max-write-buffer-number": 5,
"min-write-buffer-number-to-merge": 1,
"num-levels": 7,
"optimize-filters-for-hits": true,
"pin-l0-filter-and-index-blocks": true,
"prop-keys-index-distance": 40960,
"prop-size-index-distance": 4.194304e+06,
"read-amp-bytes-per-bit": 0,
"soft-pending-compaction-bytes-limit": "192GiB",
- "target-file-size-base": "8MiB",
+ "target-file-size-base": null,
"titan": {
"blob-cache-size": "0KiB",
"blob-file-compression": "lz4",
"blob-run-mode": "normal",
"discardable-ratio": 0.5,
"level-merge": false,
"max-gc-batch-size": "64MiB",
"max-sorted-runs": 20,
"merge-small-file-threshold": "8MiB",
"min-blob-size": "1KiB",
"min-gc-batch-size": "16MiB",
"range-merge": true
},
"use-bloom-filter": false,
"whole-key-filtering": true,
"write-buffer-size": "128MiB"
+ "checksum": "crc32c"
+ "format-version": 2
+ "max-compactions": 0
+ "prepopulate-block-cache": "disabled"
},
"enable-pipelined-write": true,
- "enable-statistics": true,
"enable-unordered-write": false,
"info-log-dir": "",
"info-log-keep-log-file-num": 10,
"info-log-level": "info",
"info-log-max-size": "1GiB",
"info-log-roll-time": "0s",
"max-background-flushes": 1,
"max-background-jobs": 4,
"max-manifest-file-size": "20MiB",
"max-open-files": 256,
"max-sub-compactions": 2,
"max-total-wal-size": "4GiB",
"stats-dump-period": "10m",
"titan": {
"dirname": "",
"disable-gc": false,
"enabled": false,
"max-background-gc": 4,
"purge-obsolete-files-period": "10s"
},
"use-direct-io-for-flush-and-compaction": false,
"wal-bytes-per-sync": "512KiB",
"wal-dir": "",
"wal-recovery-mode": 2,
"wal-size-limit": "0KiB",
"wal-ttl-seconds": 0,
"writable-file-max-buffer-size": "1MiB"
},
"raftstore": {
"abnormal-leader-missing-duration": "10m",
"apply-low-priority-pool-size": 1,
"apply-max-batch-size": 256,
"apply-pool-size": 2,
"apply-reschedule-duration": "5s",
"capacity": "0KiB",
"check-leader-lease-interval": "2s250ms",
"cleanup-import-sst-interval": "10m",
"cmd-batch": true,
"cmd-batch-concurrent-ready-max-count": 1,
"consistency-check-interval": "0s",
"evict-cache-on-memory-ratio": 0,
"future-poll-size": 1,
"hibernate-regions": true,
"inspect-interval": "500ms",
"io-reschedule-concurrent-max-count": 4,
"io-reschedule-hotpot-duration": "5s",
"local-read-batch-size": 1024,
"lock-cf-compact-bytes-threshold": "256MiB",
"lock-cf-compact-interval": "10m",
"max-leader-missing-duration": "2h",
"max-peer-down-duration": "10m",
"max-snapshot-file-raw-size": "100MiB",
"merge-check-tick-interval": "2s",
"messages-per-tick": 4096,
"notify-capacity": 40960,
"pd-heartbeat-tick-interval": "1m",
"pd-store-heartbeat-tick-interval": "10s",
"peer-stale-state-check-interval": "5m",
"perf-level": 0,
"prevote": true,
"raft-engine-purge-interval": "10s",
"raft-entry-cache-life-time": "30s",
"raft-entry-max-size": "8MiB",
"raft-log-compact-sync-interval": "2s",
"raft-log-gc-count-limit": 73728,
"raft-log-gc-size-limit": "72MiB",
"raft-log-gc-threshold": 50,
"raft-log-gc-tick-interval": "3s",
"raft-max-inflight-msgs": 256,
"raft-max-size-per-msg": "1MiB",
"raft-store-max-leader-lease": "9s",
"raft-write-size-limit": "1MiB",
- "raftdb-path": "/root/.tiup/data/TiSzfi6/tikv-0/data/raft",
+ "raftdb-path": "/root/.tiup/data/TiSzmGl/tikv-0/data/raft",
"reactive-memory-lock-tick-interval": "2s",
"reactive-memory-lock-timeout-tick": 5,
"region-compact-check-interval": "5m",
"region-compact-check-step": 100,
"region-compact-min-tombstones": 10000,
"region-compact-tombstones-percent": 30,
"region-split-check-diff": "6MiB",
"renew-leader-lease-advance-duration": "2s250ms",
- "report-min-resolved-ts-interval": "0s",
+ "report-min-resolved-ts-interval": "1s",
"report-region-buckets-tick-interval": "10s",
"snap-apply-batch-size": "10MiB",
"snap-gc-timeout": "4h",
"snap-generator-pool-size": 2,
"snap-mgr-gc-tick-interval": "1m",
"split-region-check-tick-interval": "10s",
"store-io-notify-capacity": 40960,
"store-io-pool-size": 0,
"store-low-priority-pool-size": 0,
"store-max-batch-size": 256,
"store-pool-size": 2,
"store-reschedule-duration": "5s",
"unreachable-backoff": "10s",
"waterfall-metrics": true
+ "apply-yield-write-size": "32KiB"
+ "check-long-uncommitted-interval": "10s"
+ "clean-stale-ranges-tick": 10
+ "long-uncommitted-base-threshold": "20s"
+ "max-entry-cache-warmup-duration": "1s"
+ "region-compact-min-redundant-rows": 50000
+ "region-compact-redundant-rows-percent": 20
+ "region-worker-tick-interval": "1s"
+ "request-voter-replicated-index-interval": "5m"
+ "slow-trend-unsensitive-cause": 10
+ "slow-trend-unsensitive-result": 0.5
},
"readpool": {
"coprocessor": {
"high-concurrency": 12,
"low-concurrency": 12,
"max-tasks-per-worker-high": 2000,
"max-tasks-per-worker-low": 2000,
"max-tasks-per-worker-normal": 2000,
"normal-concurrency": 12,
"stack-size": "10MiB",
"use-unified-pool": true
},
"storage": {
"high-concurrency": 8,
"low-concurrency": 8,
"max-tasks-per-worker-high": 2000,
"max-tasks-per-worker-low": 2000,
"max-tasks-per-worker-normal": 2000,
"normal-concurrency": 8,
"stack-size": "10MiB",
"use-unified-pool": true
},
"unified": {
"max-tasks-per-worker": 2000,
"max-thread-count": 12,
"min-thread-count": 1,
"stack-size": "10MiB"
+ "auto-adjust-pool-size": false
}
},
"resolved-ts": {
- "advance-ts-interval": "1s",
+ "advance-ts-interval": "20s",
"enable": true,
"scan-lock-pool-size": 2
},
"resource-metering": {
"max-resource-groups": 100,
"precision": "1s",
"receiver-address": "",
"report-receiver-interval": "1m"
},
"rocksdb": {
"bytes-per-sync": "1MiB",
"compaction-readahead-size": "0KiB",
"create-if-missing": true,
"defaultcf": {
"block-based-bloom-filter": false,
- "block-cache-size": "8236MiB",
+ "block-cache-size": "8043MiB",
- "block-size": "64KiB",
+ "block-size": "32KiB",
"bloom-filter-bits-per-key": 10,
"bottommost-level-compression": "zstd",
"bottommost-zstd-compression-dict-size": 0,
"bottommost-zstd-compression-sample-size": 0,
"cache-index-and-filter-blocks": true,
"compaction-guard-max-output-file-size": "128MiB",
"compaction-guard-min-output-file-size": "8MiB",
"compaction-pri": 3,
"compaction-style": 0,
"compression-per-level": [
: "no",
: "no",
: "lz4",
: "lz4",
: "lz4",
: "zstd",
: "zstd"
],
"disable-auto-compactions": false,
"disable-block-cache": false,
"disable-write-stall": true,
"dynamic-level-bytes": true,
"enable-compaction-guard": true,
"enable-doubly-skiplist": true,
"force-consistency-checks": false,
"hard-pending-compaction-bytes-limit": "1TiB",
"level0-file-num-compaction-trigger": 4,
"level0-slowdown-writes-trigger": 20,
"level0-stop-writes-trigger": 20,
"max-bytes-for-level-base": "512MiB",
"max-bytes-for-level-multiplier": 10,
"max-compaction-bytes": "2GiB",
"max-write-buffer-number": 5,
"min-write-buffer-number-to-merge": 1,
"num-levels": 7,
"optimize-filters-for-hits": true,
"pin-l0-filter-and-index-blocks": true,
"prop-keys-index-distance": 40960,
"prop-size-index-distance": 4.194304e+06,
"read-amp-bytes-per-bit": 0,
"soft-pending-compaction-bytes-limit": "192GiB",
- "target-file-size-base": "8MiB",
+ "target-file-size-base": null,
"titan": {
"blob-cache-size": "0KiB",
"blob-file-compression": "lz4",
"blob-run-mode": "normal",
"discardable-ratio": 0.5,
"level-merge": false,
"max-gc-batch-size": "64MiB",
"max-sorted-runs": 20,
"merge-small-file-threshold": "8MiB",
"min-blob-size": "1KiB",
"min-gc-batch-size": "16MiB",
"range-merge": true
},
"use-bloom-filter": true,
"whole-key-filtering": true,
"write-buffer-size": "128MiB"
+ "checksum": "crc32c"
+ "format-version": 2
+ "max-compactions": 0
+ "prepopulate-block-cache": "disabled"
},
- "enable-multi-batch-write": true,
+ "enable-multi-batch-write": null,
"enable-pipelined-write": false,
- "enable-statistics": true,
"enable-unordered-write": false,
"info-log-dir": "",
"info-log-keep-log-file-num": 10,
"info-log-level": "info",
"info-log-max-size": "1GiB",
"info-log-roll-time": "0s",
"lockcf": {
"block-based-bloom-filter": false,
- "block-cache-size": "658MiB",
+ "block-cache-size": "643MiB",
"block-size": "16KiB",
"bloom-filter-bits-per-key": 10,
"bottommost-level-compression": "disable",
"bottommost-zstd-compression-dict-size": 0,
"bottommost-zstd-compression-sample-size": 0,
"cache-index-and-filter-blocks": true,
"compaction-guard-max-output-file-size": "128MiB",
"compaction-guard-min-output-file-size": "8MiB",
"compaction-pri": 0,
"compaction-style": 0,
"compression-per-level": [
: "no",
: "no",
: "no",
: "no",
: "no",
: "no",
: "no"
],
"disable-auto-compactions": false,
"disable-block-cache": false,
"disable-write-stall": true,
"dynamic-level-bytes": true,
- "enable-compaction-guard": false,
+ "enable-compaction-guard": null,
"enable-doubly-skiplist": true,
"force-consistency-checks": false,
"hard-pending-compaction-bytes-limit": "1TiB",
"level0-file-num-compaction-trigger": 1,
"level0-slowdown-writes-trigger": 20,
"level0-stop-writes-trigger": 20,
"max-bytes-for-level-base": "128MiB",
"max-bytes-for-level-multiplier": 10,
"max-compaction-bytes": "2GiB",
"max-write-buffer-number": 5,
"min-write-buffer-number-to-merge": 1,
"num-levels": 7,
"optimize-filters-for-hits": false,
"pin-l0-filter-and-index-blocks": true,
"prop-keys-index-distance": 40960,
"prop-size-index-distance": 4.194304e+06,
"read-amp-bytes-per-bit": 0,
"soft-pending-compaction-bytes-limit": "192GiB",
- "target-file-size-base": "8MiB",
+ "target-file-size-base": null,
"titan": {
"blob-cache-size": "0KiB",
"blob-file-compression": "lz4",
"blob-run-mode": "read-only",
"discardable-ratio": 0.5,
"level-merge": false,
"max-gc-batch-size": "64MiB",
"max-sorted-runs": 20,
"merge-small-file-threshold": "8MiB",
"min-blob-size": "1KiB",
"min-gc-batch-size": "16MiB",
"range-merge": true
},
"use-bloom-filter": true,
"whole-key-filtering": true,
"write-buffer-size": "32MiB"
+ "checksum": "crc32c"
+ "format-version": 2
+ "max-compactions": 0
+ "prepopulate-block-cache": "disabled"
},
"max-background-flushes": 3,
"max-background-jobs": 9,
"max-manifest-file-size": "128MiB",
"max-open-files": 256,
"max-sub-compactions": 3,
"max-total-wal-size": "4GiB",
"raftcf": {
"block-based-bloom-filter": false,
"block-cache-size": "128MiB",
"block-size": "16KiB",
"bloom-filter-bits-per-key": 10,
"bottommost-level-compression": "disable",
"bottommost-zstd-compression-dict-size": 0,
"bottommost-zstd-compression-sample-size": 0,
"cache-index-and-filter-blocks": true,
"compaction-guard-max-output-file-size": "128MiB",
"compaction-guard-min-output-file-size": "8MiB",
"compaction-pri": 0,
"compaction-style": 0,
"compression-per-level": [
: "no",
: "no",
: "no",
: "no",
: "no",
: "no",
: "no"
],
"disable-auto-compactions": false,
"disable-block-cache": false,
"disable-write-stall": true,
"dynamic-level-bytes": true,
- "enable-compaction-guard": false,
+ "enable-compaction-guard": null,
"enable-doubly-skiplist": true,
"force-consistency-checks": false,
"hard-pending-compaction-bytes-limit": "1TiB",
"level0-file-num-compaction-trigger": 1,
"level0-slowdown-writes-trigger": 20,
"level0-stop-writes-trigger": 20,
"max-bytes-for-level-base": "128MiB",
"max-bytes-for-level-multiplier": 10,
"max-compaction-bytes": "2GiB",
"max-write-buffer-number": 5,
"min-write-buffer-number-to-merge": 1,
"num-levels": 7,
"optimize-filters-for-hits": true,
"pin-l0-filter-and-index-blocks": true,
"prop-keys-index-distance": 40960,
"prop-size-index-distance": 4.194304e+06,
"read-amp-bytes-per-bit": 0,
"soft-pending-compaction-bytes-limit": "192GiB",
- "target-file-size-base": "8MiB",
+ "target-file-size-base": null,
"titan": {
"blob-cache-size": "0KiB",
"blob-file-compression": "lz4",
"blob-run-mode": "read-only",
"discardable-ratio": 0.5,
"level-merge": false,
"max-gc-batch-size": "64MiB",
"max-sorted-runs": 20,
"merge-small-file-threshold": "8MiB",
"min-blob-size": "1KiB",
"min-gc-batch-size": "16MiB",
"range-merge": true
},
"use-bloom-filter": true,
"whole-key-filtering": true,
"write-buffer-size": "128MiB"
+ "checksum": "crc32c"
+ "format-version": 2
+ "max-compactions": 0
+ "prepopulate-block-cache": "disabled"
},
"rate-bytes-per-sec": "10GiB",
"rate-limiter-auto-tuned": true,
"rate-limiter-mode": 2,
"rate-limiter-refill-period": "100ms",
"stats-dump-period": "10m",
"titan": {
"dirname": "",
"disable-gc": false,
"enabled": false,
"max-background-gc": 4,
"purge-obsolete-files-period": "10s"
},
"use-direct-io-for-flush-and-compaction": false,
"wal-bytes-per-sync": "512KiB",
"wal-dir": "",
"wal-recovery-mode": 2,
"wal-size-limit": "0KiB",
"wal-ttl-seconds": 0,
"writable-file-max-buffer-size": "1MiB",
"writecf": {
"block-based-bloom-filter": false,
- "block-cache-size": "4941MiB",
+ "block-cache-size": "4825MiB",
- "block-size": "64KiB",
+ "block-size": "32KiB",
"bloom-filter-bits-per-key": 10,
"bottommost-level-compression": "zstd",
"bottommost-zstd-compression-dict-size": 0,
"bottommost-zstd-compression-sample-size": 0,
"cache-index-and-filter-blocks": true,
"compaction-guard-max-output-file-size": "128MiB",
"compaction-guard-min-output-file-size": "8MiB",
"compaction-pri": 3,
"compaction-style": 0,
"compression-per-level": [
: "no",
: "no",
: "lz4",
: "lz4",
: "lz4",
: "zstd",
: "zstd"
],
"disable-auto-compactions": false,
"disable-block-cache": false,
"disable-write-stall": true,
"dynamic-level-bytes": true,
"enable-compaction-guard": true,
"enable-doubly-skiplist": true,
"force-consistency-checks": false,
"hard-pending-compaction-bytes-limit": "1TiB",
"level0-file-num-compaction-trigger": 4,
"level0-slowdown-writes-trigger": 20,
"level0-stop-writes-trigger": 20,
"max-bytes-for-level-base": "512MiB",
"max-bytes-for-level-multiplier": 10,
"max-compaction-bytes": "2GiB",
"max-write-buffer-number": 5,
"min-write-buffer-number-to-merge": 1,
"num-levels": 7,
"optimize-filters-for-hits": false,
"pin-l0-filter-and-index-blocks": true,
"prop-keys-index-distance": 40960,
"prop-size-index-distance": 4.194304e+06,
"read-amp-bytes-per-bit": 0,
"soft-pending-compaction-bytes-limit": "192GiB",
- "target-file-size-base": "8MiB",
+ "target-file-size-base": null,
"titan": {
"blob-cache-size": "0KiB",
"blob-file-compression": "lz4",
"blob-run-mode": "read-only",
"discardable-ratio": 0.5,
"level-merge": false,
"max-gc-batch-size": "64MiB",
"max-sorted-runs": 20,
"merge-small-file-threshold": "8MiB",
"min-blob-size": "1KiB",
"min-gc-batch-size": "16MiB",
"range-merge": true
},
"use-bloom-filter": true,
"whole-key-filtering": false,
"write-buffer-size": "128MiB"
+ "checksum": "crc32c"
+ "format-version": 2
+ "max-compactions": 0
+ "prepopulate-block-cache": "disabled"
}
+ "allow-concurrent-memtable-write": true
+ "write-buffer-limit": null
},
"security": {
"ca-path": "",
"cert-allowed-cn": [
],
"cert-path": "",
"encryption": {
"data-encryption-method": "plaintext",
"data-key-rotation-period": "7d",
"enable-file-dictionary-log": true,
"file-dictionary-rewrite-threshold": 1e+06,
"master-key": {
"type": "plaintext"
},
"previous-master-key": {
"type": "plaintext"
}
},
"key-path": "",
"redact-info-log": null
},
"server": {
- "addr": "127.0.0.1:20160",
+ "addr": "127.0.0.1:33200",
- "advertise-addr": "127.0.0.1:20160",
+ "advertise-addr": "127.0.0.1:33200",
- "advertise-status-addr": "127.0.0.1:20180",
+ "advertise-status-addr": "127.0.0.1:35014",
"background-thread-count": 2,
"concurrent-recv-snap-limit": 32,
"concurrent-send-snap-limit": 32,
"enable-request-batch": true,
"end-point-batch-row-limit": 64,
"end-point-enable-batch-if-possible": true,
"end-point-max-concurrency": 16,
"end-point-perf-level": 0,
"end-point-recursion-limit": 1000,
"end-point-request-max-handle-duration": "1m",
"end-point-slow-log-threshold": "1s",
"end-point-stream-batch-row-limit": 128,
"end-point-stream-channel-size": 8,
"forward-max-connections-per-address": 4,
"grpc-compression-type": "none",
"grpc-concurrency": 5,
"grpc-concurrent-stream": 1024,
"grpc-keepalive-time": "10s",
"grpc-keepalive-timeout": "3s",
"grpc-memory-pool-quota": "9223372036854775807B",
"grpc-raft-conn-num": 1,
"grpc-stream-initial-window-size": "2MiB",
"heavy-load-threshold": 75,
"heavy-load-wait-duration": null,
"labels": {
},
"max-grpc-send-msg-len": 1.048576e+07,
"raft-client-grpc-send-msg-buffer": 524288,
"raft-client-queue-size": 8192,
"raft-msg-max-batch-size": 128,
"reject-messages-on-memory-ratio": 0.2,
"simplify-metrics": false,
"snap-max-total-size": "0KiB",
- "snap-max-write-bytes-per-sec": "100MiB",
"stats-concurrency": 1,
- "status-addr": "127.0.0.1:20180",
+ "status-addr": "127.0.0.1:35014",
"status-thread-pool-size": 1
+ "grpc-gzip-compression-level": 2
+ "grpc-min-message-size-to-compress": 4096
+ "snap-io-max-bytes-per-sec": "100MiB"
},
"slow-log-file": "",
"slow-log-threshold": "1s",
"split": {
"byte-threshold": 3.145728e+07,
"detect-times": 10,
"qps-threshold": 3000,
"sample-num": 20,
"sample-threshold": 100,
"split-balance-score": 0.25,
"split-contained-score": 0.5
+ "grpc-thread-cpu-overload-threshold-ratio": 0.5
+ "region-cpu-overload-threshold-ratio": 0.25
+ "unified-read-pool-thread-cpu-overload-threshold-ratio": 0.8
},
"storage": {
"api-version": 1,
"background-error-recovery-window": "1h",
"block-cache": {
- "capacity": "14493MiB",
+ "capacity": "14154MiB",
"high-pri-pool-ratio": 0.8,
"memory-allocator": "nodump",
"num-shard-bits": 6,
- "shared": true,
+ "shared": null,
"strict-capacity-limit": false
},
- "data-dir": "/root/.tiup/data/TiSzfi6/tikv-0/data",
+ "data-dir": "/root/.tiup/data/TiSzmGl/tikv-0/data",
"enable-async-apply-prewrite": false,
"enable-ttl": false,
"flow-control": {
"enable": true,
"hard-pending-compaction-bytes-limit": "1TiB",
"l0-files-threshold": 20,
"memtables-threshold": 5,
"soft-pending-compaction-bytes-limit": "192GiB"
},
"gc-ratio-threshold": 1.1,
"io-rate-limit": {
"compaction-priority": "low",
"export-priority": "medium",
"flush-priority": "high",
"foreground-read-priority": "high",
"foreground-write-priority": "high",
"gc-priority": "high",
"import-priority": "medium",
"level-zero-compaction-priority": "medium",
"load-balance-priority": "high",
"max-bytes-per-sec": "0KiB",
"mode": "write-only",
"other-priority": "high",
"replication-priority": "high",
"strict": false
},
"max-key-size": 8192,
"reserve-space": "0KiB",
"scheduler-concurrency": 524288,
"scheduler-pending-write-threshold": "100MiB",
"scheduler-worker-pool-size": 8,
"ttl-check-poll-interval": "12h"
+ "engine": "raft-kv"
+ "reserve-raft-space": "0KiB"
}
+ "resource-control": {
+ "enabled": true
+ }
}
五.总结
-前缀的参数为老集群的参数，+前缀的参数为新集群的参数，我们只需关注带前缀的参数就能直观地看到两套 TiDB 集群的参数差异以及新增和减少的参数，就可以针对这些参数去了解和调优。比如下图中可以直观地看到 6.16 版本中 tidb_enable_paging 默认值为 OFF，而在 7.1.0 版本中默认值变成了 ON。
综上，通过 TiDBA 对集群参数进行验证比对后，DBA 可以更丝滑地进行升级操作了。