From 7b66cb09d8688f3d74dc7b7b00b5000213d26758 Mon Sep 17 00:00:00 2001 From: ShuNing Date: Wed, 20 May 2020 14:38:23 +0800 Subject: [PATCH 1/7] Update command line flags for pd configruation (#2596) Signed-off-by: nolouch --- command-line-flags-for-pd-configuration.md | 6 ------ 1 file changed, 6 deletions(-) diff --git a/command-line-flags-for-pd-configuration.md b/command-line-flags-for-pd-configuration.md index 54685c083ac7b..bf4ed6144d9f2 100644 --- a/command-line-flags-for-pd-configuration.md +++ b/command-line-flags-for-pd-configuration.md @@ -102,12 +102,6 @@ PD is configurable using command-line flags and environment variables. - The path of the PEM file including the X509 key, used to enable TLS - Default: "" -## `--namespace-classifier` - -- To specify the namespace classifier used by PD -- Default: "table" -- If you use TiKV separately, not in the entire TiDB cluster, it is recommended to configure the value to 'default'. - ## `--metrics-addr` - The address of Prometheus Pushgateway, which does not push data to Promethus by default. From e2ab2ec5d89ab9e49d7a8c508c7f09d9b49146c0 Mon Sep 17 00:00:00 2001 From: Flowyi Date: Thu, 21 May 2020 15:56:33 +0800 Subject: [PATCH 2/7] Remove upgrade-tiflash.md (#2597) * Remove upgrade-tiflash.md Duplicated with TiDB's upgrade manual. * delete a link Co-authored-by: yikeke --- TOC.md | 1 - tiflash/upgrade-tiflash.md | 44 -------------------------------------- 2 files changed, 45 deletions(-) delete mode 100644 tiflash/upgrade-tiflash.md diff --git a/TOC.md b/TOC.md index 9498a94de02ab..ab60d0cd6de13 100644 --- a/TOC.md +++ b/TOC.md @@ -331,7 +331,6 @@ - [Maintain a TiFlash Cluster](/tiflash/maintain-tiflash.md) - [Monitor TiFlash](/tiflash/monitor-tiflash.md) - [Scale TiFlash](/scale-tidb-using-tiup.md#scale-out-a-tiflash-node) - - [Upgrade TiFlash Nodes](/tiflash/upgrade-tiflash.md) - [Configure TiFlash](/tiflash/tiflash-configuration.md) - [TiFlash Alert Rules](/tiflash/tiflash-alert-rules.md) - [Tune TiFlash Performance](/tiflash/tune-tiflash-performance.md) diff --git a/tiflash/upgrade-tiflash.md b/tiflash/upgrade-tiflash.md deleted file mode 100644 index f1afa313fd6d5..0000000000000 --- a/tiflash/upgrade-tiflash.md +++ /dev/null @@ -1,44 +0,0 @@ ---- -title: Upgrade TiFlash Nodes -summary: Learn how to upgrade TiFlash nodes. -category: reference -aliases: ['/docs/dev/reference/tiflash/upgrade/'] ---- - -# Upgrade TiFlash Nodes - -> **Note:** -> -> To upgrade TiFlash from the Pre-RC version to a later version, contact [PingCAP](mailto:info@pingcap.com) for more information and help. - -Currently, you cannot upgrade TiFlash by running the `tiup cluster upgrade` command. Instead, take the following steps to upgrade TiFlash: - -Before the upgrade, make sure that the cluster is started. To upgrade TiFlash nodes, take the following steps: - -1. Refer to [Scale in a TiFlash node](/scale-tidb-using-tiup.md#sclale-in-a-tiflash-node), and scale in all the TiFlash nodes. - -2. Run the upgrade command: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster upgrade test v4.0.0-rc - ``` - -3. Run the scale-out command: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster scale-out test scale-out.yaml - ``` - -4. View the cluster status: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster display test - ``` - -5. Access the monitoring platform using your browser, and view the status of the cluster. From 5ab88cd8ced159e5e806063203ad49836aa1f76b Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 25 May 2020 11:04:09 +0800 Subject: [PATCH 3/7] add Coprocessor cache doc --- TOC.md | 1 + coprocessor-cache.md | 39 ++++++++++++++++++++++++++++++++++++++ tidb-configuration-file.md | 27 ++++++++++++++++++++++++++ 3 files changed, 67 insertions(+) create mode 100644 coprocessor-cache.md diff --git a/TOC.md b/TOC.md index ab60d0cd6de13..54e2492ec88c4 100644 --- a/TOC.md +++ b/TOC.md @@ -309,6 +309,7 @@ - [Execution Plan Binding](/execution-plan-binding.md) - [Access Tables Using `IndexMerge`](/index-merge.md) - [Statement Summary Table](/statement-summary-tables.md) + - [Coprocessor Cache](/coprocessor-cache.md) - [Tune TiKV](/tune-tikv-performance.md) + Key Monitoring Metrics - [Overview](/grafana-overview-dashboard.md) diff --git a/coprocessor-cache.md b/coprocessor-cache.md new file mode 100644 index 0000000000000..59dfd0b418289 --- /dev/null +++ b/coprocessor-cache.md @@ -0,0 +1,39 @@ +--- +title: Coprocessor Cache +summary: Learn the features of Coprocessor Cache. +category: reference +--- + +# Coprocessor Cache + +Starting from v4.0, the TiDB instance supports caching the results of the calculation that is pushed down to TiKV (the Coprocessor Cache feature), which can accelerate the calculation process in some scenarios. + +## Configuration + +You can configure Coprocessor Cache via the `tikv-client.copr-cache` configuration items in the TiDB configuration file. For details about how to enable and configure Coprocessor Cache, see [TiDB Configuration File](/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v40). + +## Feature description + ++ When a SQL statement is executed on a single TiDB instance for the first time, the execution result is not cached. ++ Calculation results are cache in the memory of TiDB. If the TiDB instance is restarted, the cache becomes invalid. ++ The cache is not shared among TiDB instances. ++ The cached result is the result of push-down calculations. Even if the result hits the cache, TiDB still performs calculation. ++ The cache is in the unit of Region. Writing data to a Region causes the Region cache to be invalid. For this reason, the Coprocessor Cache feature mainly takes effect on the data that rarely changes. ++ When push-down calculation requests are the same, the cache is hit. Usually in the following scenarios, the push-down calculation requests are the same or partially the same: + - The SQL statements are the same. For example, the same SQL statement is executed repeatedly. + + In this scenario, all the push-down calculation requests are consistent, and all requests can use the push-down calculation cache. + + - The SQL statements contain a changing condition, and the other parts are consistent. The changing condition is the primary key of the table or the partition. + + In this scenario, some of the push-down calculation requests are the same with some previous requests, and these calculation requests can use the cached (previous) push-down calculation result. + + - The SQL statements contain multiple changing conditions and the other parts are consistent. The changing conditions exactly match a compound index column. + + In this scenario, some of the push-down calculation requests are the same with some previous requests, and these calculation requests can use the cached (previous) push-down calculation result. + ++ This feature is transparent to users. Enabling or disabling this feature does not affect the calculation result and only affects the SQL execution time. + +## Check the cache effect + +Currently (TiDB v4.0.0-rc.2), Coprocessor cache is still an experimental feature. Users cannot learn how many push-down requests hit the cache in a SQL statement, or understand the overall cache hits. The corresponding monitoring and checking method will be introduced in later TiDB versions. diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index 9e62cd3b22163..980ee4f4daa1a 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -417,6 +417,33 @@ The Plan Cache configuration of the `PREPARE` statement. - The threshold of the TiKV load. If the TiKV load exceeds this threshold, more `batch` packets are collected to relieve the pressure of TiKV. It is valid only when the value of `tikv-client.max-batch-size` is greater than `0`. It is recommended not to modify this value. - Default value: `200` +## tikv-client.copr-cache New in v4.0.0 + +This section introduces configuration items related to the Coprocessor Cache feature. + +### `enable` + +- Determines whether to enable [Coprocessor Cache](/coprocessor-cache.md). +- Default value: `false` (which means that Coprocessor Cache is disabled by default) + +### `capacity-mb` + +- The total size of the cached data. When the cache space is full, old cache entries is evicted. +- Default value: `1000` +- Unit: MB + +### `admission-max-result-mb` + +- Specifies the largest single push-down calculation result set that can be cached. If the result set of a single push-down calculation returned on the Coprocessor is larger than the result set specified by this parameter, the result set is cached. Increasing this value means that more types of push-down requests are cached, but also cause the cache space to be occupied more easily. Note that the size of each push-down calculation result set is generally smaller than the size of the Region. Therefore, it is meaningless to set this value far beyond the size of a Region. +- Default value: `10` +- Unit: MB + +### `admission-min-process-ms` + +- Specifies the minimum calculation time for a single push-down calculation result set that can be cached. If the calculation time of a single push-down calculation on the Coprocessor is less than the time specified by this parameter, the result set is not cached. Requests that are processed quickly do not need to be cached, and only the requests that take a long time to process need to be cached, which makes the cache less likely to be evicted. +- Default value: `5` +- Unit: ms + ### txn-local-latches Configuration related to the transaction latch. It is recommended to enable it when many local transaction conflicts occur. From 9445c98ebef77e26d5aa0ab3212bdc4084d8a745 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 25 May 2020 11:07:38 +0800 Subject: [PATCH 4/7] Update coprocessor-cache.md --- coprocessor-cache.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/coprocessor-cache.md b/coprocessor-cache.md index 59dfd0b418289..d471639c66e45 100644 --- a/coprocessor-cache.md +++ b/coprocessor-cache.md @@ -10,7 +10,7 @@ Starting from v4.0, the TiDB instance supports caching the results of the calcul ## Configuration -You can configure Coprocessor Cache via the `tikv-client.copr-cache` configuration items in the TiDB configuration file. For details about how to enable and configure Coprocessor Cache, see [TiDB Configuration File](/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v40). +You can configure Coprocessor Cache via the `tikv-client.copr-cache` configuration items in the TiDB configuration file. For details about how to enable and configure Coprocessor Cache, see [TiDB Configuration File](/tidb-configuration-file.md#tikv-clientcopr-cache-new-in-v400). ## Feature description From cbcb042228bf6811b00b955facf2ef85a3854c89 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 25 May 2020 11:10:45 +0800 Subject: [PATCH 5/7] Update tidb-configuration-file.md --- tidb-configuration-file.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index 980ee4f4daa1a..0e4619b668499 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -6,6 +6,7 @@ aliases: ['/docs/dev/reference/configuration/tidb-server/configuration-file/'] --- + # TiDB Configuration File From 1adb44f94eb90d90277a0f4adaa70f7200f6c6b7 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 27 May 2020 12:00:59 +0800 Subject: [PATCH 6/7] Apply suggestions from code review Co-authored-by: Wenxuan --- coprocessor-cache.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/coprocessor-cache.md b/coprocessor-cache.md index d471639c66e45..a6bfb10d15cac 100644 --- a/coprocessor-cache.md +++ b/coprocessor-cache.md @@ -15,9 +15,9 @@ You can configure Coprocessor Cache via the `tikv-client.copr-cache` configurati ## Feature description + When a SQL statement is executed on a single TiDB instance for the first time, the execution result is not cached. -+ Calculation results are cache in the memory of TiDB. If the TiDB instance is restarted, the cache becomes invalid. ++ Calculation results are cached in the memory of TiDB. If the TiDB instance is restarted, the cache becomes invalid. + The cache is not shared among TiDB instances. -+ The cached result is the result of push-down calculations. Even if the result hits the cache, TiDB still performs calculation. ++ Only push-down calculation result is cached. Even if cache is hit, TiDB still need to perform subsequent calculation. + The cache is in the unit of Region. Writing data to a Region causes the Region cache to be invalid. For this reason, the Coprocessor Cache feature mainly takes effect on the data that rarely changes. + When push-down calculation requests are the same, the cache is hit. Usually in the following scenarios, the push-down calculation requests are the same or partially the same: - The SQL statements are the same. For example, the same SQL statement is executed repeatedly. From 8cd695501f0c5b7e86e51de01c99061bee028aa9 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 28 May 2020 20:11:02 +0800 Subject: [PATCH 7/7] Update tidb-configuration-file.md Co-authored-by: Keke Yi <40977455+yikeke@users.noreply.github.com> --- tidb-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index 0e4619b668499..3b1f6d30fdcdd 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -429,7 +429,7 @@ This section introduces configuration items related to the Coprocessor Cache fea ### `capacity-mb` -- The total size of the cached data. When the cache space is full, old cache entries is evicted. +- The total size of the cached data. When the cache space is full, old cache entries are evicted. - Default value: `1000` - Unit: MB