From 7d9bdc2ba90e13cb3117d349f9e1a69790aec7f1 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 16 Aug 2021 16:02:51 +0800 Subject: [PATCH 1/4] tikv: add storage flow control doc --- tikv-configuration-file.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index ad34ce379ffd1..537ac23619348 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -312,6 +312,35 @@ Configuration items related to the sharing of block cache among multiple RocksDB + Default value: 45% of the size of total system memory + Unit: KB|MB|GB +## storage.flow-control + +Configuration items related the flow control mechanism in TiKV. This mechanism replaces the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids the issue of QPS drop caused by the stuck Raftstore or Apply thread when the write traffic is high. + +### `enable` + ++ Determines whether to enable the flow control mechanism. After it is enabled, TiKV automatically disables the write stall mechanism of KvDB and the write stall mechanism of RaftDB (excluding memtable). ++ Default value: `true` + +### `memtables-threshold` + ++ When the number of kvDB memtables reaches this threshold, the flow control mechanism starts to work. ++ Default value: `5` + +### `l0-files-threshold` + ++ When the number of kvDB L0 files reaches this threshold, the flow control mechanism starts to work. ++ Default value: `9` + +### `soft-pending-compaction-bytes-limit` + ++ When the pending compaction bytes in KvDB reaches this threshold, the flow control mechanism starts to reject some write requests and reports the `ServerIsBusy` error. ++ Default value: `"192GB"` + +### `hard-pending-compaction-bytes-limit` + ++ When the pending compaction bytes in KvDB reaches this threshold, the flow control mechanism rejects all write requests and reports the `ServerIsBusy` error. ++ Default value: `"1024GB"` + ## storage.io-rate-limit Configuration items related to the I/O rate limiter. From fd153b678db2f9e75b1c5b168488fc3a4eb5de06 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 16 Aug 2021 16:04:27 +0800 Subject: [PATCH 2/4] Update tikv-configuration-file.md --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 537ac23619348..2e472c39ebc0e 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -314,7 +314,7 @@ Configuration items related to the sharing of block cache among multiple RocksDB ## storage.flow-control -Configuration items related the flow control mechanism in TiKV. This mechanism replaces the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids the issue of QPS drop caused by the stuck Raftstore or Apply thread when the write traffic is high. +Configuration items related to the flow control mechanism in TiKV. This mechanism replaces the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids the issue of QPS drop caused by the stuck Raftstore or Apply threads when the write traffic is high. ### `enable` From 6a7340b0f93c4f2ce9b441cebeb1b7ff1f6ef1c1 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 16 Aug 2021 16:07:32 +0800 Subject: [PATCH 3/4] Update tikv-configuration-file.md --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 2e472c39ebc0e..7e1a82f6eff60 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -314,7 +314,7 @@ Configuration items related to the sharing of block cache among multiple RocksDB ## storage.flow-control -Configuration items related to the flow control mechanism in TiKV. This mechanism replaces the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids the issue of QPS drop caused by the stuck Raftstore or Apply threads when the write traffic is high. +Configuration items related to the flow control mechanism in TiKV. This mechanism supersedes the write stall mechanism in RocksDB and controls flow at the scheduler layer, which avoids the issue of QPS drop caused by the stuck Raftstore or Apply threads when the write traffic is high. ### `enable` From 3b91037e5a47e70e9e806e8f9c2f208b2c7498ad Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 18 Aug 2021 15:10:38 +0800 Subject: [PATCH 4/4] Apply suggestions from code review --- tikv-configuration-file.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 7e1a82f6eff60..d413a29199b79 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -333,12 +333,12 @@ Configuration items related to the flow control mechanism in TiKV. This mechanis ### `soft-pending-compaction-bytes-limit` -+ When the pending compaction bytes in KvDB reaches this threshold, the flow control mechanism starts to reject some write requests and reports the `ServerIsBusy` error. ++ When the pending compaction bytes in KvDB reach this threshold, the flow control mechanism starts to reject some write requests and reports the `ServerIsBusy` error. + Default value: `"192GB"` ### `hard-pending-compaction-bytes-limit` -+ When the pending compaction bytes in KvDB reaches this threshold, the flow control mechanism rejects all write requests and reports the `ServerIsBusy` error. ++ When the pending compaction bytes in KvDB reach this threshold, the flow control mechanism rejects all write requests and reports the `ServerIsBusy` error. + Default value: `"1024GB"` ## storage.io-rate-limit