From 6576d525f0e5fcaff2e9333eeeafefe331760072 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Wed, 2 Jun 2021 15:07:18 +0800 Subject: [PATCH 1/6] Overhauling TiKV RocksDB configuration file --- tikv-configuration-file.md | 45 ++++---------------------------------- 1 file changed, 4 insertions(+), 41 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 00811418beea4..5bc95e928f326 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -898,7 +898,7 @@ Configuration items related to `rocksdb.defaultcf` + Determines whether to optimize the hit ratio of filters + Default value: `true` -### `whole_key_filtering` +### `whole-key-filtering` + Determines whether to put the entire key to bloom filter + Default value: `true` @@ -1013,7 +1013,7 @@ Configuration items related to `rocksdb.defaultcf` + The default amplification multiple for each layer + Default value: `10` -### `rocksdb.defaultcf.compaction-style` +### `compaction-style` + Compaction method + Optional values: `"level"`, `"universal"` @@ -1133,7 +1133,7 @@ Configuration items related to `rocksdb.defaultcf.titan` ## rocksdb.writecf -Configuration items related to `rocksdb.writecf` +Configuration items related to `rocksdb.writecf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value and are recommended to be set the same as `rocksdb.defaultcf`. ### `block-cache-size` @@ -1141,36 +1141,9 @@ Configuration items related to `rocksdb.writecf` + Default value: `Total machine memory * 15%` + Unit: MB|GB -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `whole-key-filtering` - -+ Determines whether to put the entire key to bloom filter -+ Default value: `false` - -### `enable-compaction-guard` - -+ Enables or disables the compaction guard, which is an optimization to split SST files at TiKV Region boundaries. This optimization can help reduce compaction I/O and allows TiKV to use larger SST file size (thus less SST files overall) and at the time efficiently clean up stale data when migrating Regions. -+ Default value: `true` - -### `compaction-guard-min-output-file-size` - -+ The minimum SST file size when the compaction guard is enabled. This configuration prevents SST files from being too small when the compaction guard is enabled. -+ Default value: `"8MB"` -+ Unit: KB|MB|GB - -### `compaction-guard-max-output-file-size` - -+ The maximum SST file size when the compaction guard is enabled. The configuration prevents SST files from being too large when the compaction guard is enabled. This configuration overrides `target-file-size-base` for the same column family. -+ Default value: `"128MB"` -+ Unit: KB|MB|GB - ## rocksdb.lockcf -Configuration items related to `rocksdb.lockcf` +Configuration items related to `rocksdb.lockcf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value and are recommended to be set the same as `rocksdb.defaultcf`. ### `block-cache-size` @@ -1178,16 +1151,6 @@ Configuration items related to `rocksdb.lockcf` + Default value: `Total machine memory * 2%` + Unit: MB|GB -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `level0-file-num-compaction-trigger` - -+ The number of files at L0 required to trigger compaction -+ Default value: `1` - ## `raftdb` Configuration items related to `raftdb` From 1e34be9d4cd1836da74983bf4927967c501f36af Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Wed, 2 Jun 2021 16:05:46 +0800 Subject: [PATCH 2/6] Fix inconsistency with TiKV master --- tikv-configuration-file.md | 60 +++++++++++++++++++++++++++++++++++++- 1 file changed, 59 insertions(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 5bc95e928f326..d75d7da5dbeee 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -995,7 +995,7 @@ Configuration items related to `rocksdb.defaultcf` ### `compaction-pri` + The priority type of compaction -+ Optional values: `3` (`MinOverlappingRatio`), `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`) ++ Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) + Default value: `3` ### `dynamic-level-bytes` @@ -1141,16 +1141,74 @@ Configuration items related to `rocksdb.writecf`, which are identical to `rocksd + Default value: `Total machine memory * 15%` + Unit: MB|GB +### `optimize-filters-for-hits` + ++ Determines whether to optimize the hit ratio of the filter ++ Default value: `false` + +### `whole-key-filtering` + ++ Determines whether to put the entire key to bloom filter ++ Default value: `false` + ## rocksdb.lockcf Configuration items related to `rocksdb.lockcf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value and are recommended to be set the same as `rocksdb.defaultcf`. +### `block-size` + ++ The default size of a RocksDB block ++ Default value: `"16KB"` ++ Minimum value: `"1KB"` ++ Unit: KB|MB|GB + ### `block-cache-size` + Block cache size + Default value: `Total machine memory * 2%` + Unit: MB|GB +### `optimize-filters-for-hits` + ++ Determines whether to optimize the hit ratio of the filter ++ Default value: `false` + +### `whole-key-filtering` + ++ Determines whether to put the entire key to bloom filter ++ Default value: `false` + +### `write-buffer-size` + ++ Memtable size ++ Default value: `"32MB"` ++ Minimum value: `0` ++ Unit: KB|MB|GB + +### `max-bytes-for-level-base` + ++ The maximum number of bytes at base level (L1). Generally, it is set to 4 times the size of a memtable. ++ Default value: `"128MB"` ++ Minimum value: `0` ++ Unit: KB|MB|GB + +### `level0-file-num-compaction-trigger` + ++ The maximum number of files at L0 that trigger compaction ++ Default value: `1` ++ Minimum value: `0` + +### `compaction-pri` + ++ The priority type of compaction ++ Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) ++ Default value: `1` + +### `enable-compaction-guard` + ++ Enables or disables the compaction guard, which is an optimization to split SST files at TiKV Region boundaries. This optimization can help reduce compaction I/O and allows TiKV to use larger SST file size (thus less SST files overall) and at the time efficiently clean up stale data when migrating Regions. ++ Default value: `false` + ## `raftdb` Configuration items related to `raftdb` From 0241aa9a4b3b3f3a8237b0e256daa186ded6d270 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Thu, 3 Jun 2021 14:54:08 +0800 Subject: [PATCH 3/6] Address comment --- tikv-configuration-file.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index d75d7da5dbeee..9fb678bbb338e 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1133,7 +1133,7 @@ Configuration items related to `rocksdb.defaultcf.titan` ## rocksdb.writecf -Configuration items related to `rocksdb.writecf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value and are recommended to be set the same as `rocksdb.defaultcf`. +Configuration items related to `rocksdb.writecf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value as `rocksdb.defaultcf`. ### `block-cache-size` @@ -1153,7 +1153,7 @@ Configuration items related to `rocksdb.writecf`, which are identical to `rocksd ## rocksdb.lockcf -Configuration items related to `rocksdb.lockcf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value and are recommended to be set the same as `rocksdb.defaultcf`. +Configuration items related to `rocksdb.lockcf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value as `rocksdb.defaultcf`. ### `block-size` From 87c4dee337a60e14570e06c14798e57b82c0c344 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Fri, 4 Jun 2021 18:54:09 +0800 Subject: [PATCH 4/6] Address comments --- tikv-configuration-file.md | 114 ++++++++----------------------------- 1 file changed, 23 insertions(+), 91 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 9fb678bbb338e..b1f087557b709 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -855,21 +855,24 @@ Configuration items related to Titan + Default value: `4` + Minimum value: `1` -## rocksdb.defaultcf +## rocksdb.defaultcf | rocksdb.writecf | rocksdb.lockcf -Configuration items related to `rocksdb.defaultcf` +Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf` and `rocksdb.lockcf`. ### `block-size` + The default size of a RocksDB block -+ Default value: `"64KB"` ++ Default value for `defaultcf` and `writecf`: `"64KB"` ++ Default value for `lockcf`: `"16KB"` + Minimum value: `"1KB"` + Unit: KB|MB|GB ### `block-cache-size` + The cache size of a RocksDB block -+ Default value: `Total machine memory * 25%` ++ Default value for `defaultcf`: `Total machine memory * 25%` ++ Default value for `writecf`: `Total machine memory * 15%` ++ Default value for `lockcf`: `Total machine memory * 2%` + Minimum value: `0` + Unit: KB|MB|GB @@ -896,12 +899,14 @@ Configuration items related to `rocksdb.defaultcf` ### `optimize-filters-for-hits` + Determines whether to optimize the hit ratio of filters -+ Default value: `true` ++ Default value for `defaultcf`: `true` ++ Default value for `writecf` and `lockcf`: `false` ### `whole-key-filtering` + Determines whether to put the entire key to bloom filter -+ Default value: `true` ++ Default value for `defaultcf`: `true` ++ Default value for `writecf` and `lockcf`: `true` ### `bloom-filter-bits-per-key` @@ -937,7 +942,8 @@ Configuration items related to `rocksdb.defaultcf` ### `write-buffer-size` + Memtable size -+ Default value: `"128MB"` ++ Default value for `defaultcf` and `writecf`: `"128MB"` ++ Default value for `lockcf`: `"32MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -956,7 +962,8 @@ Configuration items related to `rocksdb.defaultcf` ### `max-bytes-for-level-base` + The maximum number of bytes at base level (L1). Generally, it is set to 4 times the size of a memtable. -+ Default value: `"512MB"` ++ Default value for `defaultcf` and `writecf`: `"512MB"` ++ Default value for `lockcf`: `"128MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -970,7 +977,8 @@ Configuration items related to `rocksdb.defaultcf` ### `level0-file-num-compaction-trigger` + The maximum number of files at L0 that trigger compaction -+ Default value: `4` ++ Default value for `defaultcf` and `writecf`: `4` ++ Default value for `lockcf`: `1` + Minimum value: `0` ### `level0-slowdown-writes-trigger` @@ -996,7 +1004,8 @@ Configuration items related to `rocksdb.defaultcf` + The priority type of compaction + Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) -+ Default value: `3` ++ Default value for `defaultcf` and `writecf`: `3` ++ Default value for `lockcf`: `1` ### `dynamic-level-bytes` @@ -1039,7 +1048,8 @@ Configuration items related to `rocksdb.defaultcf` ### `enable-compaction-guard` + Enables or disables the compaction guard, which is an optimization to split SST files at TiKV Region boundaries. This optimization can help reduce compaction I/O and allows TiKV to use larger SST file size (thus less SST files overall) and at the time efficiently clean up stale data when migrating Regions. -+ Default value: `true` ++ Default value for `defaultcf` and `writecf`: `true` ++ Default value for `lockcf`: `false` ### `compaction-guard-min-output-file-size` @@ -1053,9 +1063,9 @@ Configuration items related to `rocksdb.defaultcf` + Default value: `"128MB"` + Unit: KB|MB|GB -## `rocksdb.defaultcf.titan` +## `rocksdb.defaultcf.titan` | `rocksdb.writecf.titan` | `rocksdb.lockcf.titan` -Configuration items related to `rocksdb.defaultcf.titan` +Configuration items related to `rocksdb.defaultcf.titan`, `rocksdb.writecf.titan` and `rocksdb.lockcf.titan`. ### `min-blob-size` @@ -1131,84 +1141,6 @@ Configuration items related to `rocksdb.defaultcf.titan` + Determines whether to use the merge operator to write back blob indexes for Titan GC. When `gc-merge-rewrite` is enabled, it reduces the effect of Titan GC on the writes in the foreground. + Default value: `false` -## rocksdb.writecf - -Configuration items related to `rocksdb.writecf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value as `rocksdb.defaultcf`. - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 15%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `whole-key-filtering` - -+ Determines whether to put the entire key to bloom filter -+ Default value: `false` - -## rocksdb.lockcf - -Configuration items related to `rocksdb.lockcf`, which are identical to `rocksdb.defaultcf`. Those not listed here have the same default value as `rocksdb.defaultcf`. - -### `block-size` - -+ The default size of a RocksDB block -+ Default value: `"16KB"` -+ Minimum value: `"1KB"` -+ Unit: KB|MB|GB - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 2%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `whole-key-filtering` - -+ Determines whether to put the entire key to bloom filter -+ Default value: `false` - -### `write-buffer-size` - -+ Memtable size -+ Default value: `"32MB"` -+ Minimum value: `0` -+ Unit: KB|MB|GB - -### `max-bytes-for-level-base` - -+ The maximum number of bytes at base level (L1). Generally, it is set to 4 times the size of a memtable. -+ Default value: `"128MB"` -+ Minimum value: `0` -+ Unit: KB|MB|GB - -### `level0-file-num-compaction-trigger` - -+ The maximum number of files at L0 that trigger compaction -+ Default value: `1` -+ Minimum value: `0` - -### `compaction-pri` - -+ The priority type of compaction -+ Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) -+ Default value: `1` - -### `enable-compaction-guard` - -+ Enables or disables the compaction guard, which is an optimization to split SST files at TiKV Region boundaries. This optimization can help reduce compaction I/O and allows TiKV to use larger SST file size (thus less SST files overall) and at the time efficiently clean up stale data when migrating Regions. -+ Default value: `false` - ## `raftdb` Configuration items related to `raftdb` From bd822632e34f5511af4d6c6809d140d28bcc5795 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Fri, 4 Jun 2021 19:49:07 +0800 Subject: [PATCH 5/6] Address comments --- tikv-configuration-file.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index b1f087557b709..e0332c4431afa 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -857,7 +857,7 @@ Configuration items related to Titan ## rocksdb.defaultcf | rocksdb.writecf | rocksdb.lockcf -Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf` and `rocksdb.lockcf`. +Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rocksdb.lockcf`. ### `block-size` @@ -906,7 +906,7 @@ Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf` and `rocks + Determines whether to put the entire key to bloom filter + Default value for `defaultcf`: `true` -+ Default value for `writecf` and `lockcf`: `true` ++ Default value for `writecf` and `lockcf`: `false` ### `bloom-filter-bits-per-key` @@ -1065,7 +1065,7 @@ Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf` and `rocks ## `rocksdb.defaultcf.titan` | `rocksdb.writecf.titan` | `rocksdb.lockcf.titan` -Configuration items related to `rocksdb.defaultcf.titan`, `rocksdb.writecf.titan` and `rocksdb.lockcf.titan`. +Configuration items related to `rocksdb.defaultcf.titan`, `rocksdb.writecf.titan`, and `rocksdb.lockcf.titan`. ### `min-blob-size` From d9a69c6cd6d28c1077b701345d7d55d19d9f0c68 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Fri, 4 Jun 2021 19:50:49 +0800 Subject: [PATCH 6/6] Minor fix --- tikv-configuration-file.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index e0332c4431afa..9ccd686c3f6d9 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -1063,7 +1063,7 @@ Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rock + Default value: `"128MB"` + Unit: KB|MB|GB -## `rocksdb.defaultcf.titan` | `rocksdb.writecf.titan` | `rocksdb.lockcf.titan` +## rocksdb.defaultcf.titan | rocksdb.writecf.titan | rocksdb.lockcf.titan Configuration items related to `rocksdb.defaultcf.titan`, `rocksdb.writecf.titan`, and `rocksdb.lockcf.titan`.