From 1bf6179157e0a517f4f8459a99c2480ba998dee6 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Mon, 28 Jun 2021 10:23:25 +0800 Subject: [PATCH 1/2] Overhauling TiKV RocksDB configuration file (#5746) Signed-off-by: tabokie --- tikv-configuration-file.md | 80 ++++++++++++-------------------------- 1 file changed, 24 insertions(+), 56 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 04e198c2cd811..609c850d4d350 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -771,21 +771,24 @@ Configuration items related to Titan + Default value: `1` + Minimum value: `1` -## rocksdb.defaultcf +## rocksdb.defaultcf | rocksdb.writecf | rocksdb.lockcf -Configuration items related to `rocksdb.defaultcf` +Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rocksdb.lockcf`. ### `block-size` + The default size of a RocksDB block -+ Default value: `64KB` -+ Minimum value: `1KB` ++ Default value for `defaultcf` and `writecf`: `"64KB"` ++ Default value for `lockcf`: `"16KB"` ++ Minimum value: `"1KB"` + Unit: KB|MB|GB ### `block-cache-size` + The cache size of a RocksDB block -+ Default value: `Total machine memory / 4` ++ Default value for `defaultcf`: `Total machine memory * 25%` ++ Default value for `writecf`: `Total machine memory * 15%` ++ Default value for `lockcf`: `Total machine memory * 2%` + Minimum value: `0` + Unit: KB|MB|GB @@ -812,9 +815,10 @@ Configuration items related to `rocksdb.defaultcf` ### `optimize-filters-for-hits` + Determines whether to optimize the hit ratio of filters -+ Default value: `true` ++ Default value for `defaultcf`: `true` ++ Default value for `writecf` and `lockcf`: `false` -### `whole_key_filtering` +### `whole-key-filtering` + Determines whether to put the entire key to bloom filter + Default value: `true` @@ -840,13 +844,14 @@ Configuration items related to `rocksdb.defaultcf` ### `compression-per-level` + The default compression algorithm for each level -+ Available values: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] -+ Default value: `No` for the first two levels, and `lz4` for the next five levels ++ Optional values: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] ++ Default value for `defaultcf` and `writecf`: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] ++ Default value for `lockcf`: ["no", "no", "no", "no", "no", "no", "no"] ### `write-buffer-size` + Memtable size -+ Default value: `128MB` ++ Default value: `"128MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -865,7 +870,8 @@ Configuration items related to `rocksdb.defaultcf` ### `max-bytes-for-level-base` + The maximum number of bytes at base level (L1). Generally, it is set to 4 times the size of a memtable. -+ Default value: `512MB` ++ Default value for `defaultcf` and `writecf`: `"512MB"` ++ Default value for `lockcf`: `"128MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -879,7 +885,8 @@ Configuration items related to `rocksdb.defaultcf` ### `level0-file-num-compaction-trigger` + The maximum number of files at L0 that trigger compaction -+ Default value: `4` ++ Default value for `defaultcf` and `writecf`: `4` ++ Default value for `lockcf`: `1` + Minimum value: `0` ### `level0-slowdown-writes-trigger` @@ -904,8 +911,9 @@ Configuration items related to `rocksdb.defaultcf` ### `compaction-pri` + The priority type of compaction -+ Available values: `3` (`MinOverlappingRatio`), `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`) -+ Default value: `3` ++ Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) ++ Default value for `defaultcf` and `writecf`: `3` ++ Default value for `lockcf`: `0` ### `dynamic-level-bytes` @@ -922,7 +930,7 @@ Configuration items related to `rocksdb.defaultcf` + The default amplification multiple for each layer + Default value: `10` -### `rocksdb.defaultcf.compaction-style` +### `compaction-style` + Compaction method + Available values: `level`, `universal` @@ -947,7 +955,7 @@ Configuration items related to `rocksdb.defaultcf` ## `rocksdb.defaultcf.titan` -Configuration items related to `rocksdb.defaultcf.titan` +Configuration items related to `rocksdb.defaultcf.titan`. ### `min-blob-size` @@ -1004,46 +1012,6 @@ Configuration items related to `rocksdb.defaultcf.titan` + Minimum value: `0` + Unit: KB|MB|GB -## rocksdb.writecf - -Configuration items related to `rocksdb.writecf` - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 15%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `whole-key-filtering` - -+ Determines whether to put the entire key to bloom filter -+ Default value: `false` - -## rocksdb.lockcf - -Configuration items related to `rocksdb.lockcf` - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 2%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `level0-file-num-compaction-trigger` - -+ The number of files at L0 required to trigger compaction -+ Default value: `1` - ## `raftdb` Configuration items related to `raftdb` From 8d9ebc7419a9501d17006796a0abe1a0ea0c18b5 Mon Sep 17 00:00:00 2001 From: tabokie Date: Mon, 12 Jul 2021 20:48:56 +0800 Subject: [PATCH 2/2] fix whole-key-filtering Signed-off-by: tabokie --- tikv-configuration-file.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 609c850d4d350..d609563ca3d1f 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -821,7 +821,8 @@ Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rock ### `whole-key-filtering` + Determines whether to put the entire key to bloom filter -+ Default value: `true` ++ Default value for `defaultcf`and `lockcf`: `true` ++ Default value for `writecf`: `false` ### `bloom-filter-bits-per-key`