From 62f8e603eea29dd6f2defbe6b0f16f75e8ab4704 Mon Sep 17 00:00:00 2001 From: Xinye Tao Date: Mon, 28 Jun 2021 10:23:25 +0800 Subject: [PATCH 1/2] Overhauling TiKV RocksDB configuration file (#5746) Signed-off-by: tabokie --- tikv-configuration-file.md | 80 ++++++++++++-------------------------- 1 file changed, 24 insertions(+), 56 deletions(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 3b800f6ffcea4..4ccab86c6b8cc 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -798,21 +798,24 @@ Configuration items related to Titan + Default value: `1` + Minimum value: `1` -## rocksdb.defaultcf +## rocksdb.defaultcf | rocksdb.writecf | rocksdb.lockcf -Configuration items related to `rocksdb.defaultcf` +Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rocksdb.lockcf`. ### `block-size` + The default size of a RocksDB block -+ Default value: `64KB` -+ Minimum value: `1KB` ++ Default value for `defaultcf` and `writecf`: `"64KB"` ++ Default value for `lockcf`: `"16KB"` ++ Minimum value: `"1KB"` + Unit: KB|MB|GB ### `block-cache-size` + The cache size of a RocksDB block -+ Default value: `Total machine memory / 4` ++ Default value for `defaultcf`: `Total machine memory * 25%` ++ Default value for `writecf`: `Total machine memory * 15%` ++ Default value for `lockcf`: `Total machine memory * 2%` + Minimum value: `0` + Unit: KB|MB|GB @@ -839,9 +842,10 @@ Configuration items related to `rocksdb.defaultcf` ### `optimize-filters-for-hits` + Determines whether to optimize the hit ratio of filters -+ Default value: `true` ++ Default value for `defaultcf`: `true` ++ Default value for `writecf` and `lockcf`: `false` -### `whole_key_filtering` +### `whole-key-filtering` + Determines whether to put the entire key to bloom filter + Default value: `true` @@ -867,13 +871,14 @@ Configuration items related to `rocksdb.defaultcf` ### `compression-per-level` + The default compression algorithm for each level -+ Available values: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] -+ Default value: `No` for the first two levels, and `lz4` for the next five levels ++ Optional values: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] ++ Default value for `defaultcf` and `writecf`: ["no", "no", "lz4", "lz4", "lz4", "zstd", "zstd"] ++ Default value for `lockcf`: ["no", "no", "no", "no", "no", "no", "no"] ### `write-buffer-size` + Memtable size -+ Default value: `128MB` ++ Default value: `"128MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -892,7 +897,8 @@ Configuration items related to `rocksdb.defaultcf` ### `max-bytes-for-level-base` + The maximum number of bytes at base level (L1). Generally, it is set to 4 times the size of a memtable. -+ Default value: `512MB` ++ Default value for `defaultcf` and `writecf`: `"512MB"` ++ Default value for `lockcf`: `"128MB"` + Minimum value: `0` + Unit: KB|MB|GB @@ -906,7 +912,8 @@ Configuration items related to `rocksdb.defaultcf` ### `level0-file-num-compaction-trigger` + The maximum number of files at L0 that trigger compaction -+ Default value: `4` ++ Default value for `defaultcf` and `writecf`: `4` ++ Default value for `lockcf`: `1` + Minimum value: `0` ### `level0-slowdown-writes-trigger` @@ -931,8 +938,9 @@ Configuration items related to `rocksdb.defaultcf` ### `compaction-pri` + The priority type of compaction -+ Available values: `3` (`MinOverlappingRatio`), `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`) -+ Default value: `3` ++ Optional values: `0` (`ByCompensatedSize`), `1` (`OldestLargestSeqFirst`), `2` (`OldestSmallestSeqFirst`), `3` (`MinOverlappingRatio`) ++ Default value for `defaultcf` and `writecf`: `3` ++ Default value for `lockcf`: `0` ### `dynamic-level-bytes` @@ -949,7 +957,7 @@ Configuration items related to `rocksdb.defaultcf` + The default amplification multiple for each layer + Default value: `10` -### `rocksdb.defaultcf.compaction-style` +### `compaction-style` + Compaction method + Available values: `level`, `universal` @@ -974,7 +982,7 @@ Configuration items related to `rocksdb.defaultcf` ## `rocksdb.defaultcf.titan` -Configuration items related to `rocksdb.defaultcf.titan` +Configuration items related to `rocksdb.defaultcf.titan`. ### `min-blob-size` @@ -1031,46 +1039,6 @@ Configuration items related to `rocksdb.defaultcf.titan` + Minimum value: `0` + Unit: KB|MB|GB -## rocksdb.writecf - -Configuration items related to `rocksdb.writecf` - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 15%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `whole-key-filtering` - -+ Determines whether to put the entire key to bloom filter -+ Default value: `false` - -## rocksdb.lockcf - -Configuration items related to `rocksdb.lockcf` - -### `block-cache-size` - -+ Block cache size -+ Default value: `Total machine memory * 2%` -+ Unit: MB|GB - -### `optimize-filters-for-hits` - -+ Determines whether to optimize the hit ratio of the filter -+ Default value: `false` - -### `level0-file-num-compaction-trigger` - -+ The number of files at L0 required to trigger compaction -+ Default value: `1` - ## `raftdb` Configuration items related to `raftdb` From 7c189010d0692761979b3aef188e3e73f2547427 Mon Sep 17 00:00:00 2001 From: tabokie Date: Mon, 12 Jul 2021 20:48:56 +0800 Subject: [PATCH 2/2] fix whole-key-filtering Signed-off-by: tabokie --- tikv-configuration-file.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tikv-configuration-file.md b/tikv-configuration-file.md index 4ccab86c6b8cc..a9e3f4ac9bf6b 100644 --- a/tikv-configuration-file.md +++ b/tikv-configuration-file.md @@ -848,7 +848,8 @@ Configuration items related to `rocksdb.defaultcf`, `rocksdb.writecf`, and `rock ### `whole-key-filtering` + Determines whether to put the entire key to bloom filter -+ Default value: `true` ++ Default value for `defaultcf`and `lockcf`: `true` ++ Default value for `writecf`: `false` ### `bloom-filter-bits-per-key`