From 0c8e8c6a46875437bdd7cdf067b355bb39733feb Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 Date: Thu, 20 May 2021 16:04:12 +0800 Subject: [PATCH 1/9] update TiDB 6MB limit --- error-codes.md | 4 ++-- faq/migration-tidb-faq.md | 2 +- storage-engine/titan-overview.md | 2 +- tidb-limitations.md | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/error-codes.md b/error-codes.md index 02b108b8f1b6a..98c18b472ca74 100644 --- a/error-codes.md +++ b/error-codes.md @@ -88,9 +88,9 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the * Error Number: 8025 - The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB. + The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB by default is 6 MB. - If a pair exceeds this limit, you need to manually deal with this row of data to meet the 6 MB limit. + If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) to ease the restrictions. * Error Number: 8026 diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index 290e1598f1307..9d1dab0a199c1 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) and the maximum value is 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/storage-engine/titan-overview.md b/storage-engine/titan-overview.md index db4a9fed9835b..8969385641793 100644 --- a/storage-engine/titan-overview.md +++ b/storage-engine/titan-overview.md @@ -27,7 +27,7 @@ Titan is suitable for the scenarios where a huge volume of data is written to th The prerequisites for enabling Titan are as follows: -- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 6 MB due to the limitation of the TiKV Raft layer. +- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the configuration items in [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) to ease the restrictions. - No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB. - Sufficient disk space, because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB. diff --git a/tidb-limitations.md b/tidb-limitations.md index d3eea90084627..bd538aaf06ab8 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB | +| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) | ## Limitation on a single column From b47f75ece31c20f517b2217ef8bc3848a46d242a Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 Date: Thu, 20 May 2021 16:11:11 +0800 Subject: [PATCH 2/9] add an updated file --- best-practices/tidb-best-practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index 87b0b4ccb7831..de1411c85b066 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -53,7 +53,7 @@ TiDB provides complete distributed transactions and the model has some optimizat As distributed transactions need to conduct two-phase commit and the bottom layer performs Raft replication, if a transaction is very large, the commit process would be quite slow, and the following Raft replication process is thus stuck. To avoid this problem, the transaction size is limited: - A transaction is limited to 5,000 SQL statements (by default) - - Each Key-Value entry is no more than 6 MB + - Each Key-Value entry is no more than 6 MB (by default) - The total size of Key-Value entries is no more than 10 GB. You can find similar limits in [Google Cloud Spanner](https://cloud.google.com/spanner/quotas). From feea5f629cf4c10b04f2669d7b9937ecf45b222e Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 Date: Thu, 20 May 2021 16:18:32 +0800 Subject: [PATCH 3/9] fix the dead anchor --- error-codes.md | 2 +- faq/migration-tidb-faq.md | 2 +- tidb-limitations.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/error-codes.md b/error-codes.md index 98c18b472ca74..e97f5cb680c13 100644 --- a/error-codes.md +++ b/error-codes.md @@ -90,7 +90,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB by default is 6 MB. - If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) to ease the restrictions. + If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) to ease the restrictions. * Error Number: 8026 diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index 9d1dab0a199c1..b6e7efb8c8972 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) and the maximum value is 120MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) and the maximum value is 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/tidb-limitations.md b/tidb-limitations.md index bd538aaf06ab8..9a9c256f0716b 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-New-in-v5.0) | +| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) | ## Limitation on a single column From 9a5a51870d0b9abd3aeacfcea131ff0c038d1699 Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 Date: Thu, 20 May 2021 16:23:01 +0800 Subject: [PATCH 4/9] fix the dead anchors --- error-codes.md | 2 +- faq/migration-tidb-faq.md | 2 +- tidb-limitations.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/error-codes.md b/error-codes.md index e97f5cb680c13..8aef3f62490c9 100644 --- a/error-codes.md +++ b/error-codes.md @@ -90,7 +90,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB by default is 6 MB. - If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) to ease the restrictions. + If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) to ease the restrictions. * Error Number: 8026 diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index b6e7efb8c8972..41451552c8353 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) and the maximum value is 120MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) and the maximum value is 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/tidb-limitations.md b/tidb-limitations.md index 9a9c256f0716b..9aafc2613a900 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v5.0) | +| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) | ## Limitation on a single column From f0e6435578e8f25e1fd1d72abfb74e82d1da16f9 Mon Sep 17 00:00:00 2001 From: Xiaozhen Liu <82579298+Liuxiaozhen12@users.noreply.github.com> Date: Fri, 21 May 2021 10:55:28 +0800 Subject: [PATCH 5/9] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- error-codes.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/error-codes.md b/error-codes.md index 8aef3f62490c9..0ee5bb49f1293 100644 --- a/error-codes.md +++ b/error-codes.md @@ -88,9 +88,9 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the * Error Number: 8025 - The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB by default is 6 MB. + The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB by default. - If a pair exceeds this limit, you need to properly adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) to ease the restrictions. + If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item to relax the limit. * Error Number: 8026 From a707515b946381d8870c82ac77df8bf9129f9f3b Mon Sep 17 00:00:00 2001 From: Liuxiaozhen12 Date: Fri, 21 May 2021 14:18:15 +0800 Subject: [PATCH 6/9] Apply suggestions from code review --- faq/migration-tidb-faq.md | 2 +- storage-engine/titan-overview.md | 2 +- tidb-limitations.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index 41451552c8353..862ded39fd46f 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the configuration items in the configuration file [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) and the maximum value is 120MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item and the maximum value is 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/storage-engine/titan-overview.md b/storage-engine/titan-overview.md index 8969385641793..3c5299f226871 100644 --- a/storage-engine/titan-overview.md +++ b/storage-engine/titan-overview.md @@ -27,7 +27,7 @@ Titan is suitable for the scenarios where a huge volume of data is written to th The prerequisites for enabling Titan are as follows: -- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the configuration items in [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) to ease the restrictions. +- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration item to relax the limit. - No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB. - Sufficient disk space, because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB. diff --git a/tidb-limitations.md b/tidb-limitations.md index 9aafc2613a900..7a8ad9ab3ba25 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default, you can adjust the configuration items in [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) | +| Size | 6 MB by default, you can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item | ## Limitation on a single column From 1d1b91110c0d606ccd863273eab7994d03f34b37 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 1 Jun 2021 16:27:08 +0800 Subject: [PATCH 7/9] Apply suggestions from code review --- error-codes.md | 2 +- faq/migration-tidb-faq.md | 2 +- storage-engine/titan-overview.md | 2 +- tidb-limitations.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/error-codes.md b/error-codes.md index 0ee5bb49f1293..0bb7b4e1980b0 100644 --- a/error-codes.md +++ b/error-codes.md @@ -90,7 +90,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB by default. - If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item to relax the limit. + If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value to relax the limit. * Error Number: 8026 diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index 862ded39fd46f..bd628f1e1e41b 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item and the maximum value is 120MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value up to 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/storage-engine/titan-overview.md b/storage-engine/titan-overview.md index 3c5299f226871..8f6e698d42340 100644 --- a/storage-engine/titan-overview.md +++ b/storage-engine/titan-overview.md @@ -27,7 +27,7 @@ Titan is suitable for the scenarios where a huge volume of data is written to th The prerequisites for enabling Titan are as follows: -- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration item to relax the limit. +- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration value to relax the limit. - No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB. - Sufficient disk space, because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB. diff --git a/tidb-limitations.md b/tidb-limitations.md index 7a8ad9ab3ba25..e95de75d33473 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default, you can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item | +| Size | 6 MB by default. You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item. | ## Limitation on a single column From 4ef5a98f24a24623eddeaccfe1d708e0f0f25235 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 1 Jun 2021 16:32:17 +0800 Subject: [PATCH 8/9] Update storage-engine/titan-overview.md Co-authored-by: Lei Zhao --- storage-engine/titan-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/storage-engine/titan-overview.md b/storage-engine/titan-overview.md index 8f6e698d42340..d1fa7ecac018b 100644 --- a/storage-engine/titan-overview.md +++ b/storage-engine/titan-overview.md @@ -27,7 +27,7 @@ Titan is suitable for the scenarios where a huge volume of data is written to th The prerequisites for enabling Titan are as follows: -- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration value to relax the limit. +- The average size of values is large, or the size of all large values accounts for much of the total value size. Currently, the size of a value greater than 1 KB is considered as a large value. In some situations, this number (1 KB) can be 512 B. Note that a single value written to TiKV cannot exceed 8 MB due to the limitation of the TiKV Raft layer. You can adjust the [`raft-entry-max-size`](/tikv-configuration-file.md#raft-entry-max-size) configuration value to relax the limit. - No range query will be performed or you do not need a high performance of range query. Because the data stored in Titan is not well-ordered, its performance of range query is poorer than that of RocksDB, especially for the query of a large range. According PingCAP's internal test, Titan's range query performance is 40% to a few times lower than that of RocksDB. - Sufficient disk space, because Titan reduces write amplification at the cost of disk space. In addition, Titan compresses values one by one, and its compression rate is lower than that of RocksDB. RocksDB compresses blocks one by one. Therefore, Titan consumes more storage space than RocksDB, which is expected and normal. In some situations, Titan's storage consumption can be twice that of RocksDB. From bd562322ee36e489b0e2abd7342ce9cd141b910d Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 1 Jun 2021 16:40:14 +0800 Subject: [PATCH 9/9] Apply suggestions from code review --- error-codes.md | 2 +- faq/migration-tidb-faq.md | 2 +- tidb-limitations.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/error-codes.md b/error-codes.md index 0bb7b4e1980b0..f37697d88e27d 100644 --- a/error-codes.md +++ b/error-codes.md @@ -90,7 +90,7 @@ TiDB is compatible with the error codes in MySQL, and in most cases returns the The single Key-Value pair being written is too large. The largest single Key-Value pair supported in TiDB is 6 MB by default. - If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value to relax the limit. + If a pair exceeds this limit, you need to properly adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v4010) configuration value to relax the limit. * Error Number: 8026 diff --git a/faq/migration-tidb-faq.md b/faq/migration-tidb-faq.md index bd628f1e1e41b..996af3eb42195 100644 --- a/faq/migration-tidb-faq.md +++ b/faq/migration-tidb-faq.md @@ -184,7 +184,7 @@ The total read capacity has no limit. You can increase the read capacity by addi ### The error message `transaction too large` is displayed -Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration value up to 120MB. +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. You can adjust the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v4010) configuration value up to 120MB. Distributed transactions need two-phase commit and the bottom layer performs the Raft replication. If a transaction is very large, the commit process would be quite slow and the write conflict is more likely to occur. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, modify the value of `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to 10G. The actual limitation is also affected by the physical memory of the machine. diff --git a/tidb-limitations.md b/tidb-limitations.md index e95de75d33473..403a8bd388ddd 100644 --- a/tidb-limitations.md +++ b/tidb-limitations.md @@ -48,7 +48,7 @@ This document describes the common usage limitations of TiDB, including the maxi | Type | Upper limit | |:----------|:----------| -| Size | 6 MB by default. You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v50) configuration item. | +| Size | 6 MB by default. You can adjust the size limit via the [`txn-entry-size-limit`](/tidb-configuration-file.md#txn-entry-size-limit-new-in-v4010) configuration item. | ## Limitation on a single column