From 00c6bacf859482acb2a23502ae497920554b7684 Mon Sep 17 00:00:00 2001 From: tiancaiamao Date: Wed, 26 Feb 2020 21:11:42 +0800 Subject: [PATCH 1/5] faq/tidb.md: update the wording about transaction size --- faq/tidb.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/faq/tidb.md b/faq/tidb.md index 9b23caaeed14a..154b28cf2558d 100644 --- a/faq/tidb.md +++ b/faq/tidb.md @@ -930,14 +930,15 @@ The total read capacity has no limit. You can increase the read capacity by addi #### The error message `transaction too large` is displayed -As distributed transactions need to conduct two-phase commit and the bottom layer performs Raft replication, if a transaction is very large, the commit process would be quite slow and the following Raft replication flow is thus struck. To avoid this problem, we limit the transaction size: +As distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and it's more likely to conflict with other transactions. Morever, the rollback of transactions lead to unnecessary performance penalty. To avoid this problem, we limit the transaction size: - A transaction is limited to 5000 SQL statements (by default) - Each Key-Value entry is no more than 6MB -- The total size of Key-Value entry is no more than 100MB There are [similar limits](https://cloud.google.com/spanner/docs/limits) on Google Cloud Spanner. +By default we limit the total size of Key-Value entries to no more than 100MB, if you need larger transactions, you can change `txn-total-size-limit` in the TiDB configuration file, the maximum value could be up to 10G, do not forget the actual limitation may also depend on the hardware of the RAM. + #### How to import data in batches? When you import data, insert in batches and keep the number of rows within 10,000 for each batch. From e8a653520f2d0f1393cc42d0f06391947529925c Mon Sep 17 00:00:00 2001 From: tiancaiamao Date: Thu, 27 Feb 2020 15:56:47 +0800 Subject: [PATCH 2/5] Apply suggestions from code review Co-Authored-By: TomShawn <41534398+TomShawn@users.noreply.github.com> --- faq/tidb.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/faq/tidb.md b/faq/tidb.md index 154b28cf2558d..42cb19539a281 100644 --- a/faq/tidb.md +++ b/faq/tidb.md @@ -930,14 +930,14 @@ The total read capacity has no limit. You can increase the read capacity by addi #### The error message `transaction too large` is displayed -As distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and it's more likely to conflict with other transactions. Morever, the rollback of transactions lead to unnecessary performance penalty. To avoid this problem, we limit the transaction size: +Distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and this transaction is more likely to conflict with other transactions. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the transaction size: - A transaction is limited to 5000 SQL statements (by default) - Each Key-Value entry is no more than 6MB There are [similar limits](https://cloud.google.com/spanner/docs/limits) on Google Cloud Spanner. -By default we limit the total size of Key-Value entries to no more than 100MB, if you need larger transactions, you can change `txn-total-size-limit` in the TiDB configuration file, the maximum value could be up to 10G, do not forget the actual limitation may also depend on the hardware of the RAM. +By default, we limit the total size of key-value entries to no more than 100MB. If you need larger transactions, change `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to `10G`. The actual limitation might also depend on the hardware of the RAM. #### How to import data in batches? From 2bafee45db4ece270a2f519c4070eefa00c110c9 Mon Sep 17 00:00:00 2001 From: tiancaiamao Date: Thu, 27 Feb 2020 16:09:37 +0800 Subject: [PATCH 3/5] address comment --- faq/tidb.md | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/faq/tidb.md b/faq/tidb.md index 42cb19539a281..a7949452c7d23 100644 --- a/faq/tidb.md +++ b/faq/tidb.md @@ -930,15 +930,12 @@ The total read capacity has no limit. You can increase the read capacity by addi #### The error message `transaction too large` is displayed -Distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and this transaction is more likely to conflict with other transactions. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the transaction size: +Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. -- A transaction is limited to 5000 SQL statements (by default) -- Each Key-Value entry is no more than 6MB +Distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and this transaction is more likely to conflict with other transactions. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the default transaction total size of key-value entries to no more than 100MB. If you need larger transactions, change `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to `10G`. The actual limitation might also depend on the hardware of the RAM. There are [similar limits](https://cloud.google.com/spanner/docs/limits) on Google Cloud Spanner. -By default, we limit the total size of key-value entries to no more than 100MB. If you need larger transactions, change `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to `10G`. The actual limitation might also depend on the hardware of the RAM. - #### How to import data in batches? When you import data, insert in batches and keep the number of rows within 10,000 for each batch. From 3ace40f3459a3f48e2f9e8a6c40762446fa80637 Mon Sep 17 00:00:00 2001 From: tiancaiamao Date: Thu, 27 Feb 2020 16:33:48 +0800 Subject: [PATCH 4/5] address comment --- faq/tidb.md | 4 ---- 1 file changed, 4 deletions(-) diff --git a/faq/tidb.md b/faq/tidb.md index a7949452c7d23..254fd7c5a237d 100644 --- a/faq/tidb.md +++ b/faq/tidb.md @@ -88,10 +88,6 @@ The character sets of TiDB use UTF-8 by default and currently only support UTF-8 32 characters at most. -#### What is the maximum number of statements in a transaction? - -5000 at most. - #### Does TiDB support XA? No. The JDBC driver of TiDB is MySQL JDBC (Connector/J). When using Atomikos, set the data source to `type="com.mysql.jdbc.jdbc2.optional.MysqlXADataSource"`. TiDB does not support the connection with MySQL JDBC XADataSource. MySQL JDBC XADataSource only works for MySQL (for example, using DML to modify the `redo` log). From 56d198da6ec227be27813b1a01dc89b252126dff Mon Sep 17 00:00:00 2001 From: tiancaiamao Date: Fri, 28 Feb 2020 21:52:02 +0800 Subject: [PATCH 5/5] Update faq/tidb.md Co-Authored-By: TomShawn <41534398+TomShawn@users.noreply.github.com> --- faq/tidb.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/faq/tidb.md b/faq/tidb.md index 254fd7c5a237d..4b28a43090138 100644 --- a/faq/tidb.md +++ b/faq/tidb.md @@ -928,7 +928,7 @@ The total read capacity has no limit. You can increase the read capacity by addi Due to the limitation of the underlying storage engine, each key-value entry (one row) in TiDB should be no more than 6MB. -Distributed transactions need to conduct two-phase commit and the underlying implementation performs Raft replication. If a transaction is very large, the commit process would be quite slow and this transaction is more likely to conflict with other transactions. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the default transaction total size of key-value entries to no more than 100MB. If you need larger transactions, change `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to `10G`. The actual limitation might also depend on the hardware of the RAM. +Distributed transactions need two-phase commit and the underlying implementation performs the Raft replication. If a transaction is very large, the commit process would be quite slow and this transaction is more likely to conflict with other transactions. Moreover, the rollback of a failed transaction leads to an unnecessary performance penalty. To avoid these problems, we limit the total size of key-value entries to no more than 100MB in a transaction by default. If you need larger transactions, change `txn-total-size-limit` in the TiDB configuration file. The maximum value of this configuration item is up to `10G`. The actual limitation might also depend on the hardware of the RAM. There are [similar limits](https://cloud.google.com/spanner/docs/limits) on Google Cloud Spanner.