From 2827a4ba7debe70262c7f411f246c746e5965b66 Mon Sep 17 00:00:00 2001 From: yikeke Date: Fri, 12 Jun 2020 11:00:57 +0800 Subject: [PATCH 1/3] transaction: : fix some mistakes about transaction isolation --- follower-read.md | 2 +- transaction-overview.md | 8 -------- 2 files changed, 1 insertion(+), 9 deletions(-) diff --git a/follower-read.md b/follower-read.md index 766486d570200..6d7133ea8ad09 100644 --- a/follower-read.md +++ b/follower-read.md @@ -11,7 +11,7 @@ When a read hotspot appears in a Region, the Region leader can become a read bot ## Overview -The Follower Read feature refers to using any follower replica of a Region to serve a read request under the premise of strongly consistent reads. This feature improves the throughput of the TiDB cluster and reduces the load of the leader. It contains a series of load balancing mechanisms that offload TiKV read loads from the leader replica to the follower replica in a Region. TiKV's Follower Read implementation guarantees the consistency of data reading; combined with Snapshot Isolation in TiDB, this implementation provides users with strongly consistent reads. +The Follower Read feature refers to using any follower replica of a Region to serve a read request under the premise of strongly consistent reads. This feature improves the throughput of the TiDB cluster and reduces the load of the leader. It contains a series of load balancing mechanisms that offload TiKV read loads from the leader replica to the follower replica in a Region. TiKV's Follower Read implementation provides users with strongly consistent reads. > **Note:** > diff --git a/transaction-overview.md b/transaction-overview.md index 56dd9294017f9..9ae2c6be06a7e 100644 --- a/transaction-overview.md +++ b/transaction-overview.md @@ -101,14 +101,6 @@ If you set the value of `autocommit` to `1` and start a new transaction through For DDL statements, the transaction is committed automatically and does not support rollback. If you run the DDL statement while the current session is in the process of a transaction, the DDL statement is executed after the current transaction is committed. -## Transaction isolation level - -TiDB **only supports** `SNAPSHOT ISOLATION`. You can set the isolation level of the current session to `READ COMMITTED` using the following statement. However, TiDB is only compatible with the `READ COMMITTED` isolation level in syntax and transactions are still executed at the `SNAPSHOT ISOLATION` level. - -```sql -SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED; -``` - ## Lazy check of constraints **Lazy check** means that by default TiDB will not check [primary key](/constraints.md#primary-key) or [unique constraints](/constraints.md#unique) when an `INSERT` statement is executed, but instead checks when the transaction is committed. In TiDB, the lazy check is performed for values written by ordinary `INSERT` statements. From cb6d27809d36a6d735cac1ecdd150324f9f5f728 Mon Sep 17 00:00:00 2001 From: yikeke Date: Fri, 12 Jun 2020 11:42:33 +0800 Subject: [PATCH 2/3] align https://github.com/pingcap/docs-cn/pull/3165 --- transaction-overview.md | 64 ++++++++++++----------------------------- 1 file changed, 18 insertions(+), 46 deletions(-) diff --git a/transaction-overview.md b/transaction-overview.md index 9ae2c6be06a7e..e78c95ee7bf5e 100644 --- a/transaction-overview.md +++ b/transaction-overview.md @@ -7,9 +7,9 @@ aliases: ['/docs/dev/reference/transactions/overview/'] # Transactions -TiDB supports complete distributed transactions. Both [optimistic transaction model](/optimistic-transaction.md) and [pessimistic transaction model](/pessimistic-transaction.md)(introduced in TiDB 3.0) are available. This document introduces transaction-related statements, explicit and implicit transactions, isolation levels, lazy check for constraints, and transaction sizes. +TiDB supports complete distributed transactions. Both [optimistic transaction model](/optimistic-transaction.md) and [pessimistic transaction model](/pessimistic-transaction.md)(introduced in TiDB 3.0) are available. This document introduces commonly used transaction-related statements, explicit and implicit transactions, isolation levels, lazy check for constraints, and transaction sizes. -The common variables include [`autocommit`](#autocommit), [`tidb_disable_txn_auto_retry`](/tidb-specific-system-variables.md#tidb_disable_txn_auto_retry), and [`tidb_retry_limit`](/tidb-specific-system-variables.md#tidb_retry_limit). +The common variables include [`autocommit`](#autocommit), [`tidb_disable_txn_auto_retry`](/tidb-specific-system-variables.md#tidb_disable_txn_auto_retry), [`tidb_retry_limit`](/tidb-specific-system-variables.md#tidb_retry_limit), and [`tidb_txn_mode`](/tidb-specific-system-variables.md#tidb_txn_mode). ## Common syntax @@ -37,6 +37,10 @@ START TRANSACTION WITH CONSISTENT SNAPSHOT; All of the above three statements are used to start a transaction with the same effect. You can explicitly start a new transaction by executing one of these statements. If the current session is in the process of a transaction when one of these statements is executed, TiDB automatically commits the current transaction before starting a new transaction. +> **Note:** +> +> Unlike MySQL, TiDB takes a snapshot of the current database after executing the above statements. MySQL's `BEGIN` and `START TRANSACTION` takes the snapshot after executing the first `SELECT` statement (not `SELECT FOR UPDATE`) that reads data from InnoDB when a transaction is started. `START TRANSACTION WITH CONSISTENT SNAPSHOT` takes a snapshot during the execution of the statement. As a result, `BEGIN`, `START TRANSACTION` and `START TRANSACTION WITH CONSISTENT SNAPSHOT` are equivalent to `START TRANSACTION WITH CONSISTENT SNAPSHOT` in MySQL. + ### `COMMIT` Syntax: @@ -121,11 +125,12 @@ The lazy check is important because if you perform a unique constraint check on > **Note:** > -> This optimization does not take effect for `INSERT IGNORE` and `INSERT ON DUPLICATE KEY UPDATE`, only for normal `INSERT` statements. The behavior can also be disabled by setting `tidb_constraint_check_in_place=TRUE`. +> + This optimization only takes effect in optimistic transactions. +> + This optimization does not take effect for `INSERT IGNORE` and `INSERT ON DUPLICATE KEY UPDATE`, only for normal `INSERT` statements. The behavior can also be disabled by setting `tidb_constraint_check_in_place=TRUE`. ## Statement rollback -If you execute a statement within a transaction, the statement does not take effect when an error occurs. +TiDB supports atomic rollback after statement execution failure. If you execute a statement within a transaction, the statement does not take effect when an error occurs. ```sql begin; @@ -147,53 +152,20 @@ rollback; In the above example, the second `insert` statement fails, and this transaction does not insert any data into the database because `rollback` is called. -## Transaction sizes +## Transaction size limit -In TiDB, a transaction either too small or too large can impair the overall performance. +Due to the limitations of the underlying storage engine, TiDB requires a single row to be no more than 6 MB. All columns of a row are converted to bytes according to their data types and summed up to estimate the size of a single row. -### Small transactions +TiDB supports both optimistic and pessimistic transactions, and optimistic transactions are the basis for pessimistic transactions. Because optimistic transactions first cache the changes in private memory, TiDB limits the size of a single transaction. -TiDB uses the default autocommit setting (that is, `autocommit = 1`), which automatically issues a commit when executing each SQL statement. Therefore, each of the following three statements is treated as a transaction: - -```sql -UPDATE my_table SET a = 'new_value' WHERE id = 1; -UPDATE my_table SET a = 'newer_value' WHERE id = 2; -UPDATE my_table SET a = 'newest_value' WHERE id = 3; -``` +By default, TiDB sets the total size of a single transaction to no more than 100 MB. You can modify this default value via `txn-total-size-limit` in the configuration file. The maximum value of `txn-total-size-limit` is 10 GB. -In this case, the latency is increased because each statement, as a transaction, uses the two-phase commit which consumes more execution time. +The actual individual transaction size limit also depends on the amount of remaining memory available to the server, because when a transaction is executed, the TiDB process consumes approximately six times more memory than the transaction size. -To improve the execution efficiency, you can use an explicit transaction instead, that is, to execute the above three statements within a transaction: - -```sql -START TRANSACTION; -UPDATE my_table SET a = 'new_value' WHERE id = 1; -UPDATE my_table SET a = 'newer_value' WHERE id = 2; -UPDATE my_table SET a = 'newest_value' WHERE id = 3; -COMMIT; -``` - -Similarly, it is recommended to execute `INSERT` statements within an explicit transaction. +Before v4.0, TiDB restricts the total number of key-value pairs for a single transaction to no more than 300,000. This limitation is removed since v4.0. > **Note:** > -> The single-threaded workloads in TiDB might not fully use TiDB's distributed resources, so the performance of TiDB is lower than that of a single-instance deployment of MySQL. This difference is similar to the case of transactions with higher latency in TiDB. - -### Large transaction - -Due to the requirement of the two-phase commit, a large transaction can lead to the following issues: - -* OOM (Out of Memory) when excessive data is written in the memory -* More conflicts in the prewrite phase -* Long duration before transactions are actually committed - -Therefore, TiDB intentionally imposes some limits on transaction sizes: - -* The total number of SQL statements in a transaction is no more than 5,000 (default) -* Each key-value pair is no more than 6 MB - -For each transaction, it is recommended to keep the number of SQL statements between 100 to 500 to achieve an optimal performance. - -TiDB sets a default limit of 100 MB for the total size of key-value pairs, which can be modified by the `txn-total-size-limit` configuration item in the configuration file. The maximum value of `txn-total-size-limit` is 10 GB. The actual size limit of one transaction also depends on the memory capacity. When executing large transactions, the memory usage of the TiDB process is approximately 6 times larger than the total size of transactions. - -In versions earlier than 4.0, TiDB limits the total number of key-value pairs for a single transaction to no more than 300,000. This limitation is removed since v4.0. +> Usually, TiDB Binlog is enabled to replicate data to the downstream. In some scenarios, message middleware such as Kafka is used to consume binlogs that are replicated to the downstream. +> +> Taking Kafka as an example, the upper limit of Kafka's single message processing capability is 1 GB. Therefore, when `txn-total-size-limit` is set to more than 1 GB, it might happen that the transaction is successfully executed in TiDB, but the downstream Kafka reports an error. In order to avoid this situation, you need to decide the actual value of `txn-total-size-limit` according to the limit of the end consumer. For example, if Kafka is used downstream, `txn-total-size-limit` must not exceed 1 GB. From df26936e682cc5c8de6cf55bb28474be99a443a1 Mon Sep 17 00:00:00 2001 From: Keke Yi <40977455+yikeke@users.noreply.github.com> Date: Fri, 12 Jun 2020 12:29:19 +0800 Subject: [PATCH 3/3] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- transaction-overview.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/transaction-overview.md b/transaction-overview.md index e78c95ee7bf5e..ddb89989beac0 100644 --- a/transaction-overview.md +++ b/transaction-overview.md @@ -39,7 +39,7 @@ All of the above three statements are used to start a transaction with the same > **Note:** > -> Unlike MySQL, TiDB takes a snapshot of the current database after executing the above statements. MySQL's `BEGIN` and `START TRANSACTION` takes the snapshot after executing the first `SELECT` statement (not `SELECT FOR UPDATE`) that reads data from InnoDB when a transaction is started. `START TRANSACTION WITH CONSISTENT SNAPSHOT` takes a snapshot during the execution of the statement. As a result, `BEGIN`, `START TRANSACTION` and `START TRANSACTION WITH CONSISTENT SNAPSHOT` are equivalent to `START TRANSACTION WITH CONSISTENT SNAPSHOT` in MySQL. +> Unlike MySQL, TiDB takes a snapshot of the current database after executing the statements above. MySQL's `BEGIN` and `START TRANSACTION` take a snapshot after executing the first `SELECT` statement (not `SELECT FOR UPDATE`) that reads data from InnoDB after a transaction is started. `START TRANSACTION WITH CONSISTENT SNAPSHOT` takes a snapshot during the execution of the statement. As a result, `BEGIN`, `START TRANSACTION`, and `START TRANSACTION WITH CONSISTENT SNAPSHOT` are equivalent to `START TRANSACTION WITH CONSISTENT SNAPSHOT` in MySQL. ### `COMMIT` @@ -126,7 +126,7 @@ The lazy check is important because if you perform a unique constraint check on > **Note:** > > + This optimization only takes effect in optimistic transactions. -> + This optimization does not take effect for `INSERT IGNORE` and `INSERT ON DUPLICATE KEY UPDATE`, only for normal `INSERT` statements. The behavior can also be disabled by setting `tidb_constraint_check_in_place=TRUE`. +> + This optimization does not take effect for `INSERT IGNORE` and `INSERT ON DUPLICATE KEY UPDATE`, but only for normal `INSERT` statements. The behavior can also be disabled by setting `tidb_constraint_check_in_place=TRUE`. ## Statement rollback @@ -160,7 +160,7 @@ TiDB supports both optimistic and pessimistic transactions, and optimistic trans By default, TiDB sets the total size of a single transaction to no more than 100 MB. You can modify this default value via `txn-total-size-limit` in the configuration file. The maximum value of `txn-total-size-limit` is 10 GB. -The actual individual transaction size limit also depends on the amount of remaining memory available to the server, because when a transaction is executed, the TiDB process consumes approximately six times more memory than the transaction size. +The actual individual transaction size limit also depends on the amount of remaining memory available to the server, because when a transaction is executed, the memory usage of the TiDB process is approximately six times larger than the total size of transactions. Before v4.0, TiDB restricts the total number of key-value pairs for a single transaction to no more than 300,000. This limitation is removed since v4.0. @@ -168,4 +168,4 @@ Before v4.0, TiDB restricts the total number of key-value pairs for a single tra > > Usually, TiDB Binlog is enabled to replicate data to the downstream. In some scenarios, message middleware such as Kafka is used to consume binlogs that are replicated to the downstream. > -> Taking Kafka as an example, the upper limit of Kafka's single message processing capability is 1 GB. Therefore, when `txn-total-size-limit` is set to more than 1 GB, it might happen that the transaction is successfully executed in TiDB, but the downstream Kafka reports an error. In order to avoid this situation, you need to decide the actual value of `txn-total-size-limit` according to the limit of the end consumer. For example, if Kafka is used downstream, `txn-total-size-limit` must not exceed 1 GB. +> Taking Kafka as an example, the upper limit of Kafka's single message processing capability is 1 GB. Therefore, when `txn-total-size-limit` is set to more than 1 GB, it might happen that the transaction is successfully executed in TiDB, but the downstream Kafka reports an error. To avoid this situation, you need to decide the actual value of `txn-total-size-limit` according to the limit of the end consumer. For example, if Kafka is used downstream, `txn-total-size-limit` must not exceed 1 GB.