From 865d75d12173cb5633420634c7b11c74a1b82160 Mon Sep 17 00:00:00 2001 From: ekexium Date: Tue, 18 Oct 2022 11:18:51 +0800 Subject: [PATCH 1/4] add the variables of batch-dml Signed-off-by: ekexium --- system-variables.md | 41 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) diff --git a/system-variables.md b/system-variables.md index 1de928399cf13..908c0174db780 100644 --- a/system-variables.md +++ b/system-variables.md @@ -732,6 +732,33 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a In the case of a poor network environment, appropriately increasing the value of this variable can effectively alleviate error reporting to the application end caused by timeout. If the application end wants to receive the error information more quickly, minimize the value of this variable. +### tidb_batch_commit +- Scope: Session +- Persists to cluster: No +- Type: Boolean +- Default value: `OFF` +- This variable has been deprecated and does not take effect any more. It was used to control whether to enable the batch commit feature. When truned on, transactions may be split into multiple transactions and be committed non-atomically. + +### tidb_batch_delete +- Scope: Session +- Persists to cluster: No +- Type: Boolean +- Default value: `OFF` +- This variable is used to control whether to enable the batch delete feature, which is part of the deprecated batch-dml feature. When truned on, delete statements may be split into multiple transactions and be committed non-atomically. You must also turn on `tidb_enable_batch_dml` and set a positive `tidb_dml_batch_size` to make it work. + +> **Warning:** +> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. + +### tidb_batch_insert +- Scope: Session +- Persists to cluster: No +- Type: Boolean +- Default value: `OFF` +- This variable is used to control whether to enable the batch insert feature, which is part of the deprecated batch-dml feature. When truned on, insert statements may be split into multiple transactions and be committed non-atomically. You must also turn on `tidb_enable_batch_dml` and set a positive `tidb_dml_batch_size` to make it work. + +> **Warning:** +> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. + ### tidb_batch_pending_tiflash_count New in v6.0 - Scope: SESSION | GLOBAL @@ -979,6 +1006,10 @@ Constraint checking is always performed in place for pessimistic transactions (d - Unit: Rows - When this value is greater than `0`, TiDB will batch commit statements such as `INSERT` or `LOAD DATA` into smaller transactions. This reduces memory usage and helps ensure that the `txn-total-size-limit` is not reached by bulk modifications. - Only the value `0` provides ACID compliance. Setting this to any other value will break the atomicity and isolation guarantees of TiDB. +- You must also turn on `tidb_enable_batch_dml` and at least one of `tidb_batch_insert` and `tidb_batch_delete` to use this feature. + +> **Warning:** +> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. ### tidb_enable_1pc New in v5.0 @@ -1054,6 +1085,16 @@ Constraint checking is always performed in place for pessimistic transactions (d - Default value: `OFF` - This variable is used to determine whether to include the `AUTO_INCREMENT` columns when creating a generated column or an expression index. +### tidb_enable_batch_dml +- Scope: GLOBAL +- Persists to cluster: Yes +- Type: Boolean +- Default values: `OFF` +- This variable contronls whether to enable the deprecated batch-dml feature. When it's enabled, certain statements may be split into multiple transactions, which means it's non-atomic and should be used with care. When using the feature you must ensure there is no concurrent opertaions on the data you are operating on. You must also specify a positive `tidb_batch_dml_size` and turn on at least one of `tidb_batch_insert` and `tidb_batch_delete` to make it work. + +> **Warning:** +> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. + ### tidb_enable_cascades_planner > **Warning:** From fe63df7275c74c43e24b9fcb24689ef2401655a4 Mon Sep 17 00:00:00 2001 From: ekexium Date: Tue, 18 Oct 2022 15:35:32 +0800 Subject: [PATCH 2/4] fix the description of tidb_batch_commit Signed-off-by: ekexium --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index 908c0174db780..1fe700484cfc6 100644 --- a/system-variables.md +++ b/system-variables.md @@ -737,7 +737,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Persists to cluster: No - Type: Boolean - Default value: `OFF` -- This variable has been deprecated and does not take effect any more. It was used to control whether to enable the batch commit feature. When truned on, transactions may be split into multiple transactions and be committed non-atomically. +- The variable is used to control whether to enable the deprecated batch commit feature. When truned on, a transaction may be split into multiple transactions by grouping every a few statements and be committed non-atomically. It is not recommended to turn on this feature. ### tidb_batch_delete - Scope: Session From d42fbaa70d826ceed548dbef3cd291792d538b5b Mon Sep 17 00:00:00 2001 From: ekexium Date: Wed, 19 Oct 2022 10:47:28 +0800 Subject: [PATCH 3/4] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- system-variables.md | 37 ++++++++++++++++++++++++------------- 1 file changed, 24 insertions(+), 13 deletions(-) diff --git a/system-variables.md b/system-variables.md index 1fe700484cfc6..2ec7deb9edf72 100644 --- a/system-variables.md +++ b/system-variables.md @@ -733,31 +733,40 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a In the case of a poor network environment, appropriately increasing the value of this variable can effectively alleviate error reporting to the application end caused by timeout. If the application end wants to receive the error information more quickly, minimize the value of this variable. ### tidb_batch_commit + +> **Warning:** +> +> It is **NOT** recommended to enable this variable. + - Scope: Session - Persists to cluster: No - Type: Boolean - Default value: `OFF` -- The variable is used to control whether to enable the deprecated batch commit feature. When truned on, a transaction may be split into multiple transactions by grouping every a few statements and be committed non-atomically. It is not recommended to turn on this feature. +- The variable is used to control whether to enable the deprecated batch-commit feature. When this variable is enabled, a transaction might be split into multiple transactions by grouping a few statements and committed non-atomically, which is not recommended. ### tidb_batch_delete + +> **Warning:** +> +> This variable is associated with the deprecated batch-dml feature, which might cause data corruption. Therefore, it is not recommended to enable this variable for batch-dml. Instead, use [non-transactional DML](/non-transactional-dml.md). + - Scope: Session - Persists to cluster: No - Type: Boolean - Default value: `OFF` -- This variable is used to control whether to enable the batch delete feature, which is part of the deprecated batch-dml feature. When truned on, delete statements may be split into multiple transactions and be committed non-atomically. You must also turn on `tidb_enable_batch_dml` and set a positive `tidb_dml_batch_size` to make it work. +- This variable is used to control whether to enable the batch-delete feature, which is a part of the deprecated batch-dml feature. When this variable is enabled, `DELETE` statements might be split into multiple transactions and committed non-atomically. To make it work, you also need to enable `tidb_enable_batch_dml` and set a positive value for `tidb_dml_batch_size`, which is not recommended. + +### tidb_batch_insert > **Warning:** -> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. +> +> This variable is associated with the deprecated batch-dml feature, which might cause data corruption. Therefore, it is not recommended to enable this variable for batch-dml. Instead, use [non-transactional DML](/non-transactional-dml.md). -### tidb_batch_insert - Scope: Session - Persists to cluster: No - Type: Boolean - Default value: `OFF` -- This variable is used to control whether to enable the batch insert feature, which is part of the deprecated batch-dml feature. When truned on, insert statements may be split into multiple transactions and be committed non-atomically. You must also turn on `tidb_enable_batch_dml` and set a positive `tidb_dml_batch_size` to make it work. - -> **Warning:** -> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. +- This variable is used to control whether to enable the batch-insert feature, which is a part of the deprecated batch-dml feature. When this variable is enabled, `INSERT` statements might be split into multiple transactions and committed non-atomically. To make it work, you also need to enable `tidb_enable_batch_dml` and set a positive for `tidb_dml_batch_size`, which is not recommended. ### tidb_batch_pending_tiflash_count New in v6.0 @@ -1006,7 +1015,7 @@ Constraint checking is always performed in place for pessimistic transactions (d - Unit: Rows - When this value is greater than `0`, TiDB will batch commit statements such as `INSERT` or `LOAD DATA` into smaller transactions. This reduces memory usage and helps ensure that the `txn-total-size-limit` is not reached by bulk modifications. - Only the value `0` provides ACID compliance. Setting this to any other value will break the atomicity and isolation guarantees of TiDB. -- You must also turn on `tidb_enable_batch_dml` and at least one of `tidb_batch_insert` and `tidb_batch_delete` to use this feature. +- To make this variable work, you also need to enable `tidb_enable_batch_dml` and at least one of `tidb_batch_insert` and `tidb_batch_delete`. > **Warning:** > batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. @@ -1086,14 +1095,16 @@ Constraint checking is always performed in place for pessimistic transactions (d - This variable is used to determine whether to include the `AUTO_INCREMENT` columns when creating a generated column or an expression index. ### tidb_enable_batch_dml + +> **Warning:** +> +> This variable is associated with the deprecated batch-dml feature, which might cause data corruption. Therefore, it is not recommended to enable this variable for batch-dml. Instead, use [non-transactional DML](/non-transactional-dml.md). + - Scope: GLOBAL - Persists to cluster: Yes - Type: Boolean - Default values: `OFF` -- This variable contronls whether to enable the deprecated batch-dml feature. When it's enabled, certain statements may be split into multiple transactions, which means it's non-atomic and should be used with care. When using the feature you must ensure there is no concurrent opertaions on the data you are operating on. You must also specify a positive `tidb_batch_dml_size` and turn on at least one of `tidb_batch_insert` and `tidb_batch_delete` to make it work. - -> **Warning:** -> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. +- This variable controls whether to enable the deprecated batch-dml feature. When it is enabled, certain statements might be split into multiple transactions, which is non-atomic and should be used with care. When using batch-dml, you must ensure that there are no concurrent operations on the data you are operating on. To make it work, you must also specify a positive value for `tidb_batch_dml_size` and enable at least one of `tidb_batch_insert` and `tidb_batch_delete`. ### tidb_enable_cascades_planner From 26cc19deba7a378c1d036f526f4ce83413772fc4 Mon Sep 17 00:00:00 2001 From: ekexium Date: Wed, 19 Oct 2022 11:23:33 +0800 Subject: [PATCH 4/4] change the deprecation description in nt-dml doc Signed-off-by: ekexium --- non-transactional-dml.md | 2 +- system-variables.md | 9 +++++---- 2 files changed, 6 insertions(+), 5 deletions(-) diff --git a/non-transactional-dml.md b/non-transactional-dml.md index fd3f48645885e..9a322213c13f9 100644 --- a/non-transactional-dml.md +++ b/non-transactional-dml.md @@ -251,7 +251,7 @@ batch-dml is a mechanism for splitting a transaction into multiple transaction c > **Note:** > -> It is not recommended to use batch-dml. When the batch-dml feature is not properly used, there is a risk of data index inconsistency. batch-dml will be deprecated in a later release of TiDB. +> It is not recommended to use batch-dml which has been deprecated. When the batch-dml feature is not properly used, there is a risk of data index inconsistency. Non-transactional DML statements are not yet a replacement for all batch-dml usage scenarios. Their main differences are as follows: diff --git a/system-variables.md b/system-variables.md index 2ec7deb9edf72..8ae7f1abcf694 100644 --- a/system-variables.md +++ b/system-variables.md @@ -766,7 +766,7 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Persists to cluster: No - Type: Boolean - Default value: `OFF` -- This variable is used to control whether to enable the batch-insert feature, which is a part of the deprecated batch-dml feature. When this variable is enabled, `INSERT` statements might be split into multiple transactions and committed non-atomically. To make it work, you also need to enable `tidb_enable_batch_dml` and set a positive for `tidb_dml_batch_size`, which is not recommended. +- This variable is used to control whether to enable the batch-insert feature, which is a part of the deprecated batch-dml feature. When this variable is enabled, `INSERT` statements might be split into multiple transactions and committed non-atomically. To make it work, you also need to enable `tidb_enable_batch_dml` and set a positive value for `tidb_dml_batch_size`, which is not recommended. ### tidb_batch_pending_tiflash_count New in v6.0 @@ -1007,6 +1007,10 @@ Constraint checking is always performed in place for pessimistic transactions (d ### tidb_dml_batch_size +> **Warning:** +> +> This variable is associated with the deprecated batch-dml feature, which might cause data corruption. Therefore, it is not recommended to enable this variable for batch-dml. Instead, use [non-transactional DML](/non-transactional-dml.md). + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Type: Integer @@ -1017,9 +1021,6 @@ Constraint checking is always performed in place for pessimistic transactions (d - Only the value `0` provides ACID compliance. Setting this to any other value will break the atomicity and isolation guarantees of TiDB. - To make this variable work, you also need to enable `tidb_enable_batch_dml` and at least one of `tidb_batch_insert` and `tidb_batch_delete`. -> **Warning:** -> batch-dml is deprecated because it's easy to misuse it and cause data corruption. Consider using [non-transactional DML](/non-transactional-dml.md) instead. - ### tidb_enable_1pc New in v5.0 - Scope: SESSION | GLOBAL