From 1314f95b13d4b6fcaa87d7f0d687e6c4f5184ecb Mon Sep 17 00:00:00 2001 From: Ran Date: Fri, 2 Dec 2022 14:10:26 +0800 Subject: [PATCH 01/15] sql: time to live Signed-off-by: Ran --- experimental-features.md | 1 + sql-statements/sql-statement-alter-table.md | 5 + sql-statements/sql-statement-create-table.md | 1 + system-variables.md | 108 ++++++++++++++++++ time-to-live-table.md | 109 +++++++++++++++++++ 5 files changed, 224 insertions(+) create mode 100644 time-to-live-table.md diff --git a/experimental-features.md b/experimental-features.md index eec6b510cade9..12c67eca5cb4a 100644 --- a/experimental-features.md +++ b/experimental-features.md @@ -38,6 +38,7 @@ Elastic scheduling feature. It enables the TiDB cluster to dynamically scale out + [Add index acceleration](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) (Introduced in v6.3.0) + [Restore a cluster to a specific point in time using the `FLASHBACK CLUSTER TO TIMESTAMP` syntax](/sql-statements/sql-statement-flashback-to-timestamp.md) (Introduced in v6.4.0) + [`AUTO_INCREMENT` MySQL compatibility mode](/auto-increment.md#mysql-compatibility-mode) (Introduced in v6.4.0) ++ [Time-to-live (TTL) support](/ttl-table.md) (Introduced in v6.5.0) ## Storage diff --git a/sql-statements/sql-statement-alter-table.md b/sql-statements/sql-statement-alter-table.md index fa3778bea46c4..7600d082914c2 100644 --- a/sql-statements/sql-statement-alter-table.md +++ b/sql-statements/sql-statement-alter-table.md @@ -49,6 +49,11 @@ AlterTableSpec ::= | 'SECONDARY_UNLOAD' | ( 'AUTO_INCREMENT' | 'AUTO_ID_CACHE' | 'AUTO_RANDOM_BASE' | 'SHARD_ROW_ID_BITS' ) EqOpt LengthNum | ( 'CACHE' | 'NOCACHE' ) +| ( + 'TTL' EqOpt TimeColumnName '+' 'INTERVAL' Expression TimeUnit (TTLEnable EqOpt ( 'ON' | 'OFF' ))? + | 'REMOVE' 'TTL' + | TTLEnable EqOpt ( 'ON' | 'OFF' ) + ) | PlacementPolicyOption PlacementPolicyOption ::= diff --git a/sql-statements/sql-statement-create-table.md b/sql-statements/sql-statement-create-table.md index cc0ac84d447d2..47e422dea05f8 100644 --- a/sql-statements/sql-statement-create-table.md +++ b/sql-statements/sql-statement-create-table.md @@ -83,6 +83,7 @@ TableOption ::= | 'SECONDARY_ENGINE' EqOpt ( 'NULL' | StringName ) | 'UNION' EqOpt '(' TableNameListOpt ')' | 'ENCRYPTION' EqOpt EncryptionOpt +| 'TTL' EqOpt TimeColumnName '+' 'INTERVAL' Expression TimeUnit (TTLEnable EqOpt ( 'ON' | 'OFF' ))? | PlacementPolicyOption OnCommitOpt ::= diff --git a/system-variables.md b/system-variables.md index eb3f29e203321..8eb930d2e4a00 100644 --- a/system-variables.md +++ b/system-variables.md @@ -3879,6 +3879,114 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > > Suppose that the TSO RPC latency increases for reasons other than a CPU usage bottleneck of the PD leader (such as network issues). In this case, increasing the value of `tidb_tso_client_batch_max_wait_time` might increase the execution latency in TiDB and affect the QPS performance of the cluster. +### tidb_ttl_delete_rate_limit New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `0` +- Range: `[0, 9223372036854775807]` +- This variable is used to limit the rate of `DELETE` statements in TTL jobs on each node. The value represents the maximum number of `DELETE` statements allowed per second in a single node in a TTL job. When this variable is set to `0`, no limit is applied. + +### tidb_ttl_delete_batch_size New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `100` +- Range: `[1, 10240]` +- This variable is used to set the maximum number of rows in a single `DELETE` transaction in a TTL job. + +### tidb_ttl_delete_worker_count New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `4` +- Range: `[1, 256]` +- This variable is used to set the maximum concurrency of TTL jobs on each TiDB node. + +### tidb_ttl_job_enable New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `ON` +- Type: Boolean +- This variable is used to control whether the TTL job is enabled. If it is set to `OFF`, all tables with TTL attributes automatically stops cleaning up expired data. + +### tidb_ttl_scan_batch_size New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `500` +- Range: `[1, 10240]` +- This variable is used to set the `LIMIT` value of each `SELECT` statement used to scan expired data in a TTL job. + +### tidb_ttl_scan_worker_count New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `4` +- Range: `[1, 256]` +- This variable is used to set the maximum concurrency of TTL scan jobs on each TiDB node. + +### tidb_ttl_job_run_interval New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Default value: `1h0m0s` +- Range: `[10m0s, 8760h0m0s]` +- This variable is used to control the scheduling interval of the TTL job in the background. For example, if the current value is set to `1h0m0s`, each table or partition with TTL attributes will clean up expired data once every hour. + +### tidb_ttl_job_schedule_window_start_time New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Type: Time +- Persists to cluster: Yes +- Default value: `00:00 +0000` +- This variable is used to control the start time of the scheduling window of the TTL job in the background. + +### tidb_ttl_job_schedule_window_end_time New in v6.5.0 + +> **Warning:** +> +> TTL is an experimental feature. This system variable might be changed or removed in future releases. + +- Scope: GLOBAL +- Type: Time +- Persists to cluster: Yes +- Default value: `23:59 +0000` +- This variable is used to control the end time of the scheduling window of the TTL job in the background. + ### tidb_txn_assertion_level New in v6.0.0 - Scope: SESSION | GLOBAL diff --git a/time-to-live-table.md b/time-to-live-table.md new file mode 100644 index 0000000000000..a77135d81ec90 --- /dev/null +++ b/time-to-live-table.md @@ -0,0 +1,109 @@ +--- +title: Use Time to Live +summary: Use Time to Live to automatically expire and delete old data. +--- + +# Use Time to Live + +Time to live (TTL) is a feature that provides row-level data lifetime management in TiDB. The TTL attribute of a table allows TiDB to automatically delete expired data to prevent the storage space from growing indefinitely. This feature can effectively save storage space in some scenarios, such as regularly deleting verification code records. + +> **Warning:** +> +> This is an experimental feature. It is not recommended that you use it in a production environment. + +## Syntax + +You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-statements/sql-statement-create-table.md) or [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md) statement. + +### Create a table with a TTL attribute + +- Create a table with a TTL attribute: + + ```sql + CREATE TABLE t1 ( + id int PRIMARY KEY, + created_at TIMESTAMP + ) TTL = `created_at` + INTERVAL 3 MONTH; + ``` + + The preceding example creates a table `t1` and specifies `created_at` as the TTL timestamp column, which indicates the creation time of the data. The example also sets the longest time a row is allowed to live in the table to 3 months through `INTERVAL 3 MONTH`. Data that lives longer than this value will be deleted later. + +- Set the `TTL_ENABLE` attribute to enable or disable the feature of cleaning up expired data: + + ```sql + CREATE TABLE t1 ( + id int PRIMARY KEY, + created_at TIMESTAMP + ) TTL = `created_at` + INTERVAL 3 MONTH TTL_ENABLE = 'OFF'; + ``` + + By default, `TTL_ENABLE` is set to `ON`. If `TTL_ENABLE` is set to `OFF`, even if other TTL options are set, TiDB does not automatically clean up expired data in this table. + +- To be compatible with MySQL, you can set a TTL attribute using a comment: + + ```sql + CREATE TABLE t1 ( + id int PRIMARY KEY, + created_at TIMESTAMP + ) /*T![ttl] TTL = `created_at` + INTERVAL 3 MONTH TTL_ENABLE = 'OFF'*/; + ``` + + In TiDB, using table options and comments to configure TTL is equivalent. In MySQL, the comment is ignored and an ordinary table is created. + +### Modify the TTL attribute of a table + +- Modify the TTL attribute of a table: + + ```sql + ALTER TABLE t1 TTL = `created_at` + INTERVAL 1 MONTH; + ``` + + You can use the preceding statement to modify a table with an existing TTL attribute or to add a TTL attribute to a table. + +- Modify the value of `TTL_ENABLE` for a table with the TTL attribute: + + ```sql + ALTER TABLE t1 TTL_ENABLE = 'OFF'; + ``` + +- To remove all TTL attributes of a table: + + ```sql + ALTER TABLE t1 REMOVE TTL; + ``` + +## TTL job + +For each table with a TTL attribute, TiDB internally schedules a background job to clean up expired data. You can customize the execution period of the job by setting the [`tidb_ttl_job_run_interval`](/system-variables.md#tidb_ttl_job_run_interval-new-in-v650) global variable. The following example sets the background cleanup job to run once every 24 hours: + +```sql +SET @@global.tidb_ttl_job_run_interval = '24h'; +``` + +To disable the execution of the TTL job, in addition to setting the `TTL_ENABLE='OFF'` table option, you can also disable the execution of the TTL job in the entire cluster by setting the [`tidb_ttl_job_enable`](/system-variables.md#tidb_ttl_job_enable-new-in-v650) global variable: + +```sql +SET @@global.tidb_ttl_job_enable = OFF; +``` + +In some scenarios, you might want to allow the TTL job to run only in a certain time window. In this case, you can set the [`tidb_ttl_job_schedule_window_start_time`](/system-variables.md#tidb_ttl_job_schedule_window_start_time-new-in-v650) and [`tidb_ttl_job_schedule_window_end_time`](/system-variables.md#tidb_ttl_job_schedule_window_end_time-new-in-v650) global variables to specify the time window. For example: + +```sql +SET @@global.tidb_ttl_job_schedule_window_start_time = '01:00 +0000'; +SET @@global.tidb_ttl_job_schedule_window_end_time = '05:00 +0000'; +``` + +The preceding statement allows the TTL job to be scheduled only between 1:00 and 5:00 UTC. By default, the time window is set to `00:00 +0000` to `23:59 +0000`, which allows the job to be scheduled at any time. + +## Compatibility with tools + +As an experimental feature, the TTL feature is not compatible with data import and export tools, including BR, TiDB Lightning, and TiCDC. + +## Limitations + +Currently, the TTL feature has the following limitations: + +* The TTL attribute cannot be set on temporary tables, including local temporary tables and global temporary tables. +* Tables with the TTL attribute do not support being referenced by other tables as the primary table in a foreign key constraint. +* It is not guaranteed that all expired data is deleted immediately. The time when expired data is deleted depends on the scheduling interval and scheduling window of the background cleanup job. +* Currently, a single table can only run a cleanup job on a single TiDB node at a given time. This might cause performance bottlenecks in some scenarios (for example, when the table is extremely large). This issue will be optimized in future releases. From 3dd79dd92ebf24200735a7031d173aa8bfd86db8 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 6 Dec 2022 21:39:43 +0800 Subject: [PATCH 02/15] sync changes from zh Signed-off-by: Ran --- TOC.md | 1 + experimental-features.md | 2 +- system-variables.md | 28 ++++++++++++------------ time-to-live-table.md => time-to-live.md | 20 ++++++++++++----- 4 files changed, 30 insertions(+), 21 deletions(-) rename time-to-live-table.md => time-to-live.md (77%) diff --git a/TOC.md b/TOC.md index acbd41779ae10..3495dac99e292 100644 --- a/TOC.md +++ b/TOC.md @@ -39,6 +39,7 @@ - [Update Data](/develop/dev-guide-update-data.md) - [Delete Data](/develop/dev-guide-delete-data.md) - [Prepared Statements](/develop/dev-guide-prepared-statement.md) + - [Time to Live](/time-to-live.md) - Read Data - [Query Data from a Single Table](/develop/dev-guide-get-data-from-single-table.md) - [Multi-table Join Queries](/develop/dev-guide-join-tables.md) diff --git a/experimental-features.md b/experimental-features.md index 7e09cd61e8337..aaf8716408e92 100644 --- a/experimental-features.md +++ b/experimental-features.md @@ -37,7 +37,7 @@ Elastic scheduling feature. It enables the TiDB cluster to dynamically scale out + [Range INTERVAL partitioning](/partitioned-table.md#range-interval-partitioning) (Introduced in v6.3.0) + [Add index acceleration](/system-variables.md#tidb_ddl_enable_fast_reorg-new-in-v630) (Introduced in v6.3.0) + [Restore a cluster to a specific point in time using the `FLASHBACK CLUSTER TO TIMESTAMP` syntax](/sql-statements/sql-statement-flashback-to-timestamp.md) (Introduced in v6.4.0) -+ [Time-to-live (TTL) support](/ttl-table.md) (Introduced in v6.5.0) ++ [Time to live](/time-to-live.md) (Introduced in v6.5.0) ## Storage diff --git a/system-variables.md b/system-variables.md index 8eb930d2e4a00..53f3a904b011b 100644 --- a/system-variables.md +++ b/system-variables.md @@ -3883,31 +3883,31 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes - Default value: `0` - Range: `[0, 9223372036854775807]` -- This variable is used to limit the rate of `DELETE` statements in TTL jobs on each node. The value represents the maximum number of `DELETE` statements allowed per second in a single node in a TTL job. When this variable is set to `0`, no limit is applied. +- This variable is used to limit the rate of `DELETE` statements in TTL jobs on each TiDB node. The value represents the maximum number of `DELETE` statements allowed per second in a single node in a TTL job. When this variable is set to `0`, no limit is applied. ### tidb_ttl_delete_batch_size New in v6.5.0 > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes - Default value: `100` - Range: `[1, 10240]` -- This variable is used to set the maximum number of rows in a single `DELETE` transaction in a TTL job. +- This variable is used to set the maximum number of rows that can be deleted in a single `DELETE` transaction in a TTL job. ### tidb_ttl_delete_worker_count New in v6.5.0 > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes @@ -3919,7 +3919,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes @@ -3931,7 +3931,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes @@ -3943,7 +3943,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes @@ -3955,37 +3955,37 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Persists to cluster: Yes - Default value: `1h0m0s` - Range: `[10m0s, 8760h0m0s]` -- This variable is used to control the scheduling interval of the TTL job in the background. For example, if the current value is set to `1h0m0s`, each table or partition with TTL attributes will clean up expired data once every hour. +- This variable is used to control the scheduling interval of the TTL job in the background. For example, if the current value is set to `1h0m0s`, each table with TTL attributes will clean up expired data once every hour. ### tidb_ttl_job_schedule_window_start_time New in v6.5.0 > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Type: Time - Persists to cluster: Yes - Default value: `00:00 +0000` -- This variable is used to control the start time of the scheduling window of the TTL job in the background. +- This variable is used to control the start time of the scheduling window of the TTL job in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. ### tidb_ttl_job_schedule_window_end_time New in v6.5.0 > **Warning:** > -> TTL is an experimental feature. This system variable might be changed or removed in future releases. +> [TTL](/time-to-live.md) is an experimental feature. This system variable might be changed or removed in future releases. - Scope: GLOBAL - Type: Time - Persists to cluster: Yes - Default value: `23:59 +0000` -- This variable is used to control the end time of the scheduling window of the TTL job in the background. +- This variable is used to control the end time of the scheduling window of the TTL job in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. ### tidb_txn_assertion_level New in v6.0.0 diff --git a/time-to-live-table.md b/time-to-live.md similarity index 77% rename from time-to-live-table.md rename to time-to-live.md index a77135d81ec90..453c785d359d4 100644 --- a/time-to-live-table.md +++ b/time-to-live.md @@ -1,11 +1,19 @@ --- -title: Use Time to Live +title: Time to Live (TTL) summary: Use Time to Live to automatically expire and delete old data. --- -# Use Time to Live +# Time to Live (TTL) -Time to live (TTL) is a feature that provides row-level data lifetime management in TiDB. The TTL attribute of a table allows TiDB to automatically delete expired data to prevent the storage space from growing indefinitely. This feature can effectively save storage space in some scenarios, such as regularly deleting verification code records. +Time to live (TTL) is a feature that provides row-level data lifetime management in TiDB. In TiDB, a table with the TTL attribute automatically checks data lifetime and deletes expired data at the row level. This feature can effectively save storage space and enhance performance in some scenarios. + +The following are some common scenarios for TTL: + +* Regularly delete verification codes and short URLs. +* Regularly delete unnecessary historical orders. +* Automatically delete intermediate results of calculations. + +TTL is designed to help users clean up unnecessary data periodically and in a timely manner without affecting the online read and write workloads. TTL does not guarantee that all expired data is deleted immediately. The time when expired data is deleted depends on the scheduling interval and scheduling window of the background cleanup task. > **Warning:** > @@ -37,7 +45,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ) TTL = `created_at` + INTERVAL 3 MONTH TTL_ENABLE = 'OFF'; ``` - By default, `TTL_ENABLE` is set to `ON`. If `TTL_ENABLE` is set to `OFF`, even if other TTL options are set, TiDB does not automatically clean up expired data in this table. + If `TTL_ENABLE` is set to `OFF`, even if other TTL options are set, TiDB does not automatically clean up expired data in this table. For a table with the TTL attribute, `TTL_ENABLE` is set to `ON` by default. - To be compatible with MySQL, you can set a TTL attribute using a comment: @@ -48,7 +56,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ) /*T![ttl] TTL = `created_at` + INTERVAL 3 MONTH TTL_ENABLE = 'OFF'*/; ``` - In TiDB, using table options and comments to configure TTL is equivalent. In MySQL, the comment is ignored and an ordinary table is created. + In TiDB, using the table TTL attribute or using comments to configure TTL is equivalent. In MySQL, the comment is ignored and an ordinary table is created. ### Modify the TTL attribute of a table @@ -58,7 +66,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ALTER TABLE t1 TTL = `created_at` + INTERVAL 1 MONTH; ``` - You can use the preceding statement to modify a table with an existing TTL attribute or to add a TTL attribute to a table. + You can use the preceding statement to modify a table with an existing TTL attribute or to add a TTL attribute to a table without a TTL attribute. - Modify the value of `TTL_ENABLE` for a table with the TTL attribute: From e1e33206f926adb9f948fba7839128c98dfac411 Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 12 Dec 2022 10:06:09 +0800 Subject: [PATCH 03/15] Update experimental-features.md --- experimental-features.md | 1 - 1 file changed, 1 deletion(-) diff --git a/experimental-features.md b/experimental-features.md index 31baf000a4c81..ba21a873a473a 100644 --- a/experimental-features.md +++ b/experimental-features.md @@ -33,7 +33,6 @@ Elastic scheduling feature. It enables the TiDB cluster to dynamically scale out + [Cascades Planner](/system-variables.md#tidb_enable_cascades_planner): a cascades framework-based top-down query optimizer (Introduced in v3.0) + [Table Lock](/tidb-configuration-file.md#enable-table-lock-new-in-v400) (Introduced in v4.0.0) + [Range INTERVAL partitioning](/partitioned-table.md#range-interval-partitioning) (Introduced in v6.3.0) -+ [Restore a cluster to a specific point in time using the `FLASHBACK CLUSTER TO TIMESTAMP` syntax](/sql-statements/sql-statement-flashback-to-timestamp.md) (Introduced in v6.4.0) + [Time to live](/time-to-live.md) (Introduced in v6.5.0) + [TiFlash Query Result Materialization](/tiflash/tiflash-results-materialization.md) (Introduced in v6.5.0) From 995c832b44d400ee2d107b8649421cc7ab6522bd Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 20 Dec 2022 12:22:59 +0800 Subject: [PATCH 04/15] sync changes --- TOC.md | 2 +- time-to-live.md | 58 +++++++++++++++++++++++++++++++++++++++++++++---- 2 files changed, 55 insertions(+), 5 deletions(-) diff --git a/TOC.md b/TOC.md index 5119cf36811f7..813278adf1034 100644 --- a/TOC.md +++ b/TOC.md @@ -38,8 +38,8 @@ - [Insert Data](/develop/dev-guide-insert-data.md) - [Update Data](/develop/dev-guide-update-data.md) - [Delete Data](/develop/dev-guide-delete-data.md) + - [Periodically Delete Data Using Time to Live](/time-to-live.md) - [Prepared Statements](/develop/dev-guide-prepared-statement.md) - - [Time to Live](/time-to-live.md) - Read Data - [Query Data from a Single Table](/develop/dev-guide-get-data-from-single-table.md) - [Multi-table Join Queries](/develop/dev-guide-join-tables.md) diff --git a/time-to-live.md b/time-to-live.md index 453c785d359d4..d7e15cf16b2b1 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -1,9 +1,9 @@ --- -title: Time to Live (TTL) +title: Periodically Delete Data Using Time to Live summary: Use Time to Live to automatically expire and delete old data. --- -# Time to Live (TTL) +# Periodically Delete Data Using Time to Live Time to live (TTL) is a feature that provides row-level data lifetime management in TiDB. In TiDB, a table with the TTL attribute automatically checks data lifetime and deletes expired data at the row level. This feature can effectively save storage space and enhance performance in some scenarios. @@ -13,7 +13,7 @@ The following are some common scenarios for TTL: * Regularly delete unnecessary historical orders. * Automatically delete intermediate results of calculations. -TTL is designed to help users clean up unnecessary data periodically and in a timely manner without affecting the online read and write workloads. TTL does not guarantee that all expired data is deleted immediately. The time when expired data is deleted depends on the scheduling interval and scheduling window of the background cleanup task. +TTL is designed to help users clean up unnecessary data periodically and in a timely manner without affecting the online read and write workloads. TTL concurrently dispatch different tasks to different TiDB server nodes to delete data in parallel in the unit of table. TTL does not guarantee that all expired data is deleted immediately, which means that even if the data is expired, the client might still read the expired data some time after the expiration time until the data is deleted by the background processing task. > **Warning:** > @@ -80,6 +80,56 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ALTER TABLE t1 REMOVE TTL; ``` +### TTL and the default values of data types + +You can use TTL together with [default values of data types](/data-type-default-values.md). The following are two common usage examples: + +* Use `DEFAULT CURRENT_TIMESTAMP` to specify the default value of a column as the current creation time and use this column as the TTL timestamp column: + + ```sql + CREATE TABLE t1 ( + id int PRIMARY KEY, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP + ) TTL = `created_at` + INTERVAL 3 MONTH; + ``` + +* Specify the default value of a column as the current creation time and update time: + + ```sql + CREATE TABLE t1 ( + id int PRIMARY KEY, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, + ) TTL = `created_at` + INTERVAL 3 MONTH; + ``` + +### TTL and generated columns + +You can use TTL together with [generated columns](/generated-columns.md) (experimental feature) to configure complex expiration rules. For example: + +```sql +CREATE TABLE message ( + id int PRIMARY KEY, + created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP, + image bool, + expire_at TIMESTAMP AS (IF(image, + created_at + INTERVAL 5 DAY, + created_at + INTERVAL 30 DAY + )) +) TTL = `expire_at` + INTERVAL 0 DAY; +``` + +The preceding statement uses the `expire_at` column as the TTL timestamp column and sets the expiration time according to the message type. If the message is an image, it expires in 5 days. Otherwise, it expires in 30 days. + +You can use TTL together with the [JSON type](/data-type-json.md). For example: + +```sql +CREATE TABLE orders ( + id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, + order_info JSON, + created_at DATE AS (JSON_EXTRACT(order_info, '$.created_at')) VIRTUAL +) TTL = `created_at` + INTERVAL 3 month; +``` + ## TTL job For each table with a TTL attribute, TiDB internally schedules a background job to clean up expired data. You can customize the execution period of the job by setting the [`tidb_ttl_job_run_interval`](/system-variables.md#tidb_ttl_job_run_interval-new-in-v650) global variable. The following example sets the background cleanup job to run once every 24 hours: @@ -114,4 +164,4 @@ Currently, the TTL feature has the following limitations: * The TTL attribute cannot be set on temporary tables, including local temporary tables and global temporary tables. * Tables with the TTL attribute do not support being referenced by other tables as the primary table in a foreign key constraint. * It is not guaranteed that all expired data is deleted immediately. The time when expired data is deleted depends on the scheduling interval and scheduling window of the background cleanup job. -* Currently, a single table can only run a cleanup job on a single TiDB node at a given time. This might cause performance bottlenecks in some scenarios (for example, when the table is extremely large). This issue will be optimized in future releases. +* Currently, a single table can only run a cleanup job on a single TiDB server node at a given time. This might cause performance bottlenecks in some scenarios (for example, when the table is extremely large). This issue will be optimized in future releases. From 262583d9d876243b46dda4317a73bc90c86e16a6 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 20 Dec 2022 14:51:45 +0800 Subject: [PATCH 05/15] update dev guide --- develop/dev-guide-delete-data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/develop/dev-guide-delete-data.md b/develop/dev-guide-delete-data.md index de61788c49341..f75817b9af95a 100644 --- a/develop/dev-guide-delete-data.md +++ b/develop/dev-guide-delete-data.md @@ -5,7 +5,7 @@ summary: Learn about the SQL syntax, best practices, and examples for deleting d # Delete Data -This document describes how to use the [DELETE](/sql-statements/sql-statement-delete.md) SQL statement to delete the data in TiDB. +This document describes how to use the [DELETE](/sql-statements/sql-statement-delete.md) SQL statement to delete the data in TiDB. If you need to periodically delete expired data, use the [time to live](/time-to-live.md) feature. ## Before you start From 3a012be7842064ca34c06ad0b87478673b43a3e2 Mon Sep 17 00:00:00 2001 From: Ran Date: Wed, 21 Dec 2022 21:05:34 +0800 Subject: [PATCH 06/15] Apply suggestions from code review Co-authored-by: Grace Cai --- system-variables.md | 8 ++++---- time-to-live.md | 24 ++++++++++++------------ 2 files changed, 16 insertions(+), 16 deletions(-) diff --git a/system-variables.md b/system-variables.md index 9dd828f6e7134..f92492c98f5ed 100644 --- a/system-variables.md +++ b/system-variables.md @@ -4009,7 +4009,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). - Persists to cluster: Yes - Default value: `ON` - Type: Boolean -- This variable is used to control whether the TTL job is enabled. If it is set to `OFF`, all tables with TTL attributes automatically stops cleaning up expired data. +- This variable is used to control whether TTL jobs are enabled. If it is set to `OFF`, all tables with TTL attributes automatically stop cleaning up expired data. ### tidb_ttl_scan_batch_size New in v6.5.0 @@ -4045,7 +4045,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). - Persists to cluster: Yes - Default value: `1h0m0s` - Range: `[10m0s, 8760h0m0s]` -- This variable is used to control the scheduling interval of the TTL job in the background. For example, if the current value is set to `1h0m0s`, each table with TTL attributes will clean up expired data once every hour. +- This variable is used to control the scheduling interval of TTL jobs in the background. For example, if the current value is set to `1h0m0s`, each table with TTL attributes cleans up expired data once every hour. ### tidb_ttl_job_schedule_window_start_time New in v6.5.0 @@ -4057,7 +4057,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). - Type: Time - Persists to cluster: Yes - Default value: `00:00 +0000` -- This variable is used to control the start time of the scheduling window of the TTL job in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. +- This variable is used to control the start time of the scheduling window of TTL jobs in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. ### tidb_ttl_job_schedule_window_end_time New in v6.5.0 @@ -4069,7 +4069,7 @@ For details, see [Identify Slow Queries](/identify-slow-queries.md). - Type: Time - Persists to cluster: Yes - Default value: `23:59 +0000` -- This variable is used to control the end time of the scheduling window of the TTL job in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. +- This variable is used to control the end time of the scheduling window of TTL jobs in the background. When you modify the value of this variable, be cautious that a small window might cause the cleanup of expired data to fail. ### tidb_txn_assertion_level New in v6.0.0 diff --git a/time-to-live.md b/time-to-live.md index d7e15cf16b2b1..32355a568735f 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -5,7 +5,7 @@ summary: Use Time to Live to automatically expire and delete old data. # Periodically Delete Data Using Time to Live -Time to live (TTL) is a feature that provides row-level data lifetime management in TiDB. In TiDB, a table with the TTL attribute automatically checks data lifetime and deletes expired data at the row level. This feature can effectively save storage space and enhance performance in some scenarios. +Time to live (TTL) is a feature that allows you to manage TiDB data lifetime at the row level. For a table with the TTL attribute, TiDB automatically checks data lifetime and deletes expired data at the row level. This feature can effectively save storage space and enhance performance in some scenarios. The following are some common scenarios for TTL: @@ -13,7 +13,7 @@ The following are some common scenarios for TTL: * Regularly delete unnecessary historical orders. * Automatically delete intermediate results of calculations. -TTL is designed to help users clean up unnecessary data periodically and in a timely manner without affecting the online read and write workloads. TTL concurrently dispatch different tasks to different TiDB server nodes to delete data in parallel in the unit of table. TTL does not guarantee that all expired data is deleted immediately, which means that even if the data is expired, the client might still read the expired data some time after the expiration time until the data is deleted by the background processing task. +TTL is designed to help users clean up unnecessary data periodically and in a timely manner without affecting the online read and write workloads. TTL concurrently dispatches different jobs to different TiDB nodes to delete data in parallel in the unit of table. TTL does not guarantee that all expired data is deleted immediately, which means that even if some data is expired, the client might still read that data some time after the expiration time until that data is deleted by the background TTL job. > **Warning:** > @@ -34,7 +34,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ) TTL = `created_at` + INTERVAL 3 MONTH; ``` - The preceding example creates a table `t1` and specifies `created_at` as the TTL timestamp column, which indicates the creation time of the data. The example also sets the longest time a row is allowed to live in the table to 3 months through `INTERVAL 3 MONTH`. Data that lives longer than this value will be deleted later. + The preceding example creates a table `t1` and specifies `created_at` as the TTL timestamp column, which indicates the creation time of the data. The example also sets the longest time that a row is allowed to live in the table to 3 months through `INTERVAL 3 MONTH`. Data that lives longer than this value will be deleted later. - Set the `TTL_ENABLE` attribute to enable or disable the feature of cleaning up expired data: @@ -45,7 +45,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ) TTL = `created_at` + INTERVAL 3 MONTH TTL_ENABLE = 'OFF'; ``` - If `TTL_ENABLE` is set to `OFF`, even if other TTL options are set, TiDB does not automatically clean up expired data in this table. For a table with the TTL attribute, `TTL_ENABLE` is set to `ON` by default. + If `TTL_ENABLE` is set to `OFF`, even if other TTL options are set, TiDB does not automatically clean up expired data in this table. For a table with the TTL attribute, `TTL_ENABLE` is `ON` by default. - To be compatible with MySQL, you can set a TTL attribute using a comment: @@ -82,7 +82,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s ### TTL and the default values of data types -You can use TTL together with [default values of data types](/data-type-default-values.md). The following are two common usage examples: +You can use TTL together with [default values of the data types](/data-type-default-values.md). The following are two common usage examples: * Use `DEFAULT CURRENT_TIMESTAMP` to specify the default value of a column as the current creation time and use this column as the TTL timestamp column: @@ -132,28 +132,28 @@ CREATE TABLE orders ( ## TTL job -For each table with a TTL attribute, TiDB internally schedules a background job to clean up expired data. You can customize the execution period of the job by setting the [`tidb_ttl_job_run_interval`](/system-variables.md#tidb_ttl_job_run_interval-new-in-v650) global variable. The following example sets the background cleanup job to run once every 24 hours: +For each table with a TTL attribute, TiDB internally schedules a background job to clean up expired data. You can customize the execution period of these jobs by setting the [`tidb_ttl_job_run_interval`](/system-variables.md#tidb_ttl_job_run_interval-new-in-v650) global variable. The following example sets the background cleanup jobs to run once every 24 hours: ```sql SET @@global.tidb_ttl_job_run_interval = '24h'; ``` -To disable the execution of the TTL job, in addition to setting the `TTL_ENABLE='OFF'` table option, you can also disable the execution of the TTL job in the entire cluster by setting the [`tidb_ttl_job_enable`](/system-variables.md#tidb_ttl_job_enable-new-in-v650) global variable: +To disable the execution of TTL jobs, in addition to setting the `TTL_ENABLE='OFF'` table option, you can also disable the execution of TTL jobs in the entire cluster by setting the [`tidb_ttl_job_enable`](/system-variables.md#tidb_ttl_job_enable-new-in-v650) global variable: ```sql SET @@global.tidb_ttl_job_enable = OFF; ``` -In some scenarios, you might want to allow the TTL job to run only in a certain time window. In this case, you can set the [`tidb_ttl_job_schedule_window_start_time`](/system-variables.md#tidb_ttl_job_schedule_window_start_time-new-in-v650) and [`tidb_ttl_job_schedule_window_end_time`](/system-variables.md#tidb_ttl_job_schedule_window_end_time-new-in-v650) global variables to specify the time window. For example: +In some scenarios, you might want to allow TTL jobs to run only in a certain time window. In this case, you can set the [`tidb_ttl_job_schedule_window_start_time`](/system-variables.md#tidb_ttl_job_schedule_window_start_time-new-in-v650) and [`tidb_ttl_job_schedule_window_end_time`](/system-variables.md#tidb_ttl_job_schedule_window_end_time-new-in-v650) global variables to specify the time window. For example: ```sql SET @@global.tidb_ttl_job_schedule_window_start_time = '01:00 +0000'; SET @@global.tidb_ttl_job_schedule_window_end_time = '05:00 +0000'; ``` -The preceding statement allows the TTL job to be scheduled only between 1:00 and 5:00 UTC. By default, the time window is set to `00:00 +0000` to `23:59 +0000`, which allows the job to be scheduled at any time. +The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5:00 UTC. By default, the time window is set to `00:00 +0000` to `23:59 +0000`, which allows the jobs to be scheduled at any time. -## Compatibility with tools +## Compatibility with TiDB tools As an experimental feature, the TTL feature is not compatible with data import and export tools, including BR, TiDB Lightning, and TiCDC. @@ -162,6 +162,6 @@ As an experimental feature, the TTL feature is not compatible with data import a Currently, the TTL feature has the following limitations: * The TTL attribute cannot be set on temporary tables, including local temporary tables and global temporary tables. -* Tables with the TTL attribute do not support being referenced by other tables as the primary table in a foreign key constraint. +* A table with the TTL attribute does not support being referenced by other tables as the primary table in a foreign key constraint. * It is not guaranteed that all expired data is deleted immediately. The time when expired data is deleted depends on the scheduling interval and scheduling window of the background cleanup job. -* Currently, a single table can only run a cleanup job on a single TiDB server node at a given time. This might cause performance bottlenecks in some scenarios (for example, when the table is extremely large). This issue will be optimized in future releases. +* Currently, a single table can only run a cleanup job on a single TiDB node at a given time. This might cause performance bottlenecks in some scenarios (for example, when the table is extremely large). This issue will be optimized in future releases. From 5e7be9ff85f0fa09a215a9054f0e91240a4bc52d Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 11:33:15 +0800 Subject: [PATCH 07/15] sync changes Signed-off-by: Ran --- grafana-tidb-dashboard.md | 7 +++++++ time-to-live.md | 5 +++++ 2 files changed, 12 insertions(+) diff --git a/grafana-tidb-dashboard.md b/grafana-tidb-dashboard.md index b0e921c6cc62e..06f3b6abf8765 100644 --- a/grafana-tidb-dashboard.md +++ b/grafana-tidb-dashboard.md @@ -170,3 +170,10 @@ To understand the key metrics displayed on the TiDB dashboard, check the followi - Pending Request Count by TiKV: the number of Batch messages that are pending processing - Batch Client Unavailable Duration 95: the unavailable time of the Batch client - No Available Connection Counter: the number of times the Batch client cannot find an available link + +- TTL + - TTL QPS By Type: the QPS information of different types of statements generated by TTL jobs. + - TTL Processed Rows Per Second: the number of expired rows processed by TTL jobs per second. + - TTL Scan/Delete Query Duration: the execution time of TTL scan/delete statements. + - TTL Scan/Delete Worker Time By Phase: the time consumed by different phases of TTL internal worker threads. + - TTL Job Count By Status: the number of TTL jobs currently being executed. diff --git a/time-to-live.md b/time-to-live.md index 32355a568735f..4cd31cc21d5ee 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -18,6 +18,7 @@ TTL is designed to help users clean up unnecessary data periodically and in a ti > **Warning:** > > This is an experimental feature. It is not recommended that you use it in a production environment. +> TTL is not available for [TiDB Cloud Serverless Tier](https://docs.pingcap.com/tidbcloud/select-cluster-tier#serverless-tier-beta). ## Syntax @@ -153,6 +154,10 @@ SET @@global.tidb_ttl_job_schedule_window_end_time = '05:00 +0000'; The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5:00 UTC. By default, the time window is set to `00:00 +0000` to `23:59 +0000`, which allows the jobs to be scheduled at any time. +## Monitoring metrics and charts + +TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). + ## Compatibility with TiDB tools As an experimental feature, the TTL feature is not compatible with data import and export tools, including BR, TiDB Lightning, and TiCDC. From 34f6c28218bafec1513715908b6f564048b71f8d Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Mon, 26 Dec 2022 13:40:05 +0800 Subject: [PATCH 08/15] Apply suggestions from code review Co-authored-by: Ran --- time-to-live.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/time-to-live.md b/time-to-live.md index 4cd31cc21d5ee..cd501143cd9b2 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -85,7 +85,7 @@ You can configure the TTL attribute of a table using the [`CREATE TABLE`](/sql-s You can use TTL together with [default values of the data types](/data-type-default-values.md). The following are two common usage examples: -* Use `DEFAULT CURRENT_TIMESTAMP` to specify the default value of a column as the current creation time and use this column as the TTL timestamp column: +* Use `DEFAULT CURRENT_TIMESTAMP` to specify the default value of a column as the current creation time and use this column as the TTL timestamp column. Records that were created 3 months ago are expired: ```sql CREATE TABLE t1 ( @@ -94,7 +94,7 @@ You can use TTL together with [default values of the data types](/data-type-defa ) TTL = `created_at` + INTERVAL 3 MONTH; ``` -* Specify the default value of a column as the current creation time and update time: +* Specify the default value of a column as the creation time or the latest update time and use this column as the TTL timestamp column. Records that have not been updated for 3 months are expired: ```sql CREATE TABLE t1 ( From c1bcc78213adcad3e4c0bbf8997f0b3b0426212a Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 13:45:47 +0800 Subject: [PATCH 09/15] add ttl to toc-cloud Signed-off-by: Ran --- TOC-tidb-cloud.md | 1 + 1 file changed, 1 insertion(+) diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index 88761f9284293..add82573f5598 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -34,6 +34,7 @@ - [Insert Data](/develop/dev-guide-insert-data.md) - [Update Data](/develop/dev-guide-update-data.md) - [Delete Data](/develop/dev-guide-delete-data.md) + - [Periodically Delete Data Using Time to Live](/time-to-live.md) - [Prepared Statements](/develop/dev-guide-prepared-statement.md) - Read Data - [Query Data from a Single Table](/develop/dev-guide-get-data-from-single-table.md) From 1d848ddd74ccc57498c7ffc5b7890d29da473fdb Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 13:47:37 +0800 Subject: [PATCH 10/15] add glossary Signed-off-by: Ran --- glossary.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/glossary.md b/glossary.md index ce77e4a68b56f..7dc51be06ca62 100644 --- a/glossary.md +++ b/glossary.md @@ -155,3 +155,7 @@ Top SQL helps locate SQL queries that contribute to a high load of a TiDB or TiK ### TSO Because TiKV is a distributed storage system, it requires a global timing service, Timestamp Oracle (TSO), to assign a monotonically increasing timestamp. In TiKV, such a feature is provided by PD, and in Google [Spanner](http://static.googleusercontent.com/media/research.google.com/en//archive/spanner-osdi2012.pdf), this feature is provided by multiple atomic clocks and GPS. + +### TTL + +[Time to live (TTL)](/time-to-live.md) is a feature that allows you to manage TiDB data lifetime at the row level. For a table with the TTL attribute, TiDB automatically checks data lifetime and deletes expired data at the row level. From c0a96bd2f7020cdc8d7bf1b88e418481bc4736df Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 14:32:38 +0800 Subject: [PATCH 11/15] add custom content --- time-to-live.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/time-to-live.md b/time-to-live.md index cd501143cd9b2..64a339e36f008 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -156,8 +156,18 @@ The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5: ## Monitoring metrics and charts + + TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). + + + + +This section is not applicable to TiDB Cloud. + + + ## Compatibility with TiDB tools As an experimental feature, the TTL feature is not compatible with data import and export tools, including BR, TiDB Lightning, and TiCDC. From 7a03c8aa25a14eb16df9ed61feb99e5d164aa11b Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 18:11:43 +0800 Subject: [PATCH 12/15] update note Signed-off-by: Ran --- time-to-live.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/time-to-live.md b/time-to-live.md index 64a339e36f008..9b916888d3e36 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -156,18 +156,16 @@ The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5: ## Monitoring metrics and charts - - -TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). - - - -This section is not applicable to TiDB Cloud. +> **Note:** +> +> This section is only applicable to on-premises TiDB. Currently, TiDB Cloud does not support TTL metrics. +TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). + ## Compatibility with TiDB tools As an experimental feature, the TTL feature is not compatible with data import and export tools, including BR, TiDB Lightning, and TiCDC. From 04dd2c9d25a0c0126541201efbe0c67ef02e888b Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 18:25:36 +0800 Subject: [PATCH 13/15] use customcontent for tidb Signed-off-by: Ran --- time-to-live.md | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/time-to-live.md b/time-to-live.md index 9b916888d3e36..5c4b15ba9de23 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -164,7 +164,13 @@ The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5: -TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). +TiDB collects runtime information about TTL periodically and provides visualized charts of these metrics in Grafana. You can see these metrics in the TiDB -> TTL panel in Grafana. + + + +For details of the metrics, see the TTL section in [TiDB Monitoring Metrics](/grafana-tidb-dashboard.md). + + ## Compatibility with TiDB tools From 2fd260fed0287e1152413a0973a0c70a3d9c502c Mon Sep 17 00:00:00 2001 From: Ran Date: Mon, 26 Dec 2022 18:30:15 +0800 Subject: [PATCH 14/15] fix conflict Signed-off-by: Ran --- time-to-live.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/time-to-live.md b/time-to-live.md index 5c4b15ba9de23..d2460f3e01cad 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -160,7 +160,7 @@ The preceding statement allows TTL jobs to be scheduled only between 1:00 and 5: > **Note:** > -> This section is only applicable to on-premises TiDB. Currently, TiDB Cloud does not support TTL metrics. +> This section is only applicable to on-premises TiDB. Currently, TiDB Cloud does not provide TTL metrics. From 7352630b8321b308eb569a29c4c012e8dd78b7f5 Mon Sep 17 00:00:00 2001 From: Ran Date: Tue, 27 Dec 2022 21:49:57 +0800 Subject: [PATCH 15/15] Apply suggestions from code review Co-authored-by: Feng Liyuan --- TOC-tidb-cloud.md | 2 +- time-to-live.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/TOC-tidb-cloud.md b/TOC-tidb-cloud.md index add82573f5598..c05b11eabd475 100644 --- a/TOC-tidb-cloud.md +++ b/TOC-tidb-cloud.md @@ -34,7 +34,7 @@ - [Insert Data](/develop/dev-guide-insert-data.md) - [Update Data](/develop/dev-guide-update-data.md) - [Delete Data](/develop/dev-guide-delete-data.md) - - [Periodically Delete Data Using Time to Live](/time-to-live.md) + - [Periodically Delete Expired Data Using TTL (Time to Live)](/time-to-live.md) - [Prepared Statements](/develop/dev-guide-prepared-statement.md) - Read Data - [Query Data from a Single Table](/develop/dev-guide-get-data-from-single-table.md) diff --git a/time-to-live.md b/time-to-live.md index d2460f3e01cad..c58d941ffce60 100644 --- a/time-to-live.md +++ b/time-to-live.md @@ -3,7 +3,7 @@ title: Periodically Delete Data Using Time to Live summary: Use Time to Live to automatically expire and delete old data. --- -# Periodically Delete Data Using Time to Live +# Periodically Delete Expired Data Using TTL (Time to Live) Time to live (TTL) is a feature that allows you to manage TiDB data lifetime at the row level. For a table with the TTL attribute, TiDB automatically checks data lifetime and deletes expired data at the row level. This feature can effectively save storage space and enhance performance in some scenarios.