diff --git a/experimental-features.md b/experimental-features.md index 94594876e93be..9f77f5cf18f9f 100644 --- a/experimental-features.md +++ b/experimental-features.md @@ -24,6 +24,7 @@ This document introduces the experimental features of TiDB in different versions + List Partition (Introduced in v5.0) + List COLUMNS Partition (Introduced in v5.0) ++ [Dynamic Mode for Partitioned Tables](/partitioned-table.md#dynamic-mode). (Introduced in v5.1) + The expression index feature. The expression index is also called the function-based index. When you create an index, the index fields do not have to be a specific column but can be an expression calculated from one or more columns. This feature is useful for quickly accessing the calculation-based tables. See [Expression index](/sql-statements/sql-statement-create-index.md) for details. (Introduced in v4.0) + [Generated Columns](/generated-columns.md). + [User-Defined Variables](/user-defined-variables.md). diff --git a/partitioned-table.md b/partitioned-table.md index d06b0977a08a4..c6b8a198b675c 100644 --- a/partitioned-table.md +++ b/partitioned-table.md @@ -1240,3 +1240,195 @@ select * from t; The `tidb_enable_list_partition` environment variable controls whether to enable the partitioned table feature. If this variable is set to `OFF`, the partition information will be ignored when a table is created, and this table will be created as a normal table. This variable is only used in table creation. After the table is created, modify this variable value takes no effect. For details, see [system variables](/system-variables.md#tidb_enable_list_partition-new-in-v50). + +### Dynamic mode + +> **Warning:** +> +> This is still an experimental feature. It is **NOT** recommended that you use it in the production environment. + +TiDB accesses partitioned tables in one of the two modes: `dynamic` mode and `static` mode. Currently, `static` mode is used by default. If you want to enable `dynamic` mode, you need to manually set the `tidb_partition_prune_mode` variable to `dynamic`. + +{{< copyable "sql" >}} + +```sql +set @@session.tidb_partition_prune_mode = 'dynamic' +``` + +In `static` mode, TiDB accesses each partition separately using multiple operators, and then merges the results using `Union`. The following example is a simple read operation where TiDB merges the results of two corresponding partitions using `Union`: + +{{< copyable "sql" >}} + +```sql +mysql> create table t1(id int, age int, key(id)) partition by range(id) ( + -> partition p0 values less than (100), + -> partition p1 values less than (200), + -> partition p2 values less than (300), + -> partition p3 values less than (400)); +Query OK, 0 rows affected (0.01 sec) + +mysql> explain select * from t1 where id < 150; ++------------------------------+----------+-----------+------------------------+--------------------------------+ +| id | estRows | task | access object | operator info | ++------------------------------+----------+-----------+------------------------+--------------------------------+ +| PartitionUnion_9 | 6646.67 | root | | | +| ├─TableReader_12 | 3323.33 | root | | data:Selection_11 | +| │ └─Selection_11 | 3323.33 | cop[tikv] | | lt(test.t1.id, 150) | +| │ └─TableFullScan_10 | 10000.00 | cop[tikv] | table:t1, partition:p0 | keep order:false, stats:pseudo | +| └─TableReader_18 | 3323.33 | root | | data:Selection_17 | +| └─Selection_17 | 3323.33 | cop[tikv] | | lt(test.t1.id, 150) | +| └─TableFullScan_16 | 10000.00 | cop[tikv] | table:t1, partition:p1 | keep order:false, stats:pseudo | ++------------------------------+----------+-----------+------------------------+--------------------------------+ +7 rows in set (0.00 sec) +``` + +In `dynamic` mode, each operator supports direct access to multiple partitions, so TiDB no longer uses `Union`. + +{{< copyable "sql" >}} + +```sql +mysql> set @@session.tidb_partition_prune_mode = 'dynamic'; +Query OK, 0 rows affected (0.00 sec) + +mysql> explain select * from t1 where id < 150; ++-------------------------+----------+-----------+-----------------+--------------------------------+ +| id | estRows | task | access object | operator info | ++-------------------------+----------+-----------+-----------------+--------------------------------+ +| TableReader_7 | 3323.33 | root | partition:p0,p1 | data:Selection_6 | +| └─Selection_6 | 3323.33 | cop[tikv] | | lt(test.t1.id, 150) | +| └─TableFullScan_5 | 10000.00 | cop[tikv] | table:t1 | keep order:false, stats:pseudo | ++-------------------------+----------+-----------+-----------------+--------------------------------+ +3 rows in set (0.00 sec) +``` + +From the above query results, you can see that the `Union` operator in the execution plan disappears while the partition pruning still takes effect and the execution plan only accesses `p0` and `p1`. + +`dynamic` mode makes execution plans simpler and clearer. Omitting the Union operation can improve the execution efficiency and avoid the problem of Union concurrent execution. In addition, `dynamic` mode also solves two problems that cannot be solved in `static` mode: + ++ Plan Cache cannot be used. (See example 1 and 2) ++ Execution plans with IndexJoin cannot be used. (See example 3 and 4) + +**Example 1**：In the following example, the Plan Cache feature is enabled in the configuration file and the same query is executed twice in `static` mode: + +{{< copyable "sql" >}} + +```sql +mysql> set @a=150; +Query OK, 0 rows affected (0.00 sec) + +mysql> set @@tidb_partition_prune_mode = 'static'; +Query OK, 0 rows affected (0.00 sec) + +mysql> prepare stmt from 'select * from t1 where id < ?'; +Query OK, 0 rows affected (0.00 sec) + +mysql> execute stmt using @a; +Empty set (0.00 sec) + +mysql> execute stmt using @a; +Empty set (0.00 sec) + +-- In static mode, when the same query is executed twice, the cache cannot be hit at the second time. +mysql> select @@last_plan_from_cache; ++------------------------+ +| @@last_plan_from_cache | ++------------------------+ +| 0 | ++------------------------+ +1 row in set (0.00 sec) +``` + +The `last_plan_from_cache` variable can show whether the last query hits the Plan Cache or not. From example 1, you can see that in `static` mode, even if the same query is executed multiple times on the partitioned table, the Plan Cache is not hit. + +**Example 2**: In the following example, the same operations are performed in `dynamic` mode as done in example 1: + +{{< copyable "sql" >}} + +```sql +mysql> set @@tidb_partition_prune_mode = 'dynamic'; +Query OK, 0 rows affected (0.00 sec) + +mysql> prepare stmt from 'select * from t1 where id < ?'; +Query OK, 0 rows affected (0.00 sec) + +mysql> execute stmt using @a; +Empty set (0.00 sec) + +mysql> execute stmt using @a; +Empty set (0.00 sec) + +-- In dynamic mode, the cache can be hit at the second time. +mysql> select @@last_plan_from_cache; ++------------------------+ +| @@last_plan_from_cache | ++------------------------+ +| 1 | ++------------------------+ +1 row in set (0.00 sec) +``` + +From example 2, you can see that in `dynamic` mode, querying the partitioned table hits the Plan Cache. + +**Example 3**: In the following example, a query is performed in `static` mode using the execution plan with IndexJoin: + +{{< copyable "sql" >}} + +```sql +mysql> create table t2(id int, code int); +Query OK, 0 rows affected (0.01 sec) + +mysql> set @@tidb_partition_prune_mode = 'static'; +Query OK, 0 rows affected (0.00 sec) + +mysql> explain select /*+ TIDB_INLJ(t1, t2) */ t1.* from t1, t2 where t2.code = 0 and t2.id = t1.id; ++--------------------------------+----------+-----------+------------------------+------------------------------------------------+ +| id | estRows | task | access object | operator info | ++--------------------------------+----------+-----------+------------------------+------------------------------------------------+ +| HashJoin_13 | 12.49 | root | | inner join, equal:[eq(test.t1.id, test.t2.id)] | +| ├─TableReader_42(Build) | 9.99 | root | | data:Selection_41 | +| │ └─Selection_41 | 9.99 | cop[tikv] | | eq(test.t2.code, 0), not(isnull(test.t2.id)) | +| │ └─TableFullScan_40 | 10000.00 | cop[tikv] | table:t2 | keep order:false, stats:pseudo | +| └─PartitionUnion_15(Probe) | 39960.00 | root | | | +| ├─TableReader_18 | 9990.00 | root | | data:Selection_17 | +| │ └─Selection_17 | 9990.00 | cop[tikv] | | not(isnull(test.t1.id)) | +| │ └─TableFullScan_16 | 10000.00 | cop[tikv] | table:t1, partition:p0 | keep order:false, stats:pseudo | +| ├─TableReader_24 | 9990.00 | root | | data:Selection_23 | +| │ └─Selection_23 | 9990.00 | cop[tikv] | | not(isnull(test.t1.id)) | +| │ └─TableFullScan_22 | 10000.00 | cop[tikv] | table:t1, partition:p1 | keep order:false, stats:pseudo | +| ├─TableReader_30 | 9990.00 | root | | data:Selection_29 | +| │ └─Selection_29 | 9990.00 | cop[tikv] | | not(isnull(test.t1.id)) | +| │ └─TableFullScan_28 | 10000.00 | cop[tikv] | table:t1, partition:p2 | keep order:false, stats:pseudo | +| └─TableReader_36 | 9990.00 | root | | data:Selection_35 | +| └─Selection_35 | 9990.00 | cop[tikv] | | not(isnull(test.t1.id)) | +| └─TableFullScan_34 | 10000.00 | cop[tikv] | table:t1, partition:p3 | keep order:false, stats:pseudo | ++--------------------------------+----------+-----------+------------------------+------------------------------------------------+ +17 rows in set, 1 warning (0.00 sec) +``` + +From example 3, you can see that even if the `TIDB_INLJ` hint is used, the query on the partitioned table cannot select the execution plan with IndexJoin. + +**Example 4**: In the following example, the query is performed in `dynamic` mode using the execution plan with IndexJoin: + +{{< copyable "sql" >}} + +```sql +mysql> set @@tidb_partition_prune_mode = 'dynamic'; +Query OK, 0 rows affected (0.00 sec) + +mysql> explain select /*+ TIDB_INLJ(t1, t2) */ t1.* from t1, t2 where t2.code = 0 and t2.id = t1.id; ++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+ +| id | estRows | task | access object | operator info | ++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+ +| IndexJoin_11 | 12.49 | root | | inner join, inner:IndexLookUp_10, outer key:test.t2.id, inner key:test.t1.id, equal cond:eq(test.t2.id, test.t1.id) | +| ├─TableReader_16(Build) | 9.99 | root | | data:Selection_15 | +| │ └─Selection_15 | 9.99 | cop[tikv] | | eq(test.t2.code, 0), not(isnull(test.t2.id)) | +| │ └─TableFullScan_14 | 10000.00 | cop[tikv] | table:t2 | keep order:false, stats:pseudo | +| └─IndexLookUp_10(Probe) | 1.25 | root | partition:all | | +| ├─Selection_9(Build) | 1.25 | cop[tikv] | | not(isnull(test.t1.id)) | +| │ └─IndexRangeScan_7 | 1.25 | cop[tikv] | table:t1, index:id(id) | range: decided by [eq(test.t1.id, test.t2.id)], keep order:false, stats:pseudo | +| └─TableRowIDScan_8(Probe) | 1.25 | cop[tikv] | table:t1 | keep order:false, stats:pseudo | ++---------------------------------+----------+-----------+------------------------+---------------------------------------------------------------------------------------------------------------------+ +8 rows in set (0.00 sec) +``` + +From example 4, you can see that in `dynamic` mode, the execution plan with IndexJoin is selected when you execute the query. diff --git a/system-variables.md b/system-variables.md index d58b285ef9c5d..451013d360ecd 100644 --- a/system-variables.md +++ b/system-variables.md @@ -501,6 +501,16 @@ Constraint checking is always performed in place for pessimistic transactions (d - Default value: `OFF` - This variable is used to set whether to enable the `LIST (COLUMNS) TABLE PARTITION` feature. +### `tidb_partition_prune_mode` New in v5.1 + +> **Warning:** +> +> Currently, the dynamic mode for partitioned tables is an experimental feature. It is not recommended that you use it in the production environment. + +- Scope: SESSION | GLOBAL +- Default value: `static` +- Specifies whether to enable `dynamic` mode for partitioned tables. For details about the dynamic mode, see [Dynamic Mode for Partitioned Tables](/partitioned-table.md#dynamic-mode). + ### tidb_enable_noop_functions New in v4.0 - Scope: SESSION | GLOBAL