From 0fd0b22b0f4dda9c01bb17167f5634f835d05527 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Wed, 13 Oct 2021 19:51:52 -0600 Subject: [PATCH 01/58] Add initial docs for placement rules --- TOC.md | 6 + placement-rules.md | 142 ++++++++++++++++++ schedule-replicas-by-topology-labels.md | 4 + .../sql-statement-alter-placement-policy.md | 73 +++++++++ .../sql-statement-create-placement-policy.md | 67 +++++++++ .../sql-statement-drop-placement-policy.md | 44 ++++++ ...-statement-show-create-placement-policy.md | 48 ++++++ .../sql-statement-show-placement-for.md | 85 +++++++++++ .../sql-statement-show-placement-labels.md | 43 ++++++ .../sql-statement-show-placement.md | 53 +++++++ 10 files changed, 565 insertions(+) create mode 100644 placement-rules.md create mode 100644 sql-statements/sql-statement-alter-placement-policy.md create mode 100644 sql-statements/sql-statement-create-placement-policy.md create mode 100644 sql-statements/sql-statement-drop-placement-policy.md create mode 100644 sql-statements/sql-statement-show-create-placement-policy.md create mode 100644 sql-statements/sql-statement-show-placement-for.md create mode 100644 sql-statements/sql-statement-show-placement-labels.md create mode 100644 sql-statements/sql-statement-show-placement.md diff --git a/TOC.md b/TOC.md index 9308397b8707e..ecc8aa4e1be12 100644 --- a/TOC.md +++ b/TOC.md @@ -275,6 +275,7 @@ + [`ALTER DATABASE`](/sql-statements/sql-statement-alter-database.md) + [`ALTER INDEX`](/sql-statements/sql-statement-alter-index.md) + [`ALTER INSTANCE`](/sql-statements/sql-statement-alter-instance.md) + + [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) + [`ALTER TABLE`](/sql-statements/sql-statement-alter-table.md) + [`ALTER USER`](/sql-statements/sql-statement-alter-user.md) + [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) @@ -287,6 +288,7 @@ + [`CREATE [GLOBAL|SESSION] BINDING`](/sql-statements/sql-statement-create-binding.md) + [`CREATE DATABASE`](/sql-statements/sql-statement-create-database.md) + [`CREATE INDEX`](/sql-statements/sql-statement-create-index.md) + + [`CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-create-placement-policy.md) + [`CREATE ROLE`](/sql-statements/sql-statement-create-role.md) + [`CREATE SEQUENCE`](/sql-statements/sql-statement-create-sequence.md) + [`CREATE TABLE LIKE`](/sql-statements/sql-statement-create-table-like.md) @@ -346,6 +348,7 @@ + [`SHOW COLLATION`](/sql-statements/sql-statement-show-collation.md) + [`SHOW [FULL] COLUMNS FROM`](/sql-statements/sql-statement-show-columns-from.md) + [`SHOW CONFIG`](/sql-statements/sql-statement-show-config.md) + + [`SHOW CREATE PLACEMENT POLICY`](/sql-statements/sql-statement-show-create-placement-policy.md) + [`SHOW CREATE SEQUENCE`](/sql-statements/sql-statement-show-create-sequence.md) + [`SHOW CREATE TABLE`](/sql-statements/sql-statement-show-create-table.md) + [`SHOW CREATE USER`](/sql-statements/sql-statement-show-create-user.md) @@ -359,6 +362,8 @@ + [`SHOW INDEXES [FROM|IN]`](/sql-statements/sql-statement-show-indexes.md) + [`SHOW KEYS [FROM|IN]`](/sql-statements/sql-statement-show-keys.md) + [`SHOW MASTER STATUS`](/sql-statements/sql-statement-show-master-status.md) + + [`SHOW PLACEMENT`](/sql-statements/sql-statement-show-placement.md) + + [`SHOW PLACEMENT FOR`](/sql-statements/sql-statement-show-placement-for.md) + [`SHOW PLUGINS`](/sql-statements/sql-statement-show-plugins.md) + [`SHOW PRIVILEGES`](/sql-statements/sql-statement-show-privileges.md) + [`SHOW [FULL] PROCESSSLIST`](/sql-statements/sql-statement-show-processlist.md) @@ -426,6 +431,7 @@ + [Views](/views.md) + [Partitioning](/partitioned-table.md) + [Character Set and Collation](/character-set-and-collation.md) + + [Placement Rules](/placement-rules.md) + System Tables + [`mysql`](/mysql-schema.md) + INFORMATION_SCHEMA diff --git a/placement-rules.md b/placement-rules.md new file mode 100644 index 0000000000000..bbd388a2d962b --- /dev/null +++ b/placement-rules.md @@ -0,0 +1,142 @@ +--- +title: Placement Rules +summary: Learn how to schedule placement of tables and partitions. +--- + +# Placement Rules + +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. + +Placement Rules allow you to configure where data will be stored in a TiKV cluster. This is useful for scenarios including optimizing a high availability strategy, ensuring that local copies of data will be available for local stale reads, and adhering to compliance requirements. + +## Specifying placement options + +Placement options can be specified using either _direct placement_ or using a _placement policy_. In the following example, both tables `t1` and `t2` have the same rules: + +```sql +CREATE TABLE t1 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; +CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; +CREATE TABLE t2 (a INT) PLACEMENT POLICY=eastandwest; +``` + +Using placement policies is recommended for complex scenarios because it simplifies management. If any changes are required, they can be updated using the [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) statement. There are no differences in the placement options available between direct placement and placement policies. + +A `PLACEMENT POLICY` is not associated with any database schema and has global scope. Assigning a placement policy does not require any additional privileges over `CREATE TABLE` privilege. + +## Option reference + +> **Note:** +> +> Placement options depend on labels correctly specified in the configuration of each TiKV node. For example, `PRIMARY_REGION` depends on the label `region` in TiKV. To see a summary of all labels available in your TiKV cluster, use the statement [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md): +> +> ```sql +> mysql> show placement labels; +> +--------+----------------+ +> | Key | Values | +> +--------+----------------+ +> | region | ["us-east-1"] | +> | zone | ["us-east-1a"] | +> +--------+----------------+ +> 2 rows in set (0.00 sec) +> ``` + +| Option Name | Description | +|----------------------------|------------------------------------------------------------------------------------------------| +| `PRIMARY_REGION` | Raft leaders will be placed in stores which have the `region` label that match this value. | +| `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | +| `SCHEDULE` | The strategy used to schedule the placement of followers. Either `EVEN` or `MAJORITY_IN_PRIMARY`. | +| `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 copies of the data (2 followers and 1 leader). | +| `LEARNERS` | The number of learners, such as TiFlash nodes. | +| **Advanced configuration** | +| `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | +| `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | +| `LEARNER_CONSTRAINTS` | A list of constraints that only apply to learners. | + +## Examples + +### Increasing the number of replicas + +The default configuration of [`max-replicas`](/pd-configuration-file.md#max-replicas) is `3`. To increase this for a specific set of tables, you can use a placement policy as follows: + +```sql +CREATE PLACEMENT POLICY fivereplicas FOLLOWERS=4; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=fivereplicas; +``` + +Note that the PD configuration includes the leader and follower count, thus 4 followers + 1 leader equals five replicas in total. + +To expand on this example, the placement for the followers can also be described using the `PRIMARY_REGION` and `REGIONS` placement options: + +```sql +CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" SCHEDULE="MAJORITY_IN_PRIMARY" FOLLOWERS=4; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=eastandwest; +``` + +The `SCHEDULE` instructs TiDB how to balance the followers. `MAJORITY_IN_PRIMARY` means that enough followers should be placed in the primary region (`us-east-1`) that quorum can be achieved. This helps performance at the expense of availability should the primary region completely fail. If this is not desired, the schedule of `EVEN` can be used to ensure a balance of followers in all regions. + +### Assigning placement to a partitioned table + +> **Note:** +> +> This example makes use of list partitioning, which is currently an experimental feature. Partitioned tables also require the `PRIMARY KEY` to be included in all columns in the table's partitioning function. + +As well as assigning to tables, placement options can also be assigned to table partitions. For example: + +```sql +CREATE PLACEMENT POLICY europe PRIMARY_REGION="us-east-1" REGIONS="us-east-1"; +CREATE PLACEMENT POLICY northamerica PRIMARY_REGION="us-east-1" REGIONS="us-east-1"; + +SET tidb_enable_list_partition = 1; +CREATE TABLE t1 ( + country VARCHAR(10) NOT NULL, + userdata VARCHAR(100) NOT NULL +) PARTITION BY LIST COLUMNS (country) ( + PARTITION pEurope VALUES IN ('DE', 'FR', 'GB') PLACEMENT POLICY=europe, + PARTITION pNorthAmerica VALUES IN ('US', 'CA', 'MX') PLACEMENT POLICY=northamerica +); +``` + +### Setting the default placement for a schema + +Default placement options can be directly attached to a database schema. This works similar to setting the default character set or collation for a schema, in that it will be used when no other placement options are specified. For example: + +```sql +use test; +CREATE TABLE t1 (a INT); // the table is created with no placement options +ALTER DATABASE test FOLLOWERS=4; // this changes the default, and does not apply to the existing table t1; +CREATE TABLE t2 (a INT); // the placement of FOLLOWERS=4 will be used +CREATE TABLE t3 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2"; // FOLLOWERS=4 does not apply as placement is specified. +ALTER DATABASE test FOLLOWERS=2; // this does not apply to existing tables +CREATE TABLE t4 (a INT); // the table is created with FOLLOWERS=2; +``` + +Because placement options are only inherited from the database schema default when a table is created, it is recommended to set the default to a `PLACEMENT POLICY`. This ensures that future changes to the policy will propagate to existing tables. + +### Advanced placement + +The placement options `PRIMARY_REGION`, `REGIONS` and `SCHEDULE` are designed to support the primary use cases of data placement at the loss of some flexibility. For more complex scenarios it is possible to also use the advanced placement options of `CONSTRAINTS`, `FOLLOWER_CONSTRAINTS` and `LEARNER_CONSTRAINTS`. Use of these optons is mutually exclusive, and an error will be returned if both are specified. + +For example, to set constraints that data must reside on a TiKV store where the label `disk` must match a value: + +```sql +CREATE PLACEMENT POLICY storeonfastssd CONSTRAINTS="[+disk=ssd]"; +CREATE PLACEMENT POLICY storeonhdd CONSTRAINTS="[+disk=hdd]"; + +CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE) + PARTITION BY RANGE( YEAR(purchased) ) ( + PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY='storeonhdd', + PARTITION p1 VALUES LESS THAN (2005), + PARTITION p2 VALUES LESS THAN (2010), + PARTITION p3 VALUES LESS THAN (2015), + PARTITION p4 VALUES LESS THAN MAXVALUE PLACEMENT POLICY='storeonfastssd' + ) +PLACEMENT POLICY='companystandardpolicy'; +``` + +Constraints can either be specified in list format (`[+disk=ssd]`) or dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). + +In list format, constraints are specified as a list of key-value pairs. The key starts with either a `+` or a `-` with `+disk=ssd` indicating that the label `disk` must be set to `ssd`, and `-disk=hdd` indicating that the label `disk` must not be `hdd`. + +In dictionary format, the constraint also indicates a number of instances that apply to that rule. For example `FOLLOWER_CONSTRAINTS="{+region=us-east-1:1,+region=us-east-2:1,+region=us-west-1:1,+any:1}";` indicates that 1 follower is in us-east-1, 1 follower is in us-east-2, 1 follower is in us-west-1, and 1 follower can be in any region. diff --git a/schedule-replicas-by-topology-labels.md b/schedule-replicas-by-topology-labels.md index 3f613b7a4644e..7e6ac4f6cd072 100644 --- a/schedule-replicas-by-topology-labels.md +++ b/schedule-replicas-by-topology-labels.md @@ -6,6 +6,10 @@ aliases: ['/docs/dev/location-awareness/','/docs/dev/how-to/deploy/geographic-re # Schedule Replicas by Topology Labels +> **Note:** +> +> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. + To improve the high availability and disaster recovery capability of TiDB clusters, it is recommended that TiKV nodes are physically scattered as much as possible. For example, TiKV nodes can be distributed on different racks or even in different data centers. According to the topology information of TiKV, the PD scheduler automatically performs scheduling at the background to isolate each replica of a Region as much as possible, which maximizes the capability of disaster recovery. To make this mechanism effective, you need to properly configure TiKV and PD so that the topology information of the cluster, especially the TiKV location information, is reported to PD during deployment. Before you begin, see [Deploy TiDB Using TiUP](/production-deployment-using-tiup.md) first. diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md new file mode 100644 index 0000000000000..b848705c8a52f --- /dev/null +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -0,0 +1,73 @@ +--- +title: ALTER PLACEMENT POLICY +summary: The usage of ALTER PLACEMENT POLICY in TiDB. +--- + +# ALTER PLACEMENT POLICY + +`ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions which use the placement policy will automatically be updated. + +## Synopsis + +```ebnf+diagram +AlterPolicyStmt ::= + "ALTER" "PLACEMENT" "POLICY" IfExists PolicyName PlacementOptionList + +PolicyName ::= + Identifier + +PlacementOptionList ::= + DirectPlacementOption +| PlacementOptionList DirectPlacementOption +| PlacementOptionList ',' DirectPlacementOption + +DirectPlacementOption ::= + "PRIMARY_REGION" EqOpt stringLit +| "REGIONS" EqOpt stringLit +| "FOLLOWERS" EqOpt LengthNum +| "VOTERS" EqOpt LengthNum +| "LEARNERS" EqOpt LengthNum +| "SCHEDULE" EqOpt stringLit +| "CONSTRAINTS" EqOpt stringLit +| "LEADER_CONSTRAINTS" EqOpt stringLit +| "FOLLOWER_CONSTRAINTS" EqOpt stringLit +| "VOTER_CONSTRAINTS" EqOpt stringLit +| "LEARNER_CONSTRAINTS" EqOpt stringLit +``` + +## Examples + +> **Note:** +> +> See [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md) to see which regions are available in your cluster. +> If you do not see any available, your TiKV installation might not have labels set correctly. + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; +ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; +SHOW CREATE PLACEMENT POLICY p1\G +``` + +``` +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.10 sec) + +*************************** 1. row *************************** + Policy: p1 +Create Policy: PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4 +1 row in set (0.00 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md new file mode 100644 index 0000000000000..fb4b4454bafd5 --- /dev/null +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -0,0 +1,67 @@ +--- +title: CREATE PLACEMENT POLICY +summary: The usage of CREATE PLACEMENT POLICY in TiDB. +--- + +# CREATE PLACEMENT POLICY + +`CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions or database schemas. + +## Synopsis + +```ebnf+diagram +CreatePolicyStmt ::= + "CREATE" "PLACEMENT" "POLICY" IfNotExists PolicyName PlacementOptionList + +PolicyName ::= + Identifier + +PlacementOptionList ::= + DirectPlacementOption +| PlacementOptionList DirectPlacementOption +| PlacementOptionList ',' DirectPlacementOption + +DirectPlacementOption ::= + "PRIMARY_REGION" EqOpt stringLit +| "REGIONS" EqOpt stringLit +| "FOLLOWERS" EqOpt LengthNum +| "VOTERS" EqOpt LengthNum +| "LEARNERS" EqOpt LengthNum +| "SCHEDULE" EqOpt stringLit +| "CONSTRAINTS" EqOpt stringLit +| "LEADER_CONSTRAINTS" EqOpt stringLit +| "FOLLOWER_CONSTRAINTS" EqOpt stringLit +| "VOTER_CONSTRAINTS" EqOpt stringLit +| "LEARNER_CONSTRAINTS" EqOpt stringLit +``` + +## Examples + +> **Note:** +> +> See [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md) to see which regions are available in your cluster. +> If you do not see any available, your TiKV installation might not have labels set correctly. + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=p1; +``` + +``` +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.10 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md new file mode 100644 index 0000000000000..451ed1047d941 --- /dev/null +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -0,0 +1,44 @@ +--- +title: DROP PLACEMENT POLICY +summary: The usage of ALTER PLACEMENT POLICY in TiDB. +--- + +# DROP PLACEMENT POLICY + +`DROP PLACEMENT POLICY` + +## Synopsis + +```ebnf+diagram +DropPolicyStmt ::= + "DROP" "PLACEMENT" "POLICY" IfExists PolicyName + +PolicyName ::= + Identifier +``` + +## Examples + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; +ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; +``` + +``` +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.10 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) +* [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md new file mode 100644 index 0000000000000..e978aa075795a --- /dev/null +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -0,0 +1,48 @@ +--- +title: SHOW CREATE PLACEMENT POLICY +summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. +--- + +# SHOW CREATE PLACEMENT POLICY + +`SHOW CREATE PLACEMENT POLICY` is used to show the definition of a placement policy. This can be used to see the current definition of a placement policy and recreate it in another TiDB cluster. + +## Synopsis + +```ebnf+diagram +ShowStmt ::= + "CREATE" "PLACEMENT" "POLICY" +``` + +## Examples + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=p1; +SHOW CREATE PLACEMENT POLICY p1\G +``` + +``` +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.10 sec) + +*************************** 1. row *************************** + Policy: p1 +Create Policy: PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 +1 row in set (0.00 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) +* [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md new file mode 100644 index 0000000000000..9bf08fa5646a5 --- /dev/null +++ b/sql-statements/sql-statement-show-placement-for.md @@ -0,0 +1,85 @@ +--- +title: SHOW PLACEMENT FOR +summary: The usage of SHOW PLACEMENT FOR in TiDB. +--- + +# SHOW PLACEMENT FOR + +`SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in canonical form for a specific table, database schema or partition. + +## Synopsis + +```ebnf+diagram +ShowStmt ::= + "PLACEMENT" "FOR" ShowPlacementTarget + +ShowPlacementTarget ::= + DatabaseSym DBName +| "TABLE" TableName +| "TABLE" TableName "PARTITION" Identifier +``` + +## Examples + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +use test; +ALTER DATABASE test PLACEMENT POLICY=p1; +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +SHOW PLACEMENT FOR DATABASE test; +SHOW PLACEMENT FOR TABLE t1; +SHOW CREATE TABLE t1\G +SHOW PLACEMENT FOR TABLE t2; +``` + +``` +Query OK, 0 rows affected (0.02 sec) + +Query OK, 0 rows affected (0.00 sec) + +Query OK, 0 rows affected (0.00 sec) + +Query OK, 0 rows affected (0.01 sec) + ++---------------+----------------------------------------------------------------------+ +| Target | Placement | ++---------------+----------------------------------------------------------------------+ +| DATABASE test | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | ++---------------+----------------------------------------------------------------------+ +1 row in set (0.00 sec) + ++---------------+----------------------------------------------------------------------+ +| Target | Placement | ++---------------+----------------------------------------------------------------------+ +| TABLE test.t1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | ++---------------+----------------------------------------------------------------------+ +1 row in set (0.00 sec) + +*************************** 1. row *************************** + Table: t1 +Create Table: CREATE TABLE `t1` ( + `a` int(11) DEFAULT NULL +) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin /*T![placement] PLACEMENT POLICY=`p1` */ +1 row in set (0.00 sec) + ++---------------+----------------------------------------------------------------------+ +| Target | Placement | ++---------------+----------------------------------------------------------------------+ +| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | ++---------------+----------------------------------------------------------------------+ +1 row in set (0.00 sec) + +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-placement-labels.md b/sql-statements/sql-statement-show-placement-labels.md new file mode 100644 index 0000000000000..c557c7d130e19 --- /dev/null +++ b/sql-statements/sql-statement-show-placement-labels.md @@ -0,0 +1,43 @@ +--- +title: SHOW PLACEMENT LABELS +summary: The usage of SHOW PLACEMENT LABELS in TiDB. +--- + +# SHOW PLACEMENT LABELS + +`SHOW PLACEMENT LABELS` is used to summarize the labels and values that are available for [placement rules](/placement-rules.md). + +## Synopsis + +```ebnf+diagram +ShowStmt ::= + "PLACEMENT" "LABELS" +``` + +## Examples + +{{< copyable "sql" >}} + +```sql +SHOW PLACEMENT LABELS; +``` + +``` ++--------+----------------+ +| Key | Values | ++--------+----------------+ +| region | ["us-east-1"] | +| zone | ["us-east-1a"] | ++--------+----------------+ +2 rows in set (0.00 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md new file mode 100644 index 0000000000000..4f4df1ef617c4 --- /dev/null +++ b/sql-statements/sql-statement-show-placement.md @@ -0,0 +1,53 @@ +--- +title: SHOW PLACEMENT +summary: The usage of SHOW PLACEMENT in TiDB. +--- + +# SHOW PLACEMENT + +`SHOW PLACEMENT` summarizes all placement options from direct placement and placement policies, and presents them in canonical form. + +## Synopsis + +```ebnf+diagram +ShowStmt ::= + "PLACEMENT" +``` + +## Examples + +{{< copyable "sql" >}} + +```sql +CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +CREATE TABLE t1 (a INT) PLACEMENT POLICY=p1; +CREATE TABLE t2 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; +SHOW PLACEMENT; +``` + +``` +Query OK, 0 rows affected (0.01 sec) + +Query OK, 0 rows affected (0.00 sec) + +Query OK, 0 rows affected (0.00 sec) + ++---------------+----------------------------------------------------------------------+ +| Target | Placement | ++---------------+----------------------------------------------------------------------+ +| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | +| TABLE test.t1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | +| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | ++---------------+----------------------------------------------------------------------+ +3 rows in set (0.00 sec) +``` + +## MySQL compatibility + +This statement is a TiDB extension to MySQL syntax. + +## See also + +* [Placement Rules](/placement-rules.md) +* [SHOW PLACEMENT FOR](/sql-statements/sql-statement-show-placement-for.md) +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file From d762a547c81d8b83f5af3836eddae1a827b9e00c Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 14 Oct 2021 13:39:36 -0600 Subject: [PATCH 02/58] Update with suggestions by Mattias --- placement-rules.md | 19 ++++++++++--------- .../sql-statement-alter-placement-policy.md | 2 +- ...-statement-show-create-placement-policy.md | 2 +- 3 files changed, 12 insertions(+), 11 deletions(-) diff --git a/placement-rules.md b/placement-rules.md index bbd388a2d962b..0dcabb88e7577 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -104,12 +104,12 @@ Default placement options can be directly attached to a database schema. This wo ```sql use test; -CREATE TABLE t1 (a INT); // the table is created with no placement options -ALTER DATABASE test FOLLOWERS=4; // this changes the default, and does not apply to the existing table t1; -CREATE TABLE t2 (a INT); // the placement of FOLLOWERS=4 will be used -CREATE TABLE t3 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2"; // FOLLOWERS=4 does not apply as placement is specified. -ALTER DATABASE test FOLLOWERS=2; // this does not apply to existing tables -CREATE TABLE t4 (a INT); // the table is created with FOLLOWERS=2; +CREATE TABLE t1 (a INT); -- the table is created with no placement options +ALTER DATABASE test FOLLOWERS=4; -- this changes the default, and does not apply to the existing table t1; +CREATE TABLE t2 (a INT); -- the placement of FOLLOWERS=4 will be used +CREATE TABLE t3 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2"; -- FOLLOWERS=4 does not apply as placement is specified. +ALTER DATABASE test FOLLOWERS=2; -- this does not apply to existing tables +CREATE TABLE t4 (a INT); -- the table is created with FOLLOWERS=2; ``` Because placement options are only inherited from the database schema default when a table is created, it is recommended to set the default to a `PLACEMENT POLICY`. This ensures that future changes to the policy will propagate to existing tables. @@ -123,16 +123,17 @@ For example, to set constraints that data must reside on a TiKV store where the ```sql CREATE PLACEMENT POLICY storeonfastssd CONSTRAINTS="[+disk=ssd]"; CREATE PLACEMENT POLICY storeonhdd CONSTRAINTS="[+disk=hdd]"; +CREATE PLACEMENT POLICY companystandardpolicy CONSTRAINTS=""; CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE) PARTITION BY RANGE( YEAR(purchased) ) ( - PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY='storeonhdd', + PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storeonhdd, PARTITION p1 VALUES LESS THAN (2005), PARTITION p2 VALUES LESS THAN (2010), PARTITION p3 VALUES LESS THAN (2015), - PARTITION p4 VALUES LESS THAN MAXVALUE PLACEMENT POLICY='storeonfastssd' + PARTITION p4 VALUES LESS THAN MAXVALUE PLACEMENT POLICY=storeonfastssd ) -PLACEMENT POLICY='companystandardpolicy'; +PLACEMENT POLICY=companystandardpolicy; ``` Constraints can either be specified in list format (`[+disk=ssd]`) or dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index b848705c8a52f..7735d898ba8f6 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -57,7 +57,7 @@ Query OK, 0 rows affected (0.10 sec) *************************** 1. row *************************** Policy: p1 -Create Policy: PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4 +Create Policy: CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4 1 row in set (0.00 sec) ``` diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index e978aa075795a..4cfb10ed68212 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -31,7 +31,7 @@ Query OK, 0 rows affected (0.10 sec) *************************** 1. row *************************** Policy: p1 -Create Policy: PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 +Create Policy: CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 1 row in set (0.00 sec) ``` From 18cbbeee9bc0f8543b98bdc117337f3478ec796d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sat, 16 Oct 2021 13:17:57 -0600 Subject: [PATCH 03/58] Add infoschema --- .../information-schema-placement-rules.md | 80 +++++++++++++++++++ information-schema/information-schema.md | 1 + 2 files changed, 81 insertions(+) create mode 100644 information-schema/information-schema-placement-rules.md diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md new file mode 100644 index 0000000000000..e7042a714f791 --- /dev/null +++ b/information-schema/information-schema-placement-rules.md @@ -0,0 +1,80 @@ +--- +title: PLACEMENT_RULES +summary: Learn the `PLACEMENT_RULES` information_schema table. +--- + +# PLACEMENT_RULES + +The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placment-rules.md). This includes both placement policies, and directly attached rules. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC placement_rules; +``` + +```sql ++----------------------+--------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++----------------------+--------------+------+------+---------+-------+ +| POLICY_ID | bigint(64) | NO | | NULL | | +| CATALOG_NAME | varchar(512) | NO | | NULL | | +| POLICY_NAME | varchar(5) | YES | | NULL | | +| SCHEMA_NAME | varchar(5) | YES | | NULL | | +| TABLE_NAME | varchar(5) | YES | | NULL | | +| PARTITION_NAME | varchar(5) | YES | | NULL | | +| PRIMARY_REGION | varchar(5) | NO | | NULL | | +| REGIONS | varchar(5) | NO | | NULL | | +| CONSTRAINTS | varchar(5) | NO | | NULL | | +| LEADER_CONSTRAINTS | varchar(5) | NO | | NULL | | +| FOLLOWER_CONSTRAINTS | varchar(5) | NO | | NULL | | +| LEARNER_CONSTRAINTS | varchar(5) | NO | | NULL | | +| SCHEDULE | varchar(20) | NO | | NULL | | +| FOLLOWERS | bigint(64) | NO | | NULL | | +| LEARNERS | bigint(64) | NO | | NULL | | ++----------------------+--------------+------+------+---------+-------+ +15 rows in set (0.00 sec) +``` + +## Examples + +The `PLACEMENT_RULES` table only shows explicitly configured rules. To see the canonical version of placement rules (including placement policies attached to tables), use the statement `SHOW PLACEMENT` instead: + +{{< copyable "sql" >}} + +```sql +CREATE TABLE t1 (a INT); +CREATE TABLE t2 (a INT) primary_region="us-east-1" regions="us-east-1"; +CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; +CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1; +SHOW PLACEMENT; -- includes t3 +select * from information_schema.placement_rules; -- does not include t3 +``` + +```sql +Query OK, 0 rows affected (0.09 sec) + +Query OK, 0 rows affected (0.11 sec) + +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.11 sec) + ++---------------+------------------------------------------------+ +| Target | Placement | ++---------------+------------------------------------------------+ +| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | +| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | +| TABLE test.t3 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1" | ++---------------+------------------------------------------------+ +3 rows in set (0.00 sec) + ++-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +| POLICY_ID | CATALOG_NAME | POLICY_NAME | SCHEMA_NAME | TABLE_NAME | PARTITION_NAME | PRIMARY_REGION | REGIONS | CONSTRAINTS | LEADER_CONSTRAINTS | FOLLOWER_CONSTRAINTS | LEARNER_CONSTRAINTS | SCHEDULE | FOLLOWERS | LEARNERS | ++-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +| 3 | def | p1 | NULL | NULL | NULL | us-east-1 | us-east-1 | | | | | | 0 | 0 | +| NULL | def | NULL | test | t2 | NULL | us-east-1 | us-east-1 | | | | | | 0 | 0 | ++-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ +2 rows in set (0.00 sec) +``` \ No newline at end of file diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index c06b656376e69..5ac50eca3b3a8 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -76,6 +76,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`METRICS_SUMMARY`](/information-schema/information-schema-metrics-summary.md) | A summary of metrics extracted from Prometheus. | | `METRICS_SUMMARY_BY_LABEL` | See `METRICS_SUMMARY` table. | | [`METRICS_TABLES`](/information-schema/information-schema-metrics-tables.md) | Provides the PromQL definitions for tables in `METRICS_SCHEMA`. | +| [`PLACEMENT_RULES`](/information-schema/information-schema-placement-rules.md) | Provides information on all objects that have explicit placement rules assigned. | | [`SEQUENCES`](/information-schema/information-schema-sequences.md) | The TiDB implementation of sequences is based on MariaDB. | | [`SLOW_QUERY`](/information-schema/information-schema-slow-query.md) | Provides information on slow queries on the current TiDB server. | | [`STATEMENTS_SUMMARY`](/statement-summary-tables.md) | Similar to performance_schema statement summary in MySQL. | From 5926cdfa511d17d681b6980eeb58ae2daf46d66f Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Wed, 27 Oct 2021 15:11:54 -0600 Subject: [PATCH 04/58] Address review feedback --- configure-placement-rules.md | 4 +++ placement-rules.md | 16 ++++----- schedule-replicas-by-topology-labels.md | 2 +- .../sql-statement-create-placement-policy.md | 8 +++++ .../sql-statement-drop-placement-policy.md | 35 ++++++++++++++++--- ...-statement-show-create-placement-policy.md | 6 ++-- .../sql-statement-show-placement-for.md | 10 ++++++ 7 files changed, 65 insertions(+), 16 deletions(-) diff --git a/configure-placement-rules.md b/configure-placement-rules.md index b28a4526cf270..e78145682fc4f 100644 --- a/configure-placement-rules.md +++ b/configure-placement-rules.md @@ -10,6 +10,10 @@ aliases: ['/docs/dev/configure-placement-rules/','/docs/dev/how-to/configure/pla > > In the scenario of using TiFlash, the Placement Rules feature has been extensively tested and can be used in the production environment. Except for the scenario where TiFlash is used, using Placement Rules alone has not been extensively tested, so it is not recommended to enable this feature separately in the production environment. +> **Note:** +> +> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL is expected to replace placement configuration with PD in the future. + Placement Rules is an experimental feature of the Placement Driver (PD) introduced in v4.0. It is a replica rule system that guides PD to generate corresponding schedules for different types of data. By combining different scheduling rules, you can finely control the attributes of any continuous data range, such as the number of replicas, the storage location, the host type, whether to participate in Raft election, and whether to act as the Raft leader. ## Rule system diff --git a/placement-rules.md b/placement-rules.md index 0dcabb88e7577..6bbf944ead93e 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -21,7 +21,7 @@ CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east- CREATE TABLE t2 (a INT) PLACEMENT POLICY=eastandwest; ``` -Using placement policies is recommended for complex scenarios because it simplifies management. If any changes are required, they can be updated using the [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md) statement. There are no differences in the placement options available between direct placement and placement policies. +Using placement policies is recommended to simplify rule management. Changes made to a placement policy (via [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md)) automatically propagate to all database objects. This differs from direct placement options, which the rules must be altered for each object (tables, partitions). A `PLACEMENT POLICY` is not associated with any database schema and has global scope. Assigning a placement policy does not require any additional privileges over `CREATE TABLE` privilege. @@ -36,10 +36,11 @@ A `PLACEMENT POLICY` is not associated with any database schema and has global s > +--------+----------------+ > | Key | Values | > +--------+----------------+ +> | disk | ["ssd"] | > | region | ["us-east-1"] | > | zone | ["us-east-1a"] | > +--------+----------------+ -> 2 rows in set (0.00 sec) +> 3 rows in set (0.00 sec) > ``` | Option Name | Description | @@ -70,7 +71,7 @@ Note that the PD configuration includes the leader and follower count, thus 4 fo To expand on this example, the placement for the followers can also be described using the `PRIMARY_REGION` and `REGIONS` placement options: ```sql -CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" SCHEDULE="MAJORITY_IN_PRIMARY" FOLLOWERS=4; +CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2,us-west-1" SCHEDULE="MAJORITY_IN_PRIMARY" FOLLOWERS=4; CREATE TABLE t1 (a INT) PLACEMENT POLICY=eastandwest; ``` @@ -85,7 +86,7 @@ The `SCHEDULE` instructs TiDB how to balance the followers. `MAJORITY_IN_PRIMARY As well as assigning to tables, placement options can also be assigned to table partitions. For example: ```sql -CREATE PLACEMENT POLICY europe PRIMARY_REGION="us-east-1" REGIONS="us-east-1"; +CREATE PLACEMENT POLICY europe PRIMARY_REGION="eu-central-1" REGIONS="eu-central-1,eu-west-1"; CREATE PLACEMENT POLICY northamerica PRIMARY_REGION="us-east-1" REGIONS="us-east-1"; SET tidb_enable_list_partition = 1; @@ -103,7 +104,6 @@ CREATE TABLE t1 ( Default placement options can be directly attached to a database schema. This works similar to setting the default character set or collation for a schema, in that it will be used when no other placement options are specified. For example: ```sql -use test; CREATE TABLE t1 (a INT); -- the table is created with no placement options ALTER DATABASE test FOLLOWERS=4; -- this changes the default, and does not apply to the existing table t1; CREATE TABLE t2 (a INT); -- the placement of FOLLOWERS=4 will be used @@ -126,14 +126,14 @@ CREATE PLACEMENT POLICY storeonhdd CONSTRAINTS="[+disk=hdd]"; CREATE PLACEMENT POLICY companystandardpolicy CONSTRAINTS=""; CREATE TABLE t1 (id INT, name VARCHAR(50), purchased DATE) - PARTITION BY RANGE( YEAR(purchased) ) ( +PLACEMENT POLICY=companystandardpolicy +PARTITION BY RANGE( YEAR(purchased) ) ( PARTITION p0 VALUES LESS THAN (2000) PLACEMENT POLICY=storeonhdd, PARTITION p1 VALUES LESS THAN (2005), PARTITION p2 VALUES LESS THAN (2010), PARTITION p3 VALUES LESS THAN (2015), PARTITION p4 VALUES LESS THAN MAXVALUE PLACEMENT POLICY=storeonfastssd - ) -PLACEMENT POLICY=companystandardpolicy; +); ``` Constraints can either be specified in list format (`[+disk=ssd]`) or dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). diff --git a/schedule-replicas-by-topology-labels.md b/schedule-replicas-by-topology-labels.md index 7e6ac4f6cd072..7c7bc8acd4674 100644 --- a/schedule-replicas-by-topology-labels.md +++ b/schedule-replicas-by-topology-labels.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/location-awareness/','/docs/dev/how-to/deploy/geographic-re > **Note:** > -> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. +> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL is expected to replace placement configuration with PD in the future. To improve the high availability and disaster recovery capability of TiDB clusters, it is recommended that TiKV nodes are physically scattered as much as possible. For example, TiKV nodes can be distributed on different racks or even in different data centers. According to the topology information of TiKV, the PD scheduler automatically performs scheduling at the background to isolate each replica of a Region as much as possible, which maximizes the capability of disaster recovery. diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index fb4b4454bafd5..a4d4002ab1eba 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -47,12 +47,20 @@ DirectPlacementOption ::= ```sql CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4; CREATE TABLE t1 (a INT) PLACEMENT POLICY=p1; +SHOW CREATE PLACEMENT POLICY p1; ``` ``` Query OK, 0 rows affected (0.08 sec) Query OK, 0 rows affected (0.10 sec) + ++--------+---------------------------------------------------------------------------------------------------+ +| Policy | Create Policy | ++--------+---------------------------------------------------------------------------------------------------+ +| p1 | CREATE PLACEMENT POLICY `p1` PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | ++--------+---------------------------------------------------------------------------------------------------+ +1 row in set (0.00 sec) ``` ## MySQL compatibility diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 451ed1047d941..aca108ac6b9bf 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -14,22 +14,47 @@ DropPolicyStmt ::= "DROP" "PLACEMENT" "POLICY" IfExists PolicyName PolicyName ::= - Identifier + Identifier ``` ## Examples +Placement policies can only be dropped when they are not referenced by any tables or partitions. + {{< copyable "sql" >}} ```sql -CREATE PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; -ALTER PLACEMENT POLICY p1 PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1,us-west-2" FOLLOWERS=4; -``` +CREATE PLACEMENT POLICY p1 FOLLOWERS=4; +CREATE TABLE t1 (a INT PRIMARY KEY) PLACEMENT POLICY=p1; +DROP PLACEMENT POLICY p1; -- fails + +-- Find which tables and partitions reference the placement policy +SELECT table_schema, table_name FROM information_schema.tables WHERE tidb_placement_policy_name='p1'; +SELECT table_schema, table_name FROM information_schema.partitions WHERE tidb_placement_policy_name='p1'; +ALTER TABLE t1 PLACEMENT POLICY=default; -- Remove the placement policy from t1 +DROP PLACEMENT POLICY p1; -- succeeds ``` -Query OK, 0 rows affected (0.08 sec) +```sql Query OK, 0 rows affected (0.10 sec) + +Query OK, 0 rows affected (0.11 sec) + +ERROR 8241 (HY000): Placement policy 'p1' is still in use + ++--------------+------------+ +| table_schema | table_name | ++--------------+------------+ +| test | t1 | ++--------------+------------+ +1 row in set (0.00 sec) + +Empty set (0.01 sec) + +Query OK, 0 rows affected (0.08 sec) + +Query OK, 0 rows affected (0.21 sec) ``` ## MySQL compatibility diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index 4cfb10ed68212..46c8e13f70e9e 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -10,8 +10,10 @@ summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. ## Synopsis ```ebnf+diagram -ShowStmt ::= - "CREATE" "PLACEMENT" "POLICY" +"SHOW" "CREATE" "PLACEMENT" "POLICY" PolicyName + +PolicyName ::= + Identifier ``` ## Examples diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index 9bf08fa5646a5..ca341ad19d146 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -33,6 +33,8 @@ SHOW PLACEMENT FOR DATABASE test; SHOW PLACEMENT FOR TABLE t1; SHOW CREATE TABLE t1\G SHOW PLACEMENT FOR TABLE t2; +CREATE TABLE t3 (a INT) PARTITION BY RANGE (a) (PARTITION p1 VALUES LESS THAN (10), PARTITION p2 VALUES LESS THAN (20) FOLLOWERS=4); +SHOW TABLE FOR TABLE t3 PARTITION p1; ``` ``` @@ -72,6 +74,14 @@ Create Table: CREATE TABLE `t1` ( +---------------+----------------------------------------------------------------------+ 1 row in set (0.00 sec) +Query OK, 0 rows affected (0.14 sec) + ++----------------------------+-------------+ +| Target | Placement | ++----------------------------+-------------+ +| TABLE test.t3 PARTITION p2 | FOLLOWERS=4 | ++----------------------------+-------------+ +1 row in set (0.00 sec) ``` ## MySQL compatibility From 1a20a6aa95dbc3d16b49a89efd62fa321ab28ca9 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Mon, 1 Nov 2021 21:44:59 -0600 Subject: [PATCH 05/58] Incorporate feedback, add placement-checks sysvar --- placement-rules.md | 11 +++++------ .../sql-statement-alter-placement-policy.md | 6 ++++++ .../sql-statement-create-placement-policy.md | 5 +++++ sql-statements/sql-statement-drop-placement-policy.md | 7 ++++++- .../sql-statement-show-create-placement-policy.md | 5 +++++ sql-statements/sql-statement-show-placement-for.md | 5 +++++ sql-statements/sql-statement-show-placement-labels.md | 5 +++++ sql-statements/sql-statement-show-placement.md | 5 +++++ system-variables.md | 7 +++++++ 9 files changed, 49 insertions(+), 7 deletions(-) diff --git a/placement-rules.md b/placement-rules.md index 6bbf944ead93e..96682bbf862ef 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -1,13 +1,14 @@ --- -title: Placement Rules +title: Placement Rules in SQL summary: Learn how to schedule placement of tables and partitions. --- -# Placement Rules +# Placement Rules in SQL > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. Placement Rules allow you to configure where data will be stored in a TiKV cluster. This is useful for scenarios including optimizing a high availability strategy, ensuring that local copies of data will be available for local stale reads, and adhering to compliance requirements. @@ -47,13 +48,11 @@ A `PLACEMENT POLICY` is not associated with any database schema and has global s |----------------------------|------------------------------------------------------------------------------------------------| | `PRIMARY_REGION` | Raft leaders will be placed in stores which have the `region` label that match this value. | | `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | -| `SCHEDULE` | The strategy used to schedule the placement of followers. Either `EVEN` or `MAJORITY_IN_PRIMARY`. | +| `SCHEDULE` | The strategy used to schedule the placement of followers. Either `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | | `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 copies of the data (2 followers and 1 leader). | -| `LEARNERS` | The number of learners, such as TiFlash nodes. | | **Advanced configuration** | | `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | | `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | -| `LEARNER_CONSTRAINTS` | A list of constraints that only apply to learners. | ## Examples @@ -75,7 +74,7 @@ CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east- CREATE TABLE t1 (a INT) PLACEMENT POLICY=eastandwest; ``` -The `SCHEDULE` instructs TiDB how to balance the followers. `MAJORITY_IN_PRIMARY` means that enough followers should be placed in the primary region (`us-east-1`) that quorum can be achieved. This helps performance at the expense of availability should the primary region completely fail. If this is not desired, the schedule of `EVEN` can be used to ensure a balance of followers in all regions. +The `SCHEDULE` instructs TiDB how to balance the followers. The default schedule of `EVEN` ensures a balance of followers in all regions. The schedule `MAJORITY_IN_PRIMARY` can be used to ensure that enough followers are placed in the primary region (`us-east-1`) that quorum can be achieved. This helps provide lower latency transactions at the expense of availability should the primary region completely fail. ### Assigning placement to a partitioned table diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 7735d898ba8f6..48e5959bcbc56 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -5,6 +5,12 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. # ALTER PLACEMENT POLICY +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + + `ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions which use the placement policy will automatically be updated. ## Synopsis diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index a4d4002ab1eba..79e820e589cdf 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -5,6 +5,11 @@ summary: The usage of CREATE PLACEMENT POLICY in TiDB. # CREATE PLACEMENT POLICY +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + `CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions or database schemas. ## Synopsis diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index aca108ac6b9bf..5528e47779d35 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -5,7 +5,12 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. # DROP PLACEMENT POLICY -`DROP PLACEMENT POLICY` +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + +`DROP PLACEMENT POLICY` is used to drop a previously created placement policy. ## Synopsis diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index 46c8e13f70e9e..3e1bdd55941f3 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -5,6 +5,11 @@ summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. # SHOW CREATE PLACEMENT POLICY +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + `SHOW CREATE PLACEMENT POLICY` is used to show the definition of a placement policy. This can be used to see the current definition of a placement policy and recreate it in another TiDB cluster. ## Synopsis diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index ca341ad19d146..da2f8bf962eca 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -5,6 +5,11 @@ summary: The usage of SHOW PLACEMENT FOR in TiDB. # SHOW PLACEMENT FOR +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + `SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in canonical form for a specific table, database schema or partition. ## Synopsis diff --git a/sql-statements/sql-statement-show-placement-labels.md b/sql-statements/sql-statement-show-placement-labels.md index c557c7d130e19..9b36fa7ca6856 100644 --- a/sql-statements/sql-statement-show-placement-labels.md +++ b/sql-statements/sql-statement-show-placement-labels.md @@ -5,6 +5,11 @@ summary: The usage of SHOW PLACEMENT LABELS in TiDB. # SHOW PLACEMENT LABELS +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + `SHOW PLACEMENT LABELS` is used to summarize the labels and values that are available for [placement rules](/placement-rules.md). ## Synopsis diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md index 4f4df1ef617c4..908362452e078 100644 --- a/sql-statements/sql-statement-show-placement.md +++ b/sql-statements/sql-statement-show-placement.md @@ -5,6 +5,11 @@ summary: The usage of SHOW PLACEMENT in TiDB. # SHOW PLACEMENT +> **Warning:** +> +> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. + `SHOW PLACEMENT` summarizes all placement options from direct placement and placement policies, and presents them in canonical form. ## Synopsis diff --git a/system-variables.md b/system-variables.md index fd88f3fc1b065..719c652757573 100644 --- a/system-variables.md +++ b/system-variables.md @@ -282,6 +282,13 @@ This variable is an alias for `last_insert_id`. > > Unlike in MySQL, the `max_execution_time` system variable currently works on all kinds of statements in TiDB, not only restricted to the `SELECT` statement. The precision of the timeout value is roughly 100ms. This means the statement might not be terminated in accurate milliseconds as you specify. +### placement_checks + +- Scope: SESSION | GLOBAL +- Default value: ON +- This variable controls whether DDL validates that [placement rules](/placement-rules.md) are valid when creating tables. +- It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if placement rules are violated. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. + ### plugin_dir - Scope: INSTANCE From 08e4b5c47291bf18e559aa5fe8d9a30cbb03f637 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Mon, 1 Nov 2021 22:03:39 -0600 Subject: [PATCH 06/58] add scheduling State --- .../sql-statement-show-placement-for.md | 50 +++++++++++-------- .../sql-statement-show-placement.md | 17 ++++--- 2 files changed, 38 insertions(+), 29 deletions(-) diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index da2f8bf962eca..c23e6f82f6d29 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -39,7 +39,8 @@ SHOW PLACEMENT FOR TABLE t1; SHOW CREATE TABLE t1\G SHOW PLACEMENT FOR TABLE t2; CREATE TABLE t3 (a INT) PARTITION BY RANGE (a) (PARTITION p1 VALUES LESS THAN (10), PARTITION p2 VALUES LESS THAN (20) FOLLOWERS=4); -SHOW TABLE FOR TABLE t3 PARTITION p1; +SHOW PLACEMENT FOR TABLE t3 PARTITION p1; +SHOW PLACEMENT FOR TABLE t3 PARTITION p2; ``` ``` @@ -51,18 +52,18 @@ Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.01 sec) -+---------------+----------------------------------------------------------------------+ -| Target | Placement | -+---------------+----------------------------------------------------------------------+ -| DATABASE test | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -+---------------+----------------------------------------------------------------------+ ++---------------+----------------------------------------------------------------------+------------------+ +| Target | Placement | Scheduling_State | ++---------------+----------------------------------------------------------------------+------------------+ +| DATABASE test | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | INPROGRESS | ++---------------+----------------------------------------------------------------------+------------------+ 1 row in set (0.00 sec) -+---------------+----------------------------------------------------------------------+ -| Target | Placement | -+---------------+----------------------------------------------------------------------+ -| TABLE test.t1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -+---------------+----------------------------------------------------------------------+ ++---------------+-------------+------------------+ +| Target | Placement | Scheduling_State | ++---------------+-------------+------------------+ +| TABLE test.t1 | FOLLOWERS=4 | INPROGRESS | ++---------------+-------------+------------------+ 1 row in set (0.00 sec) *************************** 1. row *************************** @@ -72,20 +73,27 @@ Create Table: CREATE TABLE `t1` ( ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin /*T![placement] PLACEMENT POLICY=`p1` */ 1 row in set (0.00 sec) -+---------------+----------------------------------------------------------------------+ -| Target | Placement | -+---------------+----------------------------------------------------------------------+ -| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -+---------------+----------------------------------------------------------------------+ ++---------------+----------------------------------------------------------------------+------------------+ +| Target | Placement | Scheduling_State | ++---------------+----------------------------------------------------------------------+------------------+ +| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | INPROGRESS | ++---------------+----------------------------------------------------------------------+------------------+ 1 row in set (0.00 sec) Query OK, 0 rows affected (0.14 sec) -+----------------------------+-------------+ -| Target | Placement | -+----------------------------+-------------+ -| TABLE test.t3 PARTITION p2 | FOLLOWERS=4 | -+----------------------------+-------------+ ++----------------------------+-----------------------------------------------------------------------+------------------+ +| Target | Placement | Scheduling_State | ++----------------------------+-----------------------------------------------------------------------+------------------+ +| TABLE test.t3 PARTITION p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,,us-west-1" FOLLOWERS=4 | INPROGRESS | ++----------------------------+-----------------------------------------------------------------------+------------------+ +1 row in set (0.00 sec) + ++----------------------------+-------------+------------------+ +| Target | Placement | Scheduling_State | ++----------------------------+-------------+------------------+ +| TABLE test.t3 PARTITION p2 | FOLLOWERS=4 | INPROGRESS | ++----------------------------+-------------+------------------+ 1 row in set (0.00 sec) ``` diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md index 908362452e078..86f477653907c 100644 --- a/sql-statements/sql-statement-show-placement.md +++ b/sql-statements/sql-statement-show-placement.md @@ -37,14 +37,15 @@ Query OK, 0 rows affected (0.00 sec) Query OK, 0 rows affected (0.00 sec) -+---------------+----------------------------------------------------------------------+ -| Target | Placement | -+---------------+----------------------------------------------------------------------+ -| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -| TABLE test.t1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | -+---------------+----------------------------------------------------------------------+ -3 rows in set (0.00 sec) ++---------------+----------------------------------------------------------------------+------------------+ +| Target | Placement | Scheduling_State | ++---------------+----------------------------------------------------------------------+------------------+ +| POLICY p1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | NULL | +| DATABASE test | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | INPROGRESS | +| TABLE test.t1 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | INPROGRESS | +| TABLE test.t2 | PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1" FOLLOWERS=4 | INPROGRESS | ++---------------+----------------------------------------------------------------------+------------------+ +4 rows in set (0.00 sec) ``` ## MySQL compatibility From e4b8655c3eb500e25cc9c34065ba894b1a87a8c2 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Tue, 2 Nov 2021 18:21:29 -0600 Subject: [PATCH 07/58] Add limitations section --- .../information-schema-placement-rules.md | 4 ++-- placement-rules.md | 11 +++++++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md index e7042a714f791..3ac0b2c344d18 100644 --- a/information-schema/information-schema-placement-rules.md +++ b/information-schema/information-schema-placement-rules.md @@ -5,7 +5,7 @@ summary: Learn the `PLACEMENT_RULES` information_schema table. # PLACEMENT_RULES -The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placment-rules.md). This includes both placement policies, and directly attached rules. +The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placement-rules.md). This includes both placement policies, and directly attached rules. {{< copyable "sql" >}} @@ -77,4 +77,4 @@ Query OK, 0 rows affected (0.11 sec) | NULL | def | NULL | test | t2 | NULL | us-east-1 | us-east-1 | | | | | | 0 | 0 | +-----------+--------------+-------------+-------------+------------+----------------+----------------+-----------+-------------+--------------------+----------------------+---------------------+----------+-----------+----------+ 2 rows in set (0.00 sec) -``` \ No newline at end of file +``` diff --git a/placement-rules.md b/placement-rules.md index 96682bbf862ef..56d6ca4c3362d 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -140,3 +140,14 @@ Constraints can either be specified in list format (`[+disk=ssd]`) or dictionary In list format, constraints are specified as a list of key-value pairs. The key starts with either a `+` or a `-` with `+disk=ssd` indicating that the label `disk` must be set to `ssd`, and `-disk=hdd` indicating that the label `disk` must not be `hdd`. In dictionary format, the constraint also indicates a number of instances that apply to that rule. For example `FOLLOWER_CONSTRAINTS="{+region=us-east-1:1,+region=us-east-2:1,+region=us-west-1:1,+any:1}";` indicates that 1 follower is in us-east-1, 1 follower is in us-east-2, 1 follower is in us-west-1, and 1 follower can be in any region. + +## Known limitations + +The following known limitations exist in the experimental release of Placement Rules in SQL: + +* Dumpling does not support dumping placement policies. See [issue #29371](https://github.com/pingcap/tidb/issues/29371) +* TiDB tools including BR, CDC, Lightning and DM do not yet support placement rules. +* Temporary tables do not support placement options (either via direct placement or placement policies). +* Syntactic sugar rules exist for setting `PRIMARY_REGION` and `REGIONS`, but in future we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE` and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030) +* TiFlash learners are not configurable through Placement Rules syntax. +* Placement rules only ensure that data at rest resides on the correct TiKV store. It does not guarantee that data in transit (via either user-queries or internal operations) will only occur in a specific region. We plan to offer additional features in future to better support compliance use-cases. From 2db5cda2c92253adcaefcd5bde018946d852f54f Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Tue, 2 Nov 2021 21:58:10 -0600 Subject: [PATCH 08/58] fix linter issues --- placement-rules.md | 8 ++--- .../sql-statement-alter-placement-policy.md | 31 +++++++++---------- .../sql-statement-create-placement-policy.md | 30 +++++++++--------- ...-statement-show-create-placement-policy.md | 2 +- .../sql-statement-show-placement-for.md | 6 ++-- 5 files changed, 38 insertions(+), 39 deletions(-) diff --git a/placement-rules.md b/placement-rules.md index 56d6ca4c3362d..27b454d5f3523 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -90,11 +90,11 @@ CREATE PLACEMENT POLICY northamerica PRIMARY_REGION="us-east-1" REGIONS="us-east SET tidb_enable_list_partition = 1; CREATE TABLE t1 ( - country VARCHAR(10) NOT NULL, - userdata VARCHAR(100) NOT NULL + country VARCHAR(10) NOT NULL, + userdata VARCHAR(100) NOT NULL ) PARTITION BY LIST COLUMNS (country) ( - PARTITION pEurope VALUES IN ('DE', 'FR', 'GB') PLACEMENT POLICY=europe, - PARTITION pNorthAmerica VALUES IN ('US', 'CA', 'MX') PLACEMENT POLICY=northamerica + PARTITION pEurope VALUES IN ('DE', 'FR', 'GB') PLACEMENT POLICY=europe, + PARTITION pNorthAmerica VALUES IN ('US', 'CA', 'MX') PLACEMENT POLICY=northamerica ); ``` diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 48e5959bcbc56..c2448bf1600f3 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -10,7 +10,6 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. - `ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions which use the placement policy will automatically be updated. ## Synopsis @@ -20,25 +19,25 @@ AlterPolicyStmt ::= "ALTER" "PLACEMENT" "POLICY" IfExists PolicyName PlacementOptionList PolicyName ::= - Identifier + Identifier PlacementOptionList ::= - DirectPlacementOption -| PlacementOptionList DirectPlacementOption -| PlacementOptionList ',' DirectPlacementOption + DirectPlacementOption +| PlacementOptionList DirectPlacementOption +| PlacementOptionList ',' DirectPlacementOption DirectPlacementOption ::= - "PRIMARY_REGION" EqOpt stringLit -| "REGIONS" EqOpt stringLit -| "FOLLOWERS" EqOpt LengthNum -| "VOTERS" EqOpt LengthNum -| "LEARNERS" EqOpt LengthNum -| "SCHEDULE" EqOpt stringLit -| "CONSTRAINTS" EqOpt stringLit -| "LEADER_CONSTRAINTS" EqOpt stringLit -| "FOLLOWER_CONSTRAINTS" EqOpt stringLit -| "VOTER_CONSTRAINTS" EqOpt stringLit -| "LEARNER_CONSTRAINTS" EqOpt stringLit + "PRIMARY_REGION" EqOpt stringLit +| "REGIONS" EqOpt stringLit +| "FOLLOWERS" EqOpt LengthNum +| "VOTERS" EqOpt LengthNum +| "LEARNERS" EqOpt LengthNum +| "SCHEDULE" EqOpt stringLit +| "CONSTRAINTS" EqOpt stringLit +| "LEADER_CONSTRAINTS" EqOpt stringLit +| "FOLLOWER_CONSTRAINTS" EqOpt stringLit +| "VOTER_CONSTRAINTS" EqOpt stringLit +| "LEARNER_CONSTRAINTS" EqOpt stringLit ``` ## Examples diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 79e820e589cdf..5ae097a29dc99 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -19,25 +19,25 @@ CreatePolicyStmt ::= "CREATE" "PLACEMENT" "POLICY" IfNotExists PolicyName PlacementOptionList PolicyName ::= - Identifier + Identifier PlacementOptionList ::= - DirectPlacementOption -| PlacementOptionList DirectPlacementOption -| PlacementOptionList ',' DirectPlacementOption + DirectPlacementOption +| PlacementOptionList DirectPlacementOption +| PlacementOptionList ',' DirectPlacementOption DirectPlacementOption ::= - "PRIMARY_REGION" EqOpt stringLit -| "REGIONS" EqOpt stringLit -| "FOLLOWERS" EqOpt LengthNum -| "VOTERS" EqOpt LengthNum -| "LEARNERS" EqOpt LengthNum -| "SCHEDULE" EqOpt stringLit -| "CONSTRAINTS" EqOpt stringLit -| "LEADER_CONSTRAINTS" EqOpt stringLit -| "FOLLOWER_CONSTRAINTS" EqOpt stringLit -| "VOTER_CONSTRAINTS" EqOpt stringLit -| "LEARNER_CONSTRAINTS" EqOpt stringLit + "PRIMARY_REGION" EqOpt stringLit +| "REGIONS" EqOpt stringLit +| "FOLLOWERS" EqOpt LengthNum +| "VOTERS" EqOpt LengthNum +| "LEARNERS" EqOpt LengthNum +| "SCHEDULE" EqOpt stringLit +| "CONSTRAINTS" EqOpt stringLit +| "LEADER_CONSTRAINTS" EqOpt stringLit +| "FOLLOWER_CONSTRAINTS" EqOpt stringLit +| "VOTER_CONSTRAINTS" EqOpt stringLit +| "LEARNER_CONSTRAINTS" EqOpt stringLit ``` ## Examples diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index 3e1bdd55941f3..b1ee5bfff19dc 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -18,7 +18,7 @@ summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. "SHOW" "CREATE" "PLACEMENT" "POLICY" PolicyName PolicyName ::= - Identifier + Identifier ``` ## Examples diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index c23e6f82f6d29..79281b19cd101 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -19,9 +19,9 @@ ShowStmt ::= "PLACEMENT" "FOR" ShowPlacementTarget ShowPlacementTarget ::= - DatabaseSym DBName -| "TABLE" TableName -| "TABLE" TableName "PARTITION" Identifier + DatabaseSym DBName +| "TABLE" TableName +| "TABLE" TableName "PARTITION" Identifier ``` ## Examples From 925ac6f5f98b7554d666b4d1e8a85808e3b673d9 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:26:30 -0600 Subject: [PATCH 09/58] Update configure-placement-rules.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- configure-placement-rules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/configure-placement-rules.md b/configure-placement-rules.md index e78145682fc4f..5d42598d069ec 100644 --- a/configure-placement-rules.md +++ b/configure-placement-rules.md @@ -12,7 +12,7 @@ aliases: ['/docs/dev/configure-placement-rules/','/docs/dev/how-to/configure/pla > **Note:** > -> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL is expected to replace placement configuration with PD in the future. +> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. Placement Rules is an experimental feature of the Placement Driver (PD) introduced in v4.0. It is a replica rule system that guides PD to generate corresponding schedules for different types of data. By combining different scheduling rules, you can finely control the attributes of any continuous data range, such as the number of replicas, the storage location, the host type, whether to participate in Raft election, and whether to act as the Raft leader. From 6843f0b96de559f8106e8654ae9a2c07d59aa949 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:27:17 -0600 Subject: [PATCH 10/58] Update placement-rules.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules.md b/placement-rules.md index 27b454d5f3523..f3e950302b71f 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -7,7 +7,7 @@ summary: Learn how to schedule placement of tables and partitions. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. Placement Rules allow you to configure where data will be stored in a TiKV cluster. This is useful for scenarios including optimizing a high availability strategy, ensuring that local copies of data will be available for local stale reads, and adhering to compliance requirements. From 5dc6675fbf2f6f1fff69a83e150a986791c81db3 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:27:24 -0600 Subject: [PATCH 11/58] Update schedule-replicas-by-topology-labels.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- schedule-replicas-by-topology-labels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/schedule-replicas-by-topology-labels.md b/schedule-replicas-by-topology-labels.md index 7c7bc8acd4674..51e175f2c4110 100644 --- a/schedule-replicas-by-topology-labels.md +++ b/schedule-replicas-by-topology-labels.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/location-awareness/','/docs/dev/how-to/deploy/geographic-re > **Note:** > -> TiDB 5.3 introduces experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL is expected to replace placement configuration with PD in the future. +> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. To improve the high availability and disaster recovery capability of TiDB clusters, it is recommended that TiKV nodes are physically scattered as much as possible. For example, TiKV nodes can be distributed on different racks or even in different data centers. According to the topology information of TiKV, the PD scheduler automatically performs scheduling at the background to isolate each replica of a Region as much as possible, which maximizes the capability of disaster recovery. From 0e7c8bd9d828fb9eb98378e5f49e8d0118d9646e Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:06 -0600 Subject: [PATCH 12/58] Update sql-statements/sql-statement-alter-placement-policy.md Co-authored-by: Grace Cai --- sql-statements/sql-statement-alter-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index c2448bf1600f3..8f20d76c48337 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -44,8 +44,8 @@ DirectPlacementOption ::= > **Note:** > -> See [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md) to see which regions are available in your cluster. -> If you do not see any available, your TiKV installation might not have labels set correctly. +> To know which regions are available in your cluster, see [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md). +> If you do not see any available regions, your TiKV installation might not have labels set correctly. {{< copyable "sql" >}} From ef953aef96b54e82e8dee8ac5ff77d972fc63b7a Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:17 -0600 Subject: [PATCH 13/58] Update sql-statements/sql-statement-create-placement-policy.md Co-authored-by: Grace Cai --- sql-statements/sql-statement-create-placement-policy.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 5ae097a29dc99..a333b82fa2216 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -10,7 +10,7 @@ summary: The usage of CREATE PLACEMENT POLICY in TiDB. > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. -`CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions or database schemas. +`CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions, or database schemas. ## Synopsis From df9fea4cb810ba2f9e70d4dd2b65afca51d5b39d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:24 -0600 Subject: [PATCH 14/58] Update sql-statements/sql-statement-create-placement-policy.md Co-authored-by: Grace Cai --- sql-statements/sql-statement-create-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index a333b82fa2216..27014b0077247 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -44,8 +44,8 @@ DirectPlacementOption ::= > **Note:** > -> See [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md) to see which regions are available in your cluster. -> If you do not see any available, your TiKV installation might not have labels set correctly. +> To know which regions are available in your cluster, see [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md). +> If you do not see any available regions, your TiKV installation might not have labels set correctly. {{< copyable "sql" >}} From 9929b5ffabe8a03cf7b4470f76f0974885561df9 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:33 -0600 Subject: [PATCH 15/58] Update sql-statements/sql-statement-show-placement-for.md Co-authored-by: Grace Cai --- sql-statements/sql-statement-show-placement-for.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index 79281b19cd101..fa004b66f1e78 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -10,7 +10,7 @@ summary: The usage of SHOW PLACEMENT FOR in TiDB. > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. -`SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in canonical form for a specific table, database schema or partition. +`SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in the canonical form for a specific table, database schema, or partition. ## Synopsis From ac3d6d959e92653c49aab082f94852021345d3e8 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:46 -0600 Subject: [PATCH 16/58] Update information-schema/information-schema-placement-rules.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- information-schema/information-schema-placement-rules.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md index 3ac0b2c344d18..76e7f1eda9937 100644 --- a/information-schema/information-schema-placement-rules.md +++ b/information-schema/information-schema-placement-rules.md @@ -48,8 +48,8 @@ CREATE TABLE t1 (a INT); CREATE TABLE t2 (a INT) primary_region="us-east-1" regions="us-east-1"; CREATE PLACEMENT POLICY p1 primary_region="us-east-1" regions="us-east-1"; CREATE TABLE t3 (a INT) PLACEMENT POLICY=p1; -SHOW PLACEMENT; -- includes t3 -select * from information_schema.placement_rules; -- does not include t3 +SHOW PLACEMENT; -- Includes t3. +SELECT * FROM information_schema.placement_rules; -- Does not include t3. ``` ```sql From 312a3639b16c396fc26e97d6ddc24485f911ad95 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:28:55 -0600 Subject: [PATCH 17/58] Update placement-rules.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/placement-rules.md b/placement-rules.md index f3e950302b71f..8de127c960770 100644 --- a/placement-rules.md +++ b/placement-rules.md @@ -8,7 +8,8 @@ summary: Learn how to schedule placement of tables and partitions. > **Warning:** > > Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. Placement Rules allow you to configure where data will be stored in a TiKV cluster. This is useful for scenarios including optimizing a high availability strategy, ensuring that local copies of data will be available for local stale reads, and adhering to compliance requirements. From 9659ada676ec55d2d0d69e827b849d957da4a1df Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:29:07 -0600 Subject: [PATCH 18/58] Update information-schema/information-schema-placement-rules.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- information-schema/information-schema-placement-rules.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md index 76e7f1eda9937..f194b0fe78300 100644 --- a/information-schema/information-schema-placement-rules.md +++ b/information-schema/information-schema-placement-rules.md @@ -5,7 +5,7 @@ summary: Learn the `PLACEMENT_RULES` information_schema table. # PLACEMENT_RULES -The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placement-rules.md). This includes both placement policies, and directly attached rules. +The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placement-rules.md). The information includes both placement policies and directly attached rules. {{< copyable "sql" >}} From 66a84518a324b35eb353f96cce7061cfd71e6737 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Thu, 4 Nov 2021 12:38:03 -0600 Subject: [PATCH 19/58] Rename placement-rules.md to placement-rules-in-sql.md --- TOC.md | 2 +- configure-placement-rules.md | 2 +- information-schema/information-schema-placement-rules.md | 2 +- placement-rules.md => placement-rules-in-sql.md | 2 +- schedule-replicas-by-topology-labels.md | 2 +- sql-statements/sql-statement-alter-placement-policy.md | 2 +- sql-statements/sql-statement-create-placement-policy.md | 2 +- sql-statements/sql-statement-drop-placement-policy.md | 2 +- sql-statements/sql-statement-show-create-placement-policy.md | 2 +- sql-statements/sql-statement-show-placement-for.md | 2 +- sql-statements/sql-statement-show-placement-labels.md | 4 ++-- sql-statements/sql-statement-show-placement.md | 2 +- system-variables.md | 2 +- 13 files changed, 14 insertions(+), 14 deletions(-) rename placement-rules.md => placement-rules-in-sql.md (98%) diff --git a/TOC.md b/TOC.md index d7fdee93ea9f8..19b052c5e6103 100644 --- a/TOC.md +++ b/TOC.md @@ -431,7 +431,7 @@ + [Views](/views.md) + [Partitioning](/partitioned-table.md) + [Character Set and Collation](/character-set-and-collation.md) - + [Placement Rules](/placement-rules.md) + + [Placement Rules in SQL](/placement-rules-in-sql.md) + System Tables + [`mysql`](/mysql-schema.md) + INFORMATION_SCHEMA diff --git a/configure-placement-rules.md b/configure-placement-rules.md index 5d42598d069ec..79b9134e6ba51 100644 --- a/configure-placement-rules.md +++ b/configure-placement-rules.md @@ -12,7 +12,7 @@ aliases: ['/docs/dev/configure-placement-rules/','/docs/dev/how-to/configure/pla > **Note:** > -> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. +> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules-in-sql.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. Placement Rules is an experimental feature of the Placement Driver (PD) introduced in v4.0. It is a replica rule system that guides PD to generate corresponding schedules for different types of data. By combining different scheduling rules, you can finely control the attributes of any continuous data range, such as the number of replicas, the storage location, the host type, whether to participate in Raft election, and whether to act as the Raft leader. diff --git a/information-schema/information-schema-placement-rules.md b/information-schema/information-schema-placement-rules.md index f194b0fe78300..34a51747de883 100644 --- a/information-schema/information-schema-placement-rules.md +++ b/information-schema/information-schema-placement-rules.md @@ -5,7 +5,7 @@ summary: Learn the `PLACEMENT_RULES` information_schema table. # PLACEMENT_RULES -The `PLACEMENT_RULES` table provides information on all explicitly configured [placement rules](/placement-rules.md). The information includes both placement policies and directly attached rules. +The `PLACEMENT_RULES` table provides information on all explicitly configured [Placement Rules in SQL](/placement-rules-in-sql.md). The information includes both placement policies and directly attached rules. {{< copyable "sql" >}} diff --git a/placement-rules.md b/placement-rules-in-sql.md similarity index 98% rename from placement-rules.md rename to placement-rules-in-sql.md index 8de127c960770..d78dc43e28f1f 100644 --- a/placement-rules.md +++ b/placement-rules-in-sql.md @@ -116,7 +116,7 @@ Because placement options are only inherited from the database schema default wh ### Advanced placement -The placement options `PRIMARY_REGION`, `REGIONS` and `SCHEDULE` are designed to support the primary use cases of data placement at the loss of some flexibility. For more complex scenarios it is possible to also use the advanced placement options of `CONSTRAINTS`, `FOLLOWER_CONSTRAINTS` and `LEARNER_CONSTRAINTS`. Use of these optons is mutually exclusive, and an error will be returned if both are specified. +The placement options `PRIMARY_REGION`, `REGIONS` and `SCHEDULE` are designed to support the primary use cases of data placement at the loss of some flexibility. For more complex scenarios it is possible to also use the advanced placement options of `CONSTRAINTS` and `FOLLOWER_CONSTRAINTS`. Use of these optons is mutually exclusive, and an error will be returned if both are specified. For example, to set constraints that data must reside on a TiKV store where the label `disk` must match a value: diff --git a/schedule-replicas-by-topology-labels.md b/schedule-replicas-by-topology-labels.md index 51e175f2c4110..fd5f1197c2326 100644 --- a/schedule-replicas-by-topology-labels.md +++ b/schedule-replicas-by-topology-labels.md @@ -8,7 +8,7 @@ aliases: ['/docs/dev/location-awareness/','/docs/dev/how-to/deploy/geographic-re > **Note:** > -> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. +> TiDB v5.3.0 introduces an experimental support for [Placement Rules in SQL](/placement-rules-in-sql.md). This offers a more convenient way to configure the placement of tables and partitions. Placement Rules in SQL might replace placement configuration with PD in future releases. To improve the high availability and disaster recovery capability of TiDB clusters, it is recommended that TiKV nodes are physically scattered as much as possible. For example, TiKV nodes can be distributed on different racks or even in different data centers. According to the topology information of TiKV, the PD scheduler automatically performs scheduling at the background to isolate each replica of a Region as much as possible, which maximizes the capability of disaster recovery. diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 8f20d76c48337..0f0d63c44e5f5 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -72,7 +72,7 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) * [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 27014b0077247..102055a459e8c 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -74,7 +74,7 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) * [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 5528e47779d35..0b6b9a7569f92 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -68,7 +68,7 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) * [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index b1ee5bfff19dc..b417285c38082 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -48,7 +48,7 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) * [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index fa004b66f1e78..0ae8d2c7171c2 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -103,6 +103,6 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-placement-labels.md b/sql-statements/sql-statement-show-placement-labels.md index 9b36fa7ca6856..37c4b220ff9ff 100644 --- a/sql-statements/sql-statement-show-placement-labels.md +++ b/sql-statements/sql-statement-show-placement-labels.md @@ -10,7 +10,7 @@ summary: The usage of SHOW PLACEMENT LABELS in TiDB. > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. -`SHOW PLACEMENT LABELS` is used to summarize the labels and values that are available for [placement rules](/placement-rules.md). +`SHOW PLACEMENT LABELS` is used to summarize the labels and values that are available for Placement Rules. ## Synopsis @@ -43,6 +43,6 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md index 86f477653907c..a13eb882eec3d 100644 --- a/sql-statements/sql-statement-show-placement.md +++ b/sql-statements/sql-statement-show-placement.md @@ -54,6 +54,6 @@ This statement is a TiDB extension to MySQL syntax. ## See also -* [Placement Rules](/placement-rules.md) +* [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT FOR](/sql-statements/sql-statement-show-placement-for.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file diff --git a/system-variables.md b/system-variables.md index 719c652757573..a0071e5b47e26 100644 --- a/system-variables.md +++ b/system-variables.md @@ -286,7 +286,7 @@ This variable is an alias for `last_insert_id`. - Scope: SESSION | GLOBAL - Default value: ON -- This variable controls whether DDL validates that [placement rules](/placement-rules.md) are valid when creating tables. +- This variable controls whether DDL validates that [Placement Rules in SQL](/placement-rules-in-sql.md) are valid when creating tables. - It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if placement rules are violated. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. ### plugin_dir From 27cafc66f2479821ed2469e070b290c132efe779 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:01:00 -0600 Subject: [PATCH 20/58] Update sql-statements/sql-statement-create-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-create-placement-policy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 102055a459e8c..417cccce12527 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -45,6 +45,7 @@ DirectPlacementOption ::= > **Note:** > > To know which regions are available in your cluster, see [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md). +> > If you do not see any available regions, your TiKV installation might not have labels set correctly. {{< copyable "sql" >}} From 91b50d5bac809509dedd40ac211a5a1e9c2275f7 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:01:12 -0600 Subject: [PATCH 21/58] Update sql-statements/sql-statement-alter-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-alter-placement-policy.md | 1 + 1 file changed, 1 insertion(+) diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index 0f0d63c44e5f5..cd9b134b84ec4 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -45,6 +45,7 @@ DirectPlacementOption ::= > **Note:** > > To know which regions are available in your cluster, see [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md). +> > If you do not see any available regions, your TiKV installation might not have labels set correctly. {{< copyable "sql" >}} From 83a799a5319c2629587bd2b5f89863964186ba7b Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:01:30 -0600 Subject: [PATCH 22/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index d78dc43e28f1f..e65c436683d00 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -116,7 +116,7 @@ Because placement options are only inherited from the database schema default wh ### Advanced placement -The placement options `PRIMARY_REGION`, `REGIONS` and `SCHEDULE` are designed to support the primary use cases of data placement at the loss of some flexibility. For more complex scenarios it is possible to also use the advanced placement options of `CONSTRAINTS` and `FOLLOWER_CONSTRAINTS`. Use of these optons is mutually exclusive, and an error will be returned if both are specified. +The placement options `PRIMARY_REGION`, `REGIONS`, and `SCHEDULE` meet the basic needs of data placement at the loss of some flexibility. For more complex scenarios with the need for higher flexibility, you can also use the advanced placement options of `CONSTRAINTS` and `FOLLOWER_CONSTRAINTS`. These two options are mutually exclusive. If you specify both at the same time, an error will be returned. For example, to set constraints that data must reside on a TiKV store where the label `disk` must match a value: From bdf132350051e32db8c6a2fdd633cf273c2249a9 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:01:39 -0600 Subject: [PATCH 23/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index e65c436683d00..eab6ac7fd1ed3 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -136,7 +136,7 @@ PARTITION BY RANGE( YEAR(purchased) ) ( ); ``` -Constraints can either be specified in list format (`[+disk=ssd]`) or dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). +You can either specify constraints in list format (`[+disk=ssd]`) or in dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). In list format, constraints are specified as a list of key-value pairs. The key starts with either a `+` or a `-` with `+disk=ssd` indicating that the label `disk` must be set to `ssd`, and `-disk=hdd` indicating that the label `disk` must not be `hdd`. From 42749fdc4f339ae337a5e262788f3212a986a84e Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:01:50 -0600 Subject: [PATCH 24/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index eab6ac7fd1ed3..e4af9783dc0c0 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -138,9 +138,9 @@ PARTITION BY RANGE( YEAR(purchased) ) ( You can either specify constraints in list format (`[+disk=ssd]`) or in dictionary format (`{+disk=ssd:1,+disk=hdd:2}`). -In list format, constraints are specified as a list of key-value pairs. The key starts with either a `+` or a `-` with `+disk=ssd` indicating that the label `disk` must be set to `ssd`, and `-disk=hdd` indicating that the label `disk` must not be `hdd`. +In list format, constraints are specified as a list of key-value pairs. The key starts with either a `+` or a `-` with `+disk=ssd` indicating that the label `disk` must be set to `ssd`, and `-disk=hdd`, indicating that the label `disk` must not be `hdd`. -In dictionary format, the constraint also indicates a number of instances that apply to that rule. For example `FOLLOWER_CONSTRAINTS="{+region=us-east-1:1,+region=us-east-2:1,+region=us-west-1:1,+any:1}";` indicates that 1 follower is in us-east-1, 1 follower is in us-east-2, 1 follower is in us-west-1, and 1 follower can be in any region. +In dictionary format, constraints also indicate a number of instances that apply to that rule. For example `FOLLOWER_CONSTRAINTS="{+region=us-east-1:1,+region=us-east-2:1,+region=us-west-1:1,+any:1}";` indicates that 1 follower is in us-east-1, 1 follower is in us-east-2, 1 follower is in us-west-1, and 1 follower can be in any region. ## Known limitations From 6b8d359ab9c57626ef7933a81166ce1a5eb4a7c9 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:03:25 -0600 Subject: [PATCH 25/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index e4af9783dc0c0..ed1db7d057cdb 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -146,8 +146,8 @@ In dictionary format, constraints also indicate a number of instances that apply The following known limitations exist in the experimental release of Placement Rules in SQL: -* Dumpling does not support dumping placement policies. See [issue #29371](https://github.com/pingcap/tidb/issues/29371) -* TiDB tools including BR, CDC, Lightning and DM do not yet support placement rules. +* Dumpling does not support dumping placement policies. See [issue #29371](https://github.com/pingcap/tidb/issues/29371). +* TiDB tools, including Backup & Restore (BR), TiCDC, TiDB Lightning, and TiDB Data Migration (DM), do not yet support placement rules. * Temporary tables do not support placement options (either via direct placement or placement policies). * Syntactic sugar rules exist for setting `PRIMARY_REGION` and `REGIONS`, but in future we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE` and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030) * TiFlash learners are not configurable through Placement Rules syntax. From f43fdf03c747299602e44895e571d624af028019 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:03:44 -0600 Subject: [PATCH 26/58] Update sql-statements/sql-statement-drop-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-drop-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 0b6b9a7569f92..12d45da25f3b2 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -37,8 +37,8 @@ DROP PLACEMENT POLICY p1; -- fails SELECT table_schema, table_name FROM information_schema.tables WHERE tidb_placement_policy_name='p1'; SELECT table_schema, table_name FROM information_schema.partitions WHERE tidb_placement_policy_name='p1'; -ALTER TABLE t1 PLACEMENT POLICY=default; -- Remove the placement policy from t1 -DROP PLACEMENT POLICY p1; -- succeeds +ALTER TABLE t1 PLACEMENT POLICY=default; -- Removes the placement policy from t1. +DROP PLACEMENT POLICY p1; -- Succeeds. ``` ```sql From 3b8bb24030397511c5bec5ddf622fdd09d903bf0 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:05:20 -0600 Subject: [PATCH 27/58] Update sql-statements/sql-statement-alter-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-alter-placement-policy.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-alter-placement-policy.md b/sql-statements/sql-statement-alter-placement-policy.md index cd9b134b84ec4..b15be14f8c7d0 100644 --- a/sql-statements/sql-statement-alter-placement-policy.md +++ b/sql-statements/sql-statement-alter-placement-policy.md @@ -7,8 +7,9 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. +> +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `ALTER PLACEMENT POLICY` is used to modify existing placement policies that have previously been created. All the tables and partitions which use the placement policy will automatically be updated. From 0dcc1bbd1c6ca8070feb656cad455eaddb28ce2e Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:05:34 -0600 Subject: [PATCH 28/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index ed1db7d057cdb..00f2d35b60d6d 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -47,7 +47,7 @@ A `PLACEMENT POLICY` is not associated with any database schema and has global s | Option Name | Description | |----------------------------|------------------------------------------------------------------------------------------------| -| `PRIMARY_REGION` | Raft leaders will be placed in stores which have the `region` label that match this value. | +| `PRIMARY_REGION` | Raft leaders are placed in stores which have the `region` label that matches this value. | | `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | | `SCHEDULE` | The strategy used to schedule the placement of followers. Either `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | | `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 copies of the data (2 followers and 1 leader). | From d5e3686926d0658f8930c39ff81f7e4d4d15dc6d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:05:52 -0600 Subject: [PATCH 29/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 00f2d35b60d6d..e526c9bced38a 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -31,7 +31,7 @@ A `PLACEMENT POLICY` is not associated with any database schema and has global s > **Note:** > -> Placement options depend on labels correctly specified in the configuration of each TiKV node. For example, `PRIMARY_REGION` depends on the label `region` in TiKV. To see a summary of all labels available in your TiKV cluster, use the statement [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md): +> Placement options depend on labels correctly specified in the configuration of each TiKV node. For example, the `PRIMARY_REGION` option depends on the `region` label in TiKV. To see a summary of all labels available in your TiKV cluster, use the statement [`SHOW PLACEMENT LABELS`](/sql-statements/sql-statement-show-placement-labels.md): > > ```sql > mysql> show placement labels; From 2a2a0333bae2dd8cf3bd739ecdeb46827a166713 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:06:20 -0600 Subject: [PATCH 30/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index e526c9bced38a..465fefee1dcc7 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -25,7 +25,7 @@ CREATE TABLE t2 (a INT) PLACEMENT POLICY=eastandwest; Using placement policies is recommended to simplify rule management. Changes made to a placement policy (via [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md)) automatically propagate to all database objects. This differs from direct placement options, which the rules must be altered for each object (tables, partitions). -A `PLACEMENT POLICY` is not associated with any database schema and has global scope. Assigning a placement policy does not require any additional privileges over `CREATE TABLE` privilege. +`PLACEMENT POLICY` is not associated with any database schema and has the global scope. Therefore, assigning a placement policy does not require any additional privileges over the `CREATE TABLE` privilege. ## Option reference From 769c8a8324d8f0155a9773fb1501c3d6a4225db5 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:06:36 -0600 Subject: [PATCH 31/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 465fefee1dcc7..c74168c5516f4 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -23,7 +23,9 @@ CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east- CREATE TABLE t2 (a INT) PLACEMENT POLICY=eastandwest; ``` -Using placement policies is recommended to simplify rule management. Changes made to a placement policy (via [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md)) automatically propagate to all database objects. This differs from direct placement options, which the rules must be altered for each object (tables, partitions). +It is recommended to use placement policies for simpler rule management. When you change a placement policy (via [`ALTER PLACEMENT POLICY`](/sql-statements/sql-statement-alter-placement-policy.md)), the change automatically propagates to all database objects. + +If you use direct placement options, you have to alter rules for each object (for example, tables and partitions). `PLACEMENT POLICY` is not associated with any database schema and has the global scope. Therefore, assigning a placement policy does not require any additional privileges over the `CREATE TABLE` privilege. From 2f38813e4ff625d182a66719dc5f0058f4982e0a Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:06:50 -0600 Subject: [PATCH 32/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index c74168c5516f4..1531695750b30 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -15,7 +15,9 @@ Placement Rules allow you to configure where data will be stored in a TiKV clust ## Specifying placement options -Placement options can be specified using either _direct placement_ or using a _placement policy_. In the following example, both tables `t1` and `t2` have the same rules: +To use Placement Rules in SQL, you need to specify one or more placement options in a SQL statement. To specify the Placement options, you can either use _direct placement_ or use a _placement policy_. + +In the following example, both tables `t1` and `t2` have the same rules. `t1` is specified rules using a direct placement while `t2` is specified rules using a placement policy. ```sql CREATE TABLE t1 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-west-1"; From bba9404084b96208ced0061f97788dd3eb01cc60 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:15:04 -0600 Subject: [PATCH 33/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 1531695750b30..5858d29f8874b 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -155,4 +155,4 @@ The following known limitations exist in the experimental release of Placement R * Temporary tables do not support placement options (either via direct placement or placement policies). * Syntactic sugar rules exist for setting `PRIMARY_REGION` and `REGIONS`, but in future we plan to add varieties for `PRIMARY_RACK`, `PRIMARY_ZONE` and `PRIMARY_HOST`. See [issue #18030](https://github.com/pingcap/tidb/issues/18030) * TiFlash learners are not configurable through Placement Rules syntax. -* Placement rules only ensure that data at rest resides on the correct TiKV store. It does not guarantee that data in transit (via either user-queries or internal operations) will only occur in a specific region. We plan to offer additional features in future to better support compliance use-cases. +* Placement rules only ensure that data at rest resides on the correct TiKV store. The rules do not guarantee that data in transit (via either user-queries or internal operations) only occurs in a specific region. From 3c5087a17aa8bee246615d1526826ae36f9617e2 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:15:19 -0600 Subject: [PATCH 34/58] Update sql-statements/sql-statement-drop-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-drop-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 12d45da25f3b2..df688f8b943f1 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -31,9 +31,9 @@ Placement policies can only be dropped when they are not referenced by any table ```sql CREATE PLACEMENT POLICY p1 FOLLOWERS=4; CREATE TABLE t1 (a INT PRIMARY KEY) PLACEMENT POLICY=p1; -DROP PLACEMENT POLICY p1; -- fails +DROP PLACEMENT POLICY p1; -- This statement fails because the placement policy p1 is referenced. --- Find which tables and partitions reference the placement policy +-- Finds which tables and partitions reference the placement policy. SELECT table_schema, table_name FROM information_schema.tables WHERE tidb_placement_policy_name='p1'; SELECT table_schema, table_name FROM information_schema.partitions WHERE tidb_placement_policy_name='p1'; From 53f6d9ff583128816aa70a3a22412a6c38cf2f7e Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:15:51 -0600 Subject: [PATCH 35/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 5858d29f8874b..ab4d474d01956 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -53,7 +53,7 @@ If you use direct placement options, you have to alter rules for each object (fo |----------------------------|------------------------------------------------------------------------------------------------| | `PRIMARY_REGION` | Raft leaders are placed in stores which have the `region` label that matches this value. | | `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | -| `SCHEDULE` | The strategy used to schedule the placement of followers. Either `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | +| `SCHEDULE` | The strategy used to schedule the placement of followers. The value options are `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | | `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 copies of the data (2 followers and 1 leader). | | **Advanced configuration** | | `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | From bbc9312cb117f11e792bc00d2822a262789c9e50 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:16:46 -0600 Subject: [PATCH 36/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index ab4d474d01956..251849197bae6 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -54,7 +54,7 @@ If you use direct placement options, you have to alter rules for each object (fo | `PRIMARY_REGION` | Raft leaders are placed in stores which have the `region` label that matches this value. | | `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | | `SCHEDULE` | The strategy used to schedule the placement of followers. The value options are `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | -| `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 copies of the data (2 followers and 1 leader). | +| `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 replicas of the data (2 followers and 1 leader). | | **Advanced configuration** | | `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | | `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | From 63bd2634099ceba3078740305e86899cd613302a Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:17:03 -0600 Subject: [PATCH 37/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 251849197bae6..4bf4d969f6114 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -55,7 +55,10 @@ If you use direct placement options, you have to alter rules for each object (fo | `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | | `SCHEDULE` | The strategy used to schedule the placement of followers. The value options are `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | | `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 replicas of the data (2 followers and 1 leader). | -| **Advanced configuration** | +In addition to the placement options above, you can also use the advance configurations. For details, see [Advance placement](#advanced-placement). + +| Option Name | Description | +|----------------------------|------------------------------------------------------------------------------------------------| | `CONSTRAINTS` | A list of constraints that apply to all roles. For example, `CONSTRAINTS="[+disk=ssd]`. | | `FOLLOWER_CONSTRAINTS` | A list of constraints that only apply to followers. | From 7e8764d84e47b7f265fb84110da55678ae488b2d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:17:17 -0600 Subject: [PATCH 38/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 4bf4d969f6114..cb0726b2c5183 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -13,7 +13,7 @@ summary: Learn how to schedule placement of tables and partitions. Placement Rules allow you to configure where data will be stored in a TiKV cluster. This is useful for scenarios including optimizing a high availability strategy, ensuring that local copies of data will be available for local stale reads, and adhering to compliance requirements. -## Specifying placement options +## Specify placement options To use Placement Rules in SQL, you need to specify one or more placement options in a SQL statement. To specify the Placement options, you can either use _direct placement_ or use a _placement policy_. From 0232ea668dad239c92c22cae4e86d98c6fa22e00 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:17:34 -0600 Subject: [PATCH 39/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index cb0726b2c5183..c544f842064f4 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -64,7 +64,7 @@ In addition to the placement options above, you can also use the advance configu ## Examples -### Increasing the number of replicas +### Increase the number of replicas The default configuration of [`max-replicas`](/pd-configuration-file.md#max-replicas) is `3`. To increase this for a specific set of tables, you can use a placement policy as follows: From 973ff237df9f0fedc2a7afac44ce068b1f4fcc18 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:17:44 -0600 Subject: [PATCH 40/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index c544f842064f4..ffb6c7fd31ba6 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -73,7 +73,7 @@ CREATE PLACEMENT POLICY fivereplicas FOLLOWERS=4; CREATE TABLE t1 (a INT) PLACEMENT POLICY=fivereplicas; ``` -Note that the PD configuration includes the leader and follower count, thus 4 followers + 1 leader equals five replicas in total. +Note that the PD configuration includes the leader and follower count, thus 4 followers + 1 leader equals 5 replicas in total. To expand on this example, the placement for the followers can also be described using the `PRIMARY_REGION` and `REGIONS` placement options: From 1311b9780861376845f5e45cb07a5372732c30bb Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:17:59 -0600 Subject: [PATCH 41/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index ffb6c7fd31ba6..5eddc62742caa 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -75,7 +75,7 @@ CREATE TABLE t1 (a INT) PLACEMENT POLICY=fivereplicas; Note that the PD configuration includes the leader and follower count, thus 4 followers + 1 leader equals 5 replicas in total. -To expand on this example, the placement for the followers can also be described using the `PRIMARY_REGION` and `REGIONS` placement options: +To expand on this example, you can also use `PRIMARY_REGION` and `REGIONS` placement options to describe the placement for the followers: ```sql CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2,us-west-1" SCHEDULE="MAJORITY_IN_PRIMARY" FOLLOWERS=4; From 761bc80e895e66c54514ee23be45125146d4e161 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:18:21 -0600 Subject: [PATCH 42/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 5eddc62742caa..75f88e5bbeb8b 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -82,7 +82,9 @@ CREATE PLACEMENT POLICY eastandwest PRIMARY_REGION="us-east-1" REGIONS="us-east- CREATE TABLE t1 (a INT) PLACEMENT POLICY=eastandwest; ``` -The `SCHEDULE` instructs TiDB how to balance the followers. The default schedule of `EVEN` ensures a balance of followers in all regions. The schedule `MAJORITY_IN_PRIMARY` can be used to ensure that enough followers are placed in the primary region (`us-east-1`) that quorum can be achieved. This helps provide lower latency transactions at the expense of availability should the primary region completely fail. +The `SCHEDULE` option instructs TiDB on how to balance the followers. + +The default schedule of `EVEN` ensures a balance of followers in all regions. You can use the `MAJORITY_IN_PRIMARY` schedule to ensure that enough followers are placed in the primary region (`us-east-1`) so that quorum can be achieved. If the primary region completely fails, you can the `MAJORITY_IN_PRIMARY` schedule for lower latency transactions at the expense of availability. ### Assigning placement to a partitioned table From 77be485d8164964cc3e68ae1dc9f4b18a16a6890 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:18:36 -0600 Subject: [PATCH 43/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 75f88e5bbeb8b..5f257d21cf850 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -86,7 +86,7 @@ The `SCHEDULE` option instructs TiDB on how to balance the followers. The default schedule of `EVEN` ensures a balance of followers in all regions. You can use the `MAJORITY_IN_PRIMARY` schedule to ensure that enough followers are placed in the primary region (`us-east-1`) so that quorum can be achieved. If the primary region completely fails, you can the `MAJORITY_IN_PRIMARY` schedule for lower latency transactions at the expense of availability. -### Assigning placement to a partitioned table +### Assign placement to a partitioned table > **Note:** > From d7dcda08f2bbf2065dc0905d9bd6c06d1801a0c6 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:18:50 -0600 Subject: [PATCH 44/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 5f257d21cf850..d3900f8717682 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -90,7 +90,7 @@ The default schedule of `EVEN` ensures a balance of followers in all regions. Yo > **Note:** > -> This example makes use of list partitioning, which is currently an experimental feature. Partitioned tables also require the `PRIMARY KEY` to be included in all columns in the table's partitioning function. +> The following example uses list partitioning, which is currently an experimental feature of TiDB. Partitioned tables also require the `PRIMARY KEY` to be included in all columns in the table's partitioning function. As well as assigning to tables, placement options can also be assigned to table partitions. For example: From 86f837d7fc949594edd3814eba1479a691489b86 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:19:03 -0600 Subject: [PATCH 45/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index d3900f8717682..19f7c13392b5c 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -92,7 +92,7 @@ The default schedule of `EVEN` ensures a balance of followers in all regions. Yo > > The following example uses list partitioning, which is currently an experimental feature of TiDB. Partitioned tables also require the `PRIMARY KEY` to be included in all columns in the table's partitioning function. -As well as assigning to tables, placement options can also be assigned to table partitions. For example: +In addition to assigning placement options to tables, you can also assign the options to table partitions. For example: ```sql CREATE PLACEMENT POLICY europe PRIMARY_REGION="eu-central-1" REGIONS="eu-central-1,eu-west-1"; From 66f359366951c60ffeb94d1eda1e613ded1d6445 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:19:35 -0600 Subject: [PATCH 46/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 19f7c13392b5c..e4b4090fb75c2 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -108,7 +108,7 @@ CREATE TABLE t1 ( ); ``` -### Setting the default placement for a schema +### Set the default placement for a schema Default placement options can be directly attached to a database schema. This works similar to setting the default character set or collation for a schema, in that it will be used when no other placement options are specified. For example: From 13e76a5b1fa3cf12dc45acb6b515207b27f326c5 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:19:53 -0600 Subject: [PATCH 47/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index e4b4090fb75c2..04712dcf070c6 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -110,7 +110,7 @@ CREATE TABLE t1 ( ### Set the default placement for a schema -Default placement options can be directly attached to a database schema. This works similar to setting the default character set or collation for a schema, in that it will be used when no other placement options are specified. For example: +You can directly attach the default placement options to a database schema. This works similar to setting the default character set or collation for a schema. Your specified placement options apply when no other options are specified. For example: ```sql CREATE TABLE t1 (a INT); -- the table is created with no placement options From 3e9ea397d1ef096b0a582d25942b3c2d2e6079d6 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:20:29 -0600 Subject: [PATCH 48/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 04712dcf070c6..2491897b5ad20 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -113,12 +113,17 @@ CREATE TABLE t1 ( You can directly attach the default placement options to a database schema. This works similar to setting the default character set or collation for a schema. Your specified placement options apply when no other options are specified. For example: ```sql -CREATE TABLE t1 (a INT); -- the table is created with no placement options -ALTER DATABASE test FOLLOWERS=4; -- this changes the default, and does not apply to the existing table t1; -CREATE TABLE t2 (a INT); -- the placement of FOLLOWERS=4 will be used -CREATE TABLE t3 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2"; -- FOLLOWERS=4 does not apply as placement is specified. -ALTER DATABASE test FOLLOWERS=2; -- this does not apply to existing tables -CREATE TABLE t4 (a INT); -- the table is created with FOLLOWERS=2; +CREATE TABLE t1 (a INT); -- Creates a table t1 with no placement options. + +ALTER DATABASE test FOLLOWERS=4; -- Changes the default placement option, and does not apply to the existing table t1. + +CREATE TABLE t2 (a INT); -- Creates a table t2 with the default placement of FOLLOWERS=4. + +CREATE TABLE t3 (a INT) PRIMARY_REGION="us-east-1" REGIONS="us-east-1,us-east-2"; -- Creates a table t3 without the default FOLLOWERS=4 placement, because this statement has specified another placement. + +ALTER DATABASE test FOLLOWERS=2; -- Changes the default placement, and does not apply to existing tables. + +CREATE TABLE t4 (a INT); -- Creates a table t4 with the default FOLLOWERS=2 option. ``` Because placement options are only inherited from the database schema default when a table is created, it is recommended to set the default to a `PLACEMENT POLICY`. This ensures that future changes to the policy will propagate to existing tables. From 05ff6475bf18d2f71a4675a44ba81c6603e74844 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:21:14 -0600 Subject: [PATCH 49/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index 2491897b5ad20..d6b745adeb70a 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -126,7 +126,7 @@ ALTER DATABASE test FOLLOWERS=2; -- Changes the default placement, and does not CREATE TABLE t4 (a INT); -- Creates a table t4 with the default FOLLOWERS=2 option. ``` -Because placement options are only inherited from the database schema default when a table is created, it is recommended to set the default to a `PLACEMENT POLICY`. This ensures that future changes to the policy will propagate to existing tables. +Because placement options are only inherited from the database schema when a table is created, it is recommended to set the default placement option using a `PLACEMENT POLICY`. This ensures that future changes to the policy propagate to existing tables. ### Advanced placement From 3a6a7074c7d4b0ac7ebaf8e1ed3e04214e2c3124 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Fri, 5 Nov 2021 09:22:05 -0600 Subject: [PATCH 50/58] Adjust warning --- sql-statements/sql-statement-create-placement-policy.md | 3 ++- sql-statements/sql-statement-drop-placement-policy.md | 3 ++- sql-statements/sql-statement-show-create-placement-policy.md | 3 ++- sql-statements/sql-statement-show-placement-for.md | 3 ++- sql-statements/sql-statement-show-placement.md | 3 ++- 5 files changed, 10 insertions(+), 5 deletions(-) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 417cccce12527..3b1768a7fed29 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -8,6 +8,7 @@ summary: The usage of CREATE PLACEMENT POLICY in TiDB. > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. `CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions, or database schemas. @@ -78,4 +79,4 @@ This statement is a TiDB extension to MySQL syntax. * [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) -* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 12d45da25f3b2..197441eb31348 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -8,6 +8,7 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. `DROP PLACEMENT POLICY` is used to drop a previously created placement policy. @@ -71,4 +72,4 @@ This statement is a TiDB extension to MySQL syntax. * [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) -* [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) \ No newline at end of file +* [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index b417285c38082..aa948c1afc4ea 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -8,6 +8,7 @@ summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. `SHOW CREATE PLACEMENT POLICY` is used to show the definition of a placement policy. This can be used to see the current definition of a placement policy and recreate it in another TiDB cluster. @@ -52,4 +53,4 @@ This statement is a TiDB extension to MySQL syntax. * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) * [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) * [ALTER PLACEMENT POLICY](/sql-statements/sql-statement-alter-placement-policy.md) -* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) \ No newline at end of file +* [DROP PLACEMENT POLICY](/sql-statements/sql-statement-drop-placement-policy.md) diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index 0ae8d2c7171c2..21bb9db1f5e68 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -8,6 +8,7 @@ summary: The usage of SHOW PLACEMENT FOR in TiDB. > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. `SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in the canonical form for a specific table, database schema, or partition. @@ -105,4 +106,4 @@ This statement is a TiDB extension to MySQL syntax. * [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT](/sql-statements/sql-statement-show-placement.md) -* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md index a13eb882eec3d..fe434e6fc42ec 100644 --- a/sql-statements/sql-statement-show-placement.md +++ b/sql-statements/sql-statement-show-placement.md @@ -8,6 +8,7 @@ summary: The usage of SHOW PLACEMENT in TiDB. > **Warning:** > > Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> > If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. `SHOW PLACEMENT` summarizes all placement options from direct placement and placement policies, and presents them in canonical form. @@ -56,4 +57,4 @@ This statement is a TiDB extension to MySQL syntax. * [Placement Rules in SQL](/placement-rules-in-sql.md) * [SHOW PLACEMENT FOR](/sql-statements/sql-statement-show-placement-for.md) -* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) \ No newline at end of file +* [CREATE PLACEMENT POLICY](/sql-statements/sql-statement-create-placement-policy.md) From d11405ba8e10e58c5d319f6f19156fb12346770d Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 19:26:28 -0700 Subject: [PATCH 51/58] Update system-variables.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index a0071e5b47e26..e4ebb4144e398 100644 --- a/system-variables.md +++ b/system-variables.md @@ -286,7 +286,7 @@ This variable is an alias for `last_insert_id`. - Scope: SESSION | GLOBAL - Default value: ON -- This variable controls whether DDL validates that [Placement Rules in SQL](/placement-rules-in-sql.md) are valid when creating tables. +- This variable controls whether DDL statements validate [Placement Rules in SQL](/placement-rules-in-sql.md). - It is intended to be used by logical dump/restore tools to ensure that tables can always be created even if placement rules are violated. This is similar to how mysqldump writes `SET FOREIGN_KEY_CHECKS=0;` to the start of every dump file. ### plugin_dir From 3769ea7f7ba147eb9c83be25bea7b8ab2f458a29 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:23 -0700 Subject: [PATCH 52/58] Update sql-statements/sql-statement-show-placement.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-show-placement.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-show-placement.md b/sql-statements/sql-statement-show-placement.md index fe434e6fc42ec..e14fd7a35035f 100644 --- a/sql-statements/sql-statement-show-placement.md +++ b/sql-statements/sql-statement-show-placement.md @@ -7,9 +7,9 @@ summary: The usage of SHOW PLACEMENT in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `SHOW PLACEMENT` summarizes all placement options from direct placement and placement policies, and presents them in canonical form. From d673d8db272562bf8ad438da32962ae6ac266de8 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:28 -0700 Subject: [PATCH 53/58] Update sql-statements/sql-statement-show-placement-labels.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-show-placement-labels.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-show-placement-labels.md b/sql-statements/sql-statement-show-placement-labels.md index 37c4b220ff9ff..0335fb54e9f51 100644 --- a/sql-statements/sql-statement-show-placement-labels.md +++ b/sql-statements/sql-statement-show-placement-labels.md @@ -7,8 +7,9 @@ summary: The usage of SHOW PLACEMENT LABELS in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. +> +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `SHOW PLACEMENT LABELS` is used to summarize the labels and values that are available for Placement Rules. From bbf4d006413a38366dd630011b50476752400eb1 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:33 -0700 Subject: [PATCH 54/58] Update sql-statements/sql-statement-show-placement-for.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-show-placement-for.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-show-placement-for.md b/sql-statements/sql-statement-show-placement-for.md index 21bb9db1f5e68..021bbc01b5b9d 100644 --- a/sql-statements/sql-statement-show-placement-for.md +++ b/sql-statements/sql-statement-show-placement-for.md @@ -7,9 +7,9 @@ summary: The usage of SHOW PLACEMENT FOR in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `SHOW PLACEMENT FOR` summarizes all placement options from direct placement and placement policies, and presents them in the canonical form for a specific table, database schema, or partition. From 429d1939f4746f0080b69c9f9029f191d6a52363 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:37 -0700 Subject: [PATCH 55/58] Update sql-statements/sql-statement-show-create-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-show-create-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-show-create-placement-policy.md b/sql-statements/sql-statement-show-create-placement-policy.md index aa948c1afc4ea..647f80d0f310b 100644 --- a/sql-statements/sql-statement-show-create-placement-policy.md +++ b/sql-statements/sql-statement-show-create-placement-policy.md @@ -7,9 +7,9 @@ summary: The usage of SHOW CREATE PLACEMENT POLICY in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `SHOW CREATE PLACEMENT POLICY` is used to show the definition of a placement policy. This can be used to see the current definition of a placement policy and recreate it in another TiDB cluster. From d6421be8a56f81d6153df3359ed7fba7315d9c6f Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:53 -0700 Subject: [PATCH 56/58] Update placement-rules-in-sql.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- placement-rules-in-sql.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/placement-rules-in-sql.md b/placement-rules-in-sql.md index d6b745adeb70a..3cbee7d70ecc6 100644 --- a/placement-rules-in-sql.md +++ b/placement-rules-in-sql.md @@ -51,10 +51,11 @@ If you use direct placement options, you have to alter rules for each object (fo | Option Name | Description | |----------------------------|------------------------------------------------------------------------------------------------| -| `PRIMARY_REGION` | Raft leaders are placed in stores which have the `region` label that matches this value. | -| `REGIONS` | Raft followers will be placed in stores which have the `region` label that matches this value. | +| `PRIMARY_REGION` | Raft leaders are placed in stores that have the `region` label that matches the value of this option. | +| `REGIONS` | Raft followers are placed in stores that have the `region` label that matches the value of this option. | | `SCHEDULE` | The strategy used to schedule the placement of followers. The value options are `EVEN` (default) or `MAJORITY_IN_PRIMARY`. | | `FOLLOWERS` | The number of followers. For example, `FOLLOWERS=2` means that there will be 3 replicas of the data (2 followers and 1 leader). | + In addition to the placement options above, you can also use the advance configurations. For details, see [Advance placement](#advanced-placement). | Option Name | Description | From 051b37248c10adac2100b599e422228fa05a3f60 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:14:59 -0700 Subject: [PATCH 57/58] Update sql-statements/sql-statement-drop-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-drop-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-drop-placement-policy.md b/sql-statements/sql-statement-drop-placement-policy.md index 641ab4b9588d0..12b5296b8b4c8 100644 --- a/sql-statements/sql-statement-drop-placement-policy.md +++ b/sql-statements/sql-statement-drop-placement-policy.md @@ -7,9 +7,9 @@ summary: The usage of ALTER PLACEMENT POLICY in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `DROP PLACEMENT POLICY` is used to drop a previously created placement policy. From 5422310f9efaaced9b0ca3907b7047d8e6ea1a50 Mon Sep 17 00:00:00 2001 From: Morgan Tocker Date: Sun, 7 Nov 2021 21:15:05 -0700 Subject: [PATCH 58/58] Update sql-statements/sql-statement-create-placement-policy.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- sql-statements/sql-statement-create-placement-policy.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sql-statements/sql-statement-create-placement-policy.md b/sql-statements/sql-statement-create-placement-policy.md index 3b1768a7fed29..d0f2bd006c723 100644 --- a/sql-statements/sql-statement-create-placement-policy.md +++ b/sql-statements/sql-statement-create-placement-policy.md @@ -7,9 +7,9 @@ summary: The usage of CREATE PLACEMENT POLICY in TiDB. > **Warning:** > -> Placement Rules in SQL is an experimental feature. Syntax may change before its final release, and there may be bugs. +> Placement Rules in SQL is an experimental feature. The syntax might change before its GA, and there might also be bugs. > -> If you understand the risks, run `SET GLOBAL tidb_enable_alter_placement = 1;` to enable this experiment. +> If you understand the risks, you can enable this experiment feature by executing `SET GLOBAL tidb_enable_alter_placement = 1;`. `CREATE PLACEMENT POLICY` is used to create a named placement policy that can later be assigned to tables, partitions, or database schemas.