From 175c6cf749ada669066f3eb42751f3a0d0d2d63c Mon Sep 17 00:00:00 2001 From: yikeke Date: Fri, 10 Jul 2020 23:03:14 +0800 Subject: [PATCH 1/7] align https://github.com/pingcap/docs-cn/pull/3770/ --- TOC.md | 1 + .../high-concurrency-best-practices.md | 2 +- faq/tidb-faq.md | 2 +- shard-row-id-bits.md | 23 +++++++++++++++++++ sql-statements/sql-statement-create-table.md | 2 +- 5 files changed, 27 insertions(+), 3 deletions(-) create mode 100644 shard-row-id-bits.md diff --git a/TOC.md b/TOC.md index ad1f7d71200b6..685b060bcbe0c 100644 --- a/TOC.md +++ b/TOC.md @@ -203,6 +203,7 @@ + Attributes + [AUTO_INCREMENT](/auto-increment.md) + [AUTO_RANDOM](/auto-random.md) + + [SHARD_ROW_ID_BITS](/shard-row-id-bits.md) + [Literal Values](/literal-values.md) + [Schema Object Names](/schema-object-names.md) + [Keywords and Reserved Words](/keywords.md) diff --git a/best-practices/high-concurrency-best-practices.md b/best-practices/high-concurrency-best-practices.md index be0dca2d4aaab..5687acdbb3fb7 100644 --- a/best-practices/high-concurrency-best-practices.md +++ b/best-practices/high-concurrency-best-practices.md @@ -181,7 +181,7 @@ In this case, the table is simple. In other cases, you might also need to consid **Problem one:** -If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS` description](/tidb-specific-system-variables.md#shard_row_id_bits) for more details. +If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS` description](/shard-row-id-bits.md) for more details. To avoid the hotspot problem in this situation, you can use `SHARD_ROW_ID_BITS` and `PRE_SPLIT_REGIONS` when creating a table. For more details about `PRE_SPLIT_REGIONS`, refer to [Pre-split Regions](/sql-statements/sql-statement-split-region.md#pre_split_regions). diff --git a/faq/tidb-faq.md b/faq/tidb-faq.md index 03cc0a6bf2dee..3d69c4f8d4e9c 100644 --- a/faq/tidb-faq.md +++ b/faq/tidb-faq.md @@ -1089,7 +1089,7 @@ See [The TiDB Command Options](/command-line-flags-for-tidb-configuration.md). #### How to scatter the hotspots? -In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of `SHARD_ROW_ID_BITS` in [TiDB Specific System Variables and Syntax](/tidb-specific-system-variables.md#shard_row_id_bits). +In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of `SHARD_ROW_ID_BITS` in [TiDB Specific System Variables and Syntax](/shard-row-id-bits.md). ### TiKV diff --git a/shard-row-id-bits.md b/shard-row-id-bits.md new file mode 100644 index 0000000000000..e5ec90ea1de65 --- /dev/null +++ b/shard-row-id-bits.md @@ -0,0 +1,23 @@ +--- +title: SHARD_ROW_ID_BITS +summary: Learn the SHARD_ROW_ID_BITS attribute. +--- + +# SHARD_ROW_ID_BITS + +This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the implicit sharded `_tidb_rowid`. + +## Concept + +For the tables with a non-integer primary key or no primary key, TiDB uses an implicit auto-increment row ID. When a large number of `INSERT` operations are performed, the data is written into a single Region, causing a write hot spot. + +To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The row IDs are scattered and the data are written into multiple different Regions. But setting an overlarge value might lead to an excessively large number of RPC requests, which increases the CPU and network overheads. + +- `SHARD_ROW_ID_BITS = 4` indicates 16 shards +- `SHARD_ROW_ID_BITS = 6` indicates 64 shards +- `SHARD_ROW_ID_BITS = 0` indicates the default 1 shard + +## Examples + +- `CREATE TABLE`: `CREATE TABLE t (c int) SHARD_ROW_ID_BITS = 4;` +- `ALTER TABLE`: `ALTER TABLE t SHARD_ROW_ID_BITS = 4;` diff --git a/sql-statements/sql-statement-create-table.md b/sql-statements/sql-statement-create-table.md index 0f4996704a67c..ba4571db1d5fc 100644 --- a/sql-statements/sql-statement-create-table.md +++ b/sql-statements/sql-statement-create-table.md @@ -225,7 +225,7 @@ table_option: | STATS_PERSISTENT [=] {DEFAULT|0|1} ``` -The `table_option` currently only supports `AUTO_INCREMENT`, `SHARD_ROW_ID_BITS` (see [TiDB Specific System Variables](/tidb-specific-system-variables.md#shard_row_id_bits) for details), `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details: +The `table_option` currently only supports `AUTO_INCREMENT`, `SHARD_ROW_ID_BITS` (see [TiDB Specific System Variables](/shard-row-id-bits.md) for details), `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details: | Parameters | Description | Example | | ---------- | ---------- | ------- | From 6c935778b70d570ce26e212c56778213015e6ce3 Mon Sep 17 00:00:00 2001 From: yikeke Date: Fri, 10 Jul 2020 23:04:21 +0800 Subject: [PATCH 2/7] remove SHARD_ROW_ID_BITS from tidb-specific-system-variables.md --- tidb-specific-system-variables.md | 15 --------------- 1 file changed, 15 deletions(-) diff --git a/tidb-specific-system-variables.md b/tidb-specific-system-variables.md index a17245841d3c6..362218a3dd71b 100644 --- a/tidb-specific-system-variables.md +++ b/tidb-specific-system-variables.md @@ -365,21 +365,6 @@ mysql> desc select count(distinct a) from test.t; - Default value: 0 - This variable is used to set whether to allow `insert`, `replace` and `update` statements to operate on the column `_tidb_rowid`. It is not allowed by default. This variable can be used only when importing data with TiDB tools. -## SHARD_ROW_ID_BITS - -For the tables with non-integer PK or without PK, TiDB uses an implicit auto-increment ROW ID. When a large number of `INSERT` operations occur, the data is written into a single Region, causing a write hot spot. - -To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The ROW ID is scattered and the data is written into multiple different Regions. But setting an overlarge value might lead to an excessively large number of RPC requests, which increases the CPU and network overheads. - -- `SHARD_ROW_ID_BITS = 4` indicates 16 shards -- `SHARD_ROW_ID_BITS = 6` indicates 64 shards -- `SHARD_ROW_ID_BITS = 0` indicates the default 1 shard - -Usage of statements: - -- `CREATE TABLE`: `CREATE TABLE t (c int) SHARD_ROW_ID_BITS = 4;` -- `ALTER TABLE`: `ALTER TABLE t SHARD_ROW_ID_BITS = 4;` - ### tidb_row_format_version - Scope: GLOBAL From 00e32476e17db7b9c31f6d1f4034c1380b66a701 Mon Sep 17 00:00:00 2001 From: Keke Yi <40977455+yikeke@users.noreply.github.com> Date: Mon, 13 Jul 2020 19:06:12 +0800 Subject: [PATCH 3/7] Update shard-row-id-bits.md --- shard-row-id-bits.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/shard-row-id-bits.md b/shard-row-id-bits.md index e5ec90ea1de65..1aba21c2417b3 100644 --- a/shard-row-id-bits.md +++ b/shard-row-id-bits.md @@ -5,7 +5,7 @@ summary: Learn the SHARD_ROW_ID_BITS attribute. # SHARD_ROW_ID_BITS -This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the implicit sharded `_tidb_rowid`. +This document introduces the `SHARD_ROW_ID_BITS` table attribute, which is used to set the number of bits of the shards after the implicit `_tidb_rowid` is sharded. ## Concept From e7491e14e4de15ec569ddb87bb4af810ac20716b Mon Sep 17 00:00:00 2001 From: yikeke Date: Tue, 14 Jul 2020 10:32:05 +0800 Subject: [PATCH 4/7] remove SHARD_ROW_ID_BITS from alter table doc --- sql-statements/sql-statement-alter-table.md | 15 --------------- sql-statements/sql-statement-create-table.md | 2 +- 2 files changed, 1 insertion(+), 16 deletions(-) diff --git a/sql-statements/sql-statement-alter-table.md b/sql-statements/sql-statement-alter-table.md index 256163e4b86f4..736ad452ccba9 100644 --- a/sql-statements/sql-statement-alter-table.md +++ b/sql-statements/sql-statement-alter-table.md @@ -59,21 +59,6 @@ mysql> EXPLAIN SELECT * FROM t1 WHERE c1 = 3; 2 rows in set (0.00 sec) ``` -## SHARD_ROW_ID_BITS - -For tables with a non-integer `PRIMARY KEY` or without a `PRIMARY KEY`, TiDB uses an implicit auto-increment ROW ID. Because regions are automatically sharded using a range-based scheme on the `PRIMARY KEY`, hotspots can occur when there are a large number of `INSERT` operations. - -To mitigate the hot spot issue, you can configure `SHARD_ROW_ID_BITS`. The ROW ID is scattered and the data is written into multiple different Regions. But setting an overlarge value might lead to an excessively large number of RPC requests, which increases the CPU and network overhead. - -- `SHARD_ROW_ID_BITS = 4` indicates 16 shards -- `SHARD_ROW_ID_BITS = 6` indicates 64 shards -- `SHARD_ROW_ID_BITS = 0` indicates the default 1 shard - -Usage: - -- `CREATE TABLE`: `CREATE TABLE t (c int) SHARD_ROW_ID_BITS = 4;` -- `ALTER TABLE`: `ALTER TABLE t SHARD_ROW_ID_BITS = 4;` - ## MySQL compatibility * All of the data types except spatial types are supported. For other unsupported cases, refer to: [compatibility of DDL statements with MySQL](/mysql-compatibility.md#ddl). diff --git a/sql-statements/sql-statement-create-table.md b/sql-statements/sql-statement-create-table.md index ba4571db1d5fc..80ac8463fd26b 100644 --- a/sql-statements/sql-statement-create-table.md +++ b/sql-statements/sql-statement-create-table.md @@ -225,7 +225,7 @@ table_option: | STATS_PERSISTENT [=] {DEFAULT|0|1} ``` -The `table_option` currently only supports `AUTO_INCREMENT`, `SHARD_ROW_ID_BITS` (see [TiDB Specific System Variables](/shard-row-id-bits.md) for details), `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details: +The `table_option` currently only supports `AUTO_INCREMENT`, [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md), `PRE_SPLIT_REGIONS`, `CHARACTER SET`, `COLLATE`, and `COMMENT`, while the others are only supported in syntax. The clauses are separated by a comma `,`. See the following table for details: | Parameters | Description | Example | | ---------- | ---------- | ------- | From 25e94479a8956e6cf7c835c1179141ddef0ae70e Mon Sep 17 00:00:00 2001 From: Keke Yi <40977455+yikeke@users.noreply.github.com> Date: Tue, 14 Jul 2020 10:33:13 +0800 Subject: [PATCH 5/7] Update best-practices/high-concurrency-best-practices.md --- best-practices/high-concurrency-best-practices.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/best-practices/high-concurrency-best-practices.md b/best-practices/high-concurrency-best-practices.md index 5687acdbb3fb7..1119863398afa 100644 --- a/best-practices/high-concurrency-best-practices.md +++ b/best-practices/high-concurrency-best-practices.md @@ -181,7 +181,7 @@ In this case, the table is simple. In other cases, you might also need to consid **Problem one:** -If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS` description](/shard-row-id-bits.md) for more details. +If a table does not have a primary key, or the primary key is not the `Int` type and you do not want to generate a randomly distributed primary key ID, TiDB provides an implicit `_tidb_rowid` column as the row ID. Generally, when you do not use the `SHARD_ROW_ID_BITS` parameter, the values of the `_tidb_rowid` column are also monotonically increasing, which might causes hotspots too. Refer to [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md) for more details. To avoid the hotspot problem in this situation, you can use `SHARD_ROW_ID_BITS` and `PRE_SPLIT_REGIONS` when creating a table. For more details about `PRE_SPLIT_REGIONS`, refer to [Pre-split Regions](/sql-statements/sql-statement-split-region.md#pre_split_regions). From 7273f7f783a19a11616847b8fe6baf5d074c4669 Mon Sep 17 00:00:00 2001 From: Keke Yi <40977455+yikeke@users.noreply.github.com> Date: Tue, 14 Jul 2020 10:33:51 +0800 Subject: [PATCH 6/7] Update faq/tidb-faq.md --- faq/tidb-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/faq/tidb-faq.md b/faq/tidb-faq.md index f2e10ef393adf..41cfb793ab301 100644 --- a/faq/tidb-faq.md +++ b/faq/tidb-faq.md @@ -1089,7 +1089,7 @@ See [The TiDB Command Options](/command-line-flags-for-tidb-configuration.md). #### How to scatter the hotspots? -In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of `SHARD_ROW_ID_BITS` in [TiDB Specific System Variables and Syntax](/shard-row-id-bits.md). +In TiDB, data is divided into Regions for management. Generally, the TiDB hotspot means the Read/Write hotspot in a Region. In TiDB, for the table whose primary key (PK) is not an integer or which has no PK, you can properly break Regions by configuring `SHARD_ROW_ID_BITS` to scatter the Region hotspots. For details, see the introduction of [`SHARD_ROW_ID_BITS`](/shard-row-id-bits.md). ### TiKV From 1c5608e2117f48f0c2ba81ed7470948ecc9997cb Mon Sep 17 00:00:00 2001 From: yikeke Date: Tue, 14 Jul 2020 10:37:06 +0800 Subject: [PATCH 7/7] add a missing alias --- system-variables.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index 0146628452b07..dceab63f4f55e 100644 --- a/system-variables.md +++ b/system-variables.md @@ -2,7 +2,7 @@ title: System Variables summary: Use system variables to optimize performance or alter running behavior. category: reference -aliases: ['/docs/dev/system-variables/','/docs/dev/reference/configuration/tidb-server/mysql-variables/', '/docs/dev/tidb-specific-system-variables/','/docs/dev/reference/configuration/tidb-server/tidb-specific-variables/'] +aliases: ['/tidb/dev/tidb-specific-system-variables','/docs/dev/system-variables/','/docs/dev/reference/configuration/tidb-server/mysql-variables/', '/docs/dev/tidb-specific-system-variables/','/docs/dev/reference/configuration/tidb-server/tidb-specific-variables/'] --- # System Variables