diff --git a/TOC.md b/TOC.md index ad8375f603096..8268439165f6c 100644 --- a/TOC.md +++ b/TOC.md @@ -433,7 +433,9 @@ + [`COLLATIONS`](/information-schema/information-schema-collations.md) + [`COLLATION_CHARACTER_SET_APPLICABILITY`](/information-schema/information-schema-collation-character-set-applicability.md) + [`COLUMNS`](/information-schema/information-schema-columns.md) + + [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md) + [`DDL_JOBS`](/information-schema/information-schema-ddl-jobs.md) + + [`DEADLOCKS`](/information-schema/information-schema-deadlocks.md) + [`ENGINES`](/information-schema/information-schema-engines.md) + [`INSPECTION_RESULT`](/information-schema/information-schema-inspection-result.md) + [`INSPECTION_RULES`](/information-schema/information-schema-inspection-rules.md) @@ -454,6 +456,7 @@ + [`TIDB_HOT_REGIONS`](/information-schema/information-schema-tidb-hot-regions.md) + [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) + [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) + + [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md) + [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md) + [`TIKV_REGION_PEERS`](/information-schema/information-schema-tikv-region-peers.md) + [`TIKV_REGION_STATUS`](/information-schema/information-schema-tikv-region-status.md) diff --git a/information-schema/information-schema-data-lock-waits.md b/information-schema/information-schema-data-lock-waits.md new file mode 100644 index 0000000000000..71cba4cda5758 --- /dev/null +++ b/information-schema/information-schema-data-lock-waits.md @@ -0,0 +1,86 @@ +--- +title: DATA_LOCK_WAITS +summary: Learn the `DATA_LOCK_WAITS` information_schema table. +--- + +# DATA_LOCK_WAITS + +The `DATA_LOCK_WAITS` table shows the ongoing pessimistic locks waiting on all TiKV nodes in the cluster. + +> **Warning:** +> +> Currently, this is an experimental feature. The definition and behavior of the table structure might have major changes in future releases. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC data_lock_waits; +``` + +```sql ++------------------------+---------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++------------------------+---------------------+------+------+---------+-------+ +| KEY | varchar(64) | NO | | NULL | | +| TRX_ID | bigint(21) unsigned | NO | | NULL | | +| CURRENT_HOLDING_TRX_ID | bigint(21) unsigned | NO | | NULL | | +| SQL_DIGEST | varchar(64) | YES | | NULL | | ++------------------------+---------------------+------+------+---------+-------+ +``` + +The meaning of each column field in the `DATA_LOCK_WAITS` table is as follows: + +* `KEY`: The KEY that is waiting for the lock and displayed in the form of hexadecimal string. +* `TRX_ID`: The ID of the transaction that is waiting for the lock. This ID is also the `start_ts` of the transaction. +* `CURRENT_HOLDING_TRX_ID`: The ID of the transaction that currently holds the lock. This ID is also the `start_ts` of the transaction. +* `SQL_DIGEST`: The digest of the SQL statement that is currently blocked in the lock-waiting transaction. + +> **Warning:** +> +> * The information in this table is obtained in real time from all TiKV nodes during the query. Currently, even if the `WHERE` condition is added, TiDB might still collect information from all TiKV nodes. If the cluster is large and the load is high, querying this table might cause a potential risk of performance jitter. Therefore, use this table according to your actual situation. +> * The information from different TiKV nodes is NOT guaranteed to be the snapshot at the same point in time. + +## Example + +{{< copyable "sql" >}} + +```sql +select * from information_schema.data_lock_waits\G +``` + +```sql +*************************** 1. row *************************** + KEY: 7480000000000000355f728000000000000002 + TRX_ID: 425405024158875649 +CURRENT_HOLDING_TRX_ID: 425405016242126849 + SQL_DIGEST: f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb22 +2 rows in set (0.01 sec) +``` + +The above query result shows that the transaction of the ID `425405024158875649` was trying to obtain the pessimistic lock on the key `7480000000000000355f728000000000000002` when the statement with digest `"f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb22"` was being executed, but the lock on this key was held by the transaction of the ID `425405016242126849`. + +## SQL Digest + +The `DATA_LOCK_WAITS` table records the SQL digest but not the original SQL statement. + +SQL digest is the hash value of the normalized SQL statement. To find the original SQL statement corresponding to the SQL digest, perform one of the following operations: + +- For the statements executed on the current TiDB node in the recent period of time, you can find the corresponding original SQL statement in the `STATEMENTS_SUMMARY` or `STATEMENTS_SUMMARY_HISTORY` table according to the SQL digest. +- For the statements executed on all TiDB nodes in the entire cluster in the recent period of time, you can find the corresponding SQL statement in the `CLUSTER_STATEMENTS_SUMMARY` or `CLUSTER_STATEMENTS_SUMMARY_HISTORY` table according to the SQL digest. + +{{< copyable "sql" >}} + +```sql +select digest, digest_text from information_schema.statements_summary where digest = "f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2"; +``` + +```sql ++------------------------------------------------------------------+---------------------------------------+ +| digest | digest_text | ++------------------------------------------------------------------+---------------------------------------+ +| f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2 | update `t` set `v` = ? where `id` = ? | ++------------------------------------------------------------------+---------------------------------------+ +``` + +For detailed description of SQL digest, `STATEMENTS_SUMMARY`, `STATEMENTS_SUMMARY_HISTORY`, `CLUSTER_STATEMENTS_SUMMARY`, and `CLUSTER_STATEMENTS_SUMMARY_HISTORY` tables, see [Statement Summary Tables](/statement-summary-tables.md). diff --git a/information-schema/information-schema-deadlocks.md b/information-schema/information-schema-deadlocks.md new file mode 100644 index 0000000000000..6133ac3988198 --- /dev/null +++ b/information-schema/information-schema-deadlocks.md @@ -0,0 +1,200 @@ +--- +title: DEADLOCKS +summary: Learn the `DEADLOCKS` information_schema table. +--- + +# DEADLOCKS + +The `DEADLOCKS` table shows the information of the several deadlock errors that have occurred recently on the current TiDB node. + +> **Warning:** +> +> Currently, this is an experimental feature. The definition and behavior of the table structure might have major changes in future releases. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC deadlocks; +``` + +```sql ++--------------------+---------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++--------------------+---------------------+------+------+---------+-------+ +| DEADLOCK_ID | bigint(21) | NO | | NULL | | +| OCCUR_TIME | timestamp(6) | YES | | NULL | | +| RETRYABLE | tinyint(1) | NO | | NULL | | +| TRY_LOCK_TRX_ID | bigint(21) unsigned | NO | | NULL | | +| CURRENT_SQL_DIGEST | varchar(64) | YES | | NULL | | +| KEY | text | YES | | NULL | | +| TRX_HOLDING_LOCK | bigint(21) unsigned | NO | | NULL | | ++--------------------+---------------------+------+------+---------+-------+ +``` + +The `DEADLOCKS` table uses multiple rows to show the same deadlock event, and each row displays the information about one of the transactions involved in the deadlock event. If the TiDB node records multiple deadlock errors, each error is distinguished using the `DEADLOCK_ID` column. The same `DEADLOCK_ID` indicates the same deadlock event. Note that `DEADLOCK_ID` **does not guarantee global uniqueness and will not be persisted**. It only shows the same deadlock event in the same result set. + +The meaning of each column field in the `DEADLOCKS` table is as follows: + +* `DEADLOCK_ID`: The ID of the deadlock event. When multiple deadlock errors exist in the table, you can use this column to distinguish rows that belong to different deadlock errors. +* `OCCUR_TIME`: The time when the deadlock error occurs. +* `RETRYABLE`: Whether the deadlock error can be retried. Currently, TiDB does not support collecting the information of the retryable deadlock error, so the value of this field is always `0`. For the description of retryable deadlock errors, see the [Retryable deadlock errors](#retryable-deadlock-errors) section. +* `TRY_LOCK_TRX_ID`: The ID of the transaction that tries to acquire lock. This ID is also the `start_ts` of the transaction. +* `CURRENT_SQL_DIGEST`: The digest of the SQL statement currently being executed in the lock-acquiring transaction. +* `KEY`: The blocked key that the transaction tries to lock. The value of this field is displayed in the form of hexadecimal string. +* `TRX_HOLDING_LOCK`: The ID of the transaction that currently holds the lock on the key and causes blocking. This ID is also the `start_ts` of the transaction. + +To adjust the maximum number of deadlock events that can be recorded in the `DEADLOCKS` table, adjust the [`pessimistic-txn.deadlock-history-capacity`](/tidb-configuration-file.md#deadlock-history-capacity) configuration in the TiDB configuration file. By default, the information of the recent 10 deadlock events is recorded in the table. + +## Example 1 + +Assume that the table definition and the initial data are as follows: + +{{< copyable "sql" >}} + +```sql +create table t (id int primary key, v int); +insert into t values (1, 10), (2, 20); +``` + +Execute the two transactions in the following order: + +| Transaction 1 | Transaction 2 | Description | +|--------------------------------------|--------------------------------------|----------------------| +| `update t set v = 11 where id = 1;` | | | +| | `update t set v = 21 where id = 2;` | | +| `update t set v = 12 where id = 2;` | | Transaction 1 gets blocked. | +| | `update t set v = 22 where id = 1;` | Transaction 2 reports a deadlock error. | + +Next, transaction 2 reports a deadlock error. At this time, query the `DEADLOCKS` table: + +{{< copyable "sql" >}} + +```sql +select * from information_schema.deadlocks; +``` + +The expected output is as follows: + +```sql ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| DEADLOCK_ID | OCCUR_TIME | RETRYABLE | TRY_LOCK_TRX_ID | CURRENT_SQL_DIGEST | KEY | TRX_HOLDING_LOCK | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904707 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000002 | 425405959304904708 | +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904708 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000001 | 425405959304904707 | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +``` + +Two rows of data are generated in the `DEADLOCKS` table. The `DEADLOCK_ID` field of both rows is `1`, which means that the information in both rows belongs to the same deadlock error. The first row shows that the transaction of the ID `425405959304904707` is blocked on the key of `"7480000000000000385F728000000000000002"` by the transaction of the ID `"425405959304904708"`. The second row shows that the transaction of the ID `"425405959304904708"` is blocked on the key of `"7480000000000000385F728000000000000001"` by the transaction of the ID `425405959304904707`, which constitutes mutual blocking and forms a deadlock. + +## Example 2 + +Assume that you query the `DEADLOCKS` table and get the following result: + +```sql ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| DEADLOCK_ID | OCCUR_TIME | RETRYABLE | TRY_LOCK_TRX_ID | CURRENT_SQL_DIGEST | KEY | TRX_HOLDING_LOCK | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904707 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000002 | 425405959304904708 | +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904708 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000001 | 425405959304904707 | +| 2 | 2021-06-04 08:22:56.795410 | 0 | 425405961664462853 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000002 | 425405961664462854 | +| 2 | 2021-06-04 08:22:56.795410 | 0 | 425405961664462854 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000003 | 425405961664462855 | +| 2 | 2021-06-04 08:22:56.795410 | 0 | 425405961664462855 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000001 | 425405961664462853 | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +``` + +The `DEADLOCK_ID` column in the above query result shows that the first two rows together represent the information of a deadlock error, and the two transactions that wait for each other form the deadlock. The next three rows together represent another deadlock information, and the three transactions that wait in a cycle form the deadlock. + +## Retryable deadlock errors + +> **Note:** +> +> Currently, TiDB does not support collecting retryable deadlock errors in the `DEADLOCKS` table. + +When transaction A is blocked by a lock already held by transaction B, and transaction B is directly or indirectly blocked by the lock held by the current transaction A, a deadlock error will occur. In this deadlock, there might be two cases: + ++ Case 1: Transaction B might be (directly or indirectly) blocked by a lock generated by a statement that has been executed after transaction A starts and before transaction A gets blocked. ++ Case 2: Transaction B might also be blocked by the statement currently being executed in transaction A. + +In case 1, TiDB will report a deadlock error to the client of transaction A and terminate the transaction. + +In case 2, the statement currently being executed in transaction A will be automatically retried in TiDB. For example, suppose that transaction A executes the following statement: + +{{< copyable "sql" >}} + +```sql +update t set v = v + 1 where id = 1 or id = 2; +``` + +Transaction B executes the following two statements successively. + +{{< copyable "sql" >}} + +```sql +update t set v = 4 where id = 2; +update t set v = 2 where id = 1; +``` + +Then if transaction A locks the two rows with `id = 1` and `id = 2`, and the two transactions run in the following sequence: + +1. Transaction A locks the row with `id = 1`. +2. Transaction B executes the first statement and locks the row with `id = 2`. +3. Transaction B executes the second statement and tries to lock the row with `id = 1`, which is blocked by transaction A. +4. Transaction A tries to lock the row with `id = 2` and is blocked by transaction B, which forms a deadlock. + +For this case, because the statement of transaction A that blocks other transactions is also the statement currently being executed, the pessimistic lock on the current statement can be resolved (so that transaction B can continue to run), and the current statement can be retried. TiDB uses the key hash internally to determine whether this is the case. + +When a retryable deadlock occurs, the internal automatic retry will not cause a transaction error, so it is transparent to the client. However, if this situation occurs frequently, the performance might be affected. When this occurs, you can see `single statement deadlock, retry statement` in the TiDB log. + +## CLUSTER_DEADLOCKS + +The `CLUSTER_DEADLOCKS` table returns information about the recent deadlock errors on each TiDB node in the entire cluster, which is the information of the `DEADLOCKS` table on each node combined together. `CLUSTER_DEADLOCKS` also contains an additional `INSTANCE` column to display the IP address and port of the node to distinguish between different TiDB nodes. + +Note that, because `DEADLOCK_ID` does not guarantee global uniqueness, in the query result of the `CLUSTER_DEADLOCKS` table, you need to use the `INSTANCE` and `DEADLOCK_ID` together to distinguish the information of different deadlock errors in the result set. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC cluster_deadlocks; +``` + +```sql ++--------------------+---------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++--------------------+---------------------+------+------+---------+-------+ +| INSTANCE | varchar(64) | YES | | NULL | | +| DEADLOCK_ID | bigint(21) | NO | | NULL | | +| OCCUR_TIME | timestamp(6) | YES | | NULL | | +| RETRYABLE | tinyint(1) | NO | | NULL | | +| TRY_LOCK_TRX_ID | bigint(21) unsigned | NO | | NULL | | +| CURRENT_SQL_DIGEST | varchar(64) | YES | | NULL | | +| KEY | text | YES | | NULL | | +| TRX_HOLDING_LOCK | bigint(21) unsigned | NO | | NULL | | ++--------------------+---------------------+------+------+---------+-------+ +``` + +## SQL Digest + +The `DEADLOCKS` table records the SQL digest but not the original SQL statement. + +SQL digest is the hash value after the SQL normalization. To find the original SQL statement corresponding to the SQL digest, perform one of the following operations: + +- For the statements executed on the current TiDB node in the recent period of time, you can find the corresponding original SQL statement in the `STATEMENTS_SUMMARY` or `STATEMENTS_SUMMARY_HISTORY` table according to the SQL digest. +- For the statements executed on all TiDB nodes in the entire cluster in the recent period of time, you can find the corresponding SQL statement in the `CLUSTER_STATEMENTS_SUMMARY` or `CLUSTER_STATEMENTS_SUMMARY_HISTORY` table according to the SQL digest. + +{{< copyable "sql" >}} + +```sql +select digest, digest_text from information_schema.statements_summary where digest = "f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2"; +``` + +```sql ++------------------------------------------------------------------+---------------------------------------+ +| digest | digest_text | ++------------------------------------------------------------------+---------------------------------------+ +| f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2 | update `t` set `v` = ? where `id` = ? | ++------------------------------------------------------------------+---------------------------------------+ +``` + +For detailed description of SQL digest, `STATEMENTS_SUMMARY`, `STATEMENTS_SUMMARY_HISTORY`, `CLUSTER_STATEMENTS_SUMMARY`, and `CLUSTER_STATEMENTS_SUMMARY_HISTORY` tables, see [Statement Summary Tables](/statement-summary-tables.md). diff --git a/information-schema/information-schema-tidb-trx.md b/information-schema/information-schema-tidb-trx.md new file mode 100644 index 0000000000000..1a9008bc49f98 --- /dev/null +++ b/information-schema/information-schema-tidb-trx.md @@ -0,0 +1,137 @@ +--- +title: TIDB_TRX +summary: Learn the `TIDB_TRX` information_schema table. +--- + +# TIDB_TRX + +The `TIDB_TRX` table provides information about the transactions currently being executed on the TiDB node. + +> **Warning:** +> +> * Currently, this is an experimental feature. The definition and behavior of the table structure might have major changes in future releases. +> * Currently, the `TIDB_TRX` table does not support displaying information of TiDB's internal transactions. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC tidb_trx; +``` + +```sql ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +| ID | bigint(21) unsigned | NO | PRI | NULL | | +| START_TIME | timestamp(6) | YES | | NULL | | +| CURRENT_SQL_DIGEST | varchar(64) | YES | | NULL | | +| STATE | enum('Normal','LockWaiting','Committing','RollingBack') | YES | | NULL | | +| WAITING_START_TIME | timestamp(6) | YES | | NULL | | +| MEM_BUFFER_KEYS | bigint(64) | YES | | NULL | | +| MEM_BUFFER_BYTES | bigint(64) | YES | | NULL | | +| SESSION_ID | bigint(21) unsigned | YES | | NULL | | +| USER | varchar(16) | YES | | NULL | | +| DB | varchar(64) | YES | | NULL | | +| ALL_SQL_DIGESTS | text | YES | | NULL | | ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +``` + +The meaning of each column field in the `TIDB_TRX` table is as follows: + +* `ID`: The transaction ID, which is the `start_ts` (start timestamp) of the transaction. +* `START_TIME`: The start time of the transaction, which is the physical time corresponding to the `start_ts` of the transaction. +* `CURRENT_SQL_DIGEST`: The digest of the SQL statement currently being executed in the transaction. +* `STATE`: The current state of the transaction. The possible values ​​include: + * `Normal`: The transaction is being executed normally or in an idle state. + * `LockWaiting`: The transaction is waiting for the pessimistic lock to be acquired. Note that the transaction enters this state at the beginning of the pessimistic locking operation, no matter whether it is blocked by other transactions or not. + * `Committing`: The transaction is in the process of commit. + * `RollingBack`: The transaction is being rolled back. +* `WAITING_START_TIME`: When the value of `STATE` is `LockWaiting`, this column shows the start time of the waiting. +* `MEM_BUFFER_KEYS`: The number of keys written into the memory buffer by the current transaction. +* `MEM_BUFFER_BYTES`: The total number of key-value bytes written into the memory buffer by the current transaction. +* `SESSION_ID`: The ID of the session to which this transaction belongs. +* `USER`: The name of the user who performs the transaction. +* `DB`: The current default database name of the session in which the transaction is executed. +* `ALL_SQL_DIGESTS`: The digest list of statements that have been executed in this transaction. For each transaction, the first 50 statements at most are recorded. + +## Example + +{{< copyable "sql" >}} + +```sql +select * from information_schema.tidb_trx\G +``` + +```sql +*************************** 1. row *************************** + ID: 425403705115541506 + START_TIME: 2021-06-04 05:59:10.691000 +CURRENT_SQL_DIGEST: NULL + STATE: Normal +WAITING_START_TIME: NULL + MEM_BUFFER_KEYS: 2 + MEM_BUFFER_BYTES: 48 + SESSION_ID: 7 + USER: root + DB: test + ALL_SQL_DIGESTS: [e6f07d43b5c21db0fbb9a31feac2dc599787763393dd5acbfad80e247eb02ad5, 04fa858fa491c62d194faec2ab427261cc7998b3f1ccf8f6844febca504cb5e9, f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2] +1 row in set (0.00 sec) +``` + +The query result of the above example indicates that a transaction is being executed on the current node (the `STATE` is `Normal`), and this transaction is currently idle (`CURRENT_SQL_DIGEST` is `NULL`). This transaction has executed three statements (there are three records in the `ALL_SQL_DIGESTS` list and they are the digests of the three executed statements). + +## CLUSTER_TIDB_TRX + +The `TIDB_TRX` table only provides information about the transactions that are being executed on a single TiDB node. If you want to view the information of the transactions that are being executed on all TiDB nodes in the entire cluster, you need to query the `CLUSTER_TIDB_TRX` table. Compared with the query result of the `TIDB_TRX` table, the query result of the `CLUSTER_TIDB_TRX` table contains an extra `INSTANCE` field. The `INSTANCE` field displays the IP address and port of each node in the cluster, which is used to distinguish the TiDB node where the transaction is located. + +{{< copyable "sql" >}} + +```sql +USE information_schema; +DESC cluster_tidb_trx; +``` + +```sql ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +| Field | Type | Null | Key | Default | Extra | ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +| INSTANCE | varchar(64) | YES | | NULL | | +| ID | bigint(21) unsigned | NO | PRI | NULL | | +| START_TIME | timestamp(6) | YES | | NULL | | +| CURRENT_SQL_DIGEST | varchar(64) | YES | | NULL | | +| STATE | enum('Normal','LockWaiting','Committing','RollingBack') | YES | | NULL | | +| WAITING_START_TIME | timestamp(6) | YES | | NULL | | +| MEM_BUFFER_KEYS | bigint(64) | YES | | NULL | | +| MEM_BUFFER_BYTES | bigint(64) | YES | | NULL | | +| SESSION_ID | bigint(21) unsigned | YES | | NULL | | +| USER | varchar(16) | YES | | NULL | | +| DB | varchar(64) | YES | | NULL | | +| ALL_SQL_DIGESTS | text | YES | | NULL | | ++--------------------+---------------------------------------------------------+------+------+---------+-------+ +``` + +## SQL Digest + +The `TIDB_TRX` table only records SQL digests, not the original SQL statement. + +SQL digest is the hash value after the SQL normalization. To find the original SQL statement corresponding to the SQL digest, perform one of the following operations: + +- For the statements executed on the current TiDB node in the recent period of time, you can find the corresponding orginal SQL statement from the SQL digest in `STATEMENTS_SUMMARY` or `STATEMENTS_SUMMARY_HISTORY`. +- For the statements executed on all TiDB nodes in the entire cluster in the recent period of time, you can find the corresponding SQL statement from the SQL digest in `CLUSTER_STATEMENTS_SUMMARY` or `CLUSTER_STATEMENTS_SUMMARY_HISTORY`. + +{{< copyable "sql" >}} + +```sql +select digest, digest_text from information_schema.statements_summary where digest = "f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2"; +``` + +```sql ++------------------------------------------------------------------+---------------------------------------+ +| digest | digest_text | ++------------------------------------------------------------------+---------------------------------------+ +| f7530877a35ae65300c42250abd8bc731bbaf0a7cabc05dab843565230611bb2 | update `t` set `v` = ? where `id` = ? | ++------------------------------------------------------------------+---------------------------------------+ +``` + +For detailed description of SQL digest, `STATEMENTS_SUMMARY`, `STATEMENTS_SUMMARY_HISTORY`, `CLUSTER_STATEMENTS_SUMMARY`, and `CLUSTER_STATEMENTS_SUMMARY_HISTORY` tables, see [Statement Summary Tables](/statement-summary-tables.md). diff --git a/information-schema/information-schema.md b/information-schema/information-schema.md index 33700d4bdc686..943bd0ee87291 100644 --- a/information-schema/information-schema.md +++ b/information-schema/information-schema.md @@ -55,6 +55,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`CLIENT_ERRORS_SUMMARY_BY_USER`](/information-schema/client-errors-summary-by-user.md) | Provides a summary of errors and warnings generated by clients. | | [`CLIENT_ERRORS_SUMMARY_GLOBAL`](/information-schema/client-errors-summary-global.md) | Provides a summary of errors and warnings generated by clients. | | [`CLUSTER_CONFIG`](/information-schema/information-schema-cluster-config.md) | Provides details about configuration settings for the entire TiDB cluster. | +| `CLUSTER_DEADLOCKS` | Provides a cluster-level view of the `DEADLOCKS` table. | | [`CLUSTER_HARDWARE`](/information-schema/information-schema-cluster-info.md) | Provides details on the underlying physical hardware discovered on each TiDB component. | | [`CLUSTER_INFO`](/information-schema/information-schema-cluster-info.md) | Provides details on the current cluster topology. | | [`CLUSTER_LOAD`](/information-schema/information-schema-cluster-load.md) | Provides current load information for TiDB servers in the cluster. | @@ -62,9 +63,12 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | `CLUSTER_PROCESSLIST` | Provides a cluster-level view of the `PROCESSLIST` table. | | `CLUSTER_SLOW_QUERY` | Provides a cluster-level view of the `SLOW_QUERY` table. | | `CLUSTER_STATEMENTS_SUMMARY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY` table. | -| `CLUSTER_STATEMENTS_SUMMARY_HISTORY` | Provides a cluster-level view of the `CLUSTER_STATEMENTS_SUMMARY_HISTORY` table. | +| `CLUSTER_STATEMENTS_SUMMARY_HISTORY` | Provides a cluster-level view of the `STATEMENTS_SUMMARY_HISTORY` table. | +| `CLUSTER_TIDB_TRX` | Provides a cluster-level view of the `TIDB_TRX` table. | | [`CLUSTER_SYSTEMINFO`](/information-schema/information-schema-cluster-systeminfo.md) | Provides details about kernel parameter configuration for servers in the cluster. | +| [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md) | Provides the lock-waiting information on the TiKV server. | | [`DDL_JOBS`](/information-schema/information-schema-ddl-jobs.md) | Provides similar output to `ADMIN SHOW DDL JOBS` | +| [`DEADLOCKS`](/information-schema/information-schema-deadlocks.md) | Provides the information of several deadlock errors that have recently occurred. | | [`INSPECTION_RESULT`](/information-schema/information-schema-inspection-result.md) | Triggers internal diagnostics checks. | | [`INSPECTION_RULES`](/information-schema/information-schema-inspection-rules.md) | A list of internal diagnostic checks performed. | | [`INSPECTION_SUMMARY`](/information-schema/information-schema-inspection-summary.md) | A summarized report of important monitoring metrics. | @@ -79,6 +83,7 @@ Many `INFORMATION_SCHEMA` tables have a corresponding `SHOW` command. The benefi | [`TIDB_HOT_REGIONS`](/information-schema/information-schema-tidb-hot-regions.md) | Provides statistics about which regions are hot. | | [`TIDB_INDEXES`](/information-schema/information-schema-tidb-indexes.md) | Provides index information about TiDB tables. | | [`TIDB_SERVERS_INFO`](/information-schema/information-schema-tidb-servers-info.md) | Provides a list of TiDB servers (namely, tidb-server component) | +| [`TIDB_TRX`](/information-schema/information-schema-tidb-trx.md) | Provides the information of the transactions that are being executed on the TiDB node. | | [`TIFLASH_REPLICA`](/information-schema/information-schema-tiflash-replica.md) | Provides details about TiFlash replicas. | | [`TIKV_REGION_PEERS`](/information-schema/information-schema-tikv-region-peers.md) | Provides details about where regions are stored. | | [`TIKV_REGION_STATUS`](/information-schema/information-schema-tikv-region-status.md) | Provides statistics about regions. | diff --git a/tidb-configuration-file.md b/tidb-configuration-file.md index 3a3f34f74ed34..cdea33c66d9f3 100644 --- a/tidb-configuration-file.md +++ b/tidb-configuration-file.md @@ -609,6 +609,13 @@ For pessimistic transaction usage, refer to [TiDB Pessimistic Transaction Mode]( - The maximum number of retries of each statement in pessimistic transactions. If the number of retries exceeds this limit, an error occurs. - Default value: `256` +### deadlock-history-capacity + ++ The maximum number of deadlock events that can be recorded in the [`INFORMATION_SCHEMA.DEADLOCKS`](/information-schema/information-schema-deadlocks.md) table of a single TiDB server. If this table is in full volume and an additional deadlock event occurs, the earliest record in the table will be removed to make place for the newest error. ++ Default value: `10` ++ Minimum value: `0` ++ Maximum value: `10000` + ## experimental The `experimental` section, introduced in v3.1.0, describes configurations related to the experimental features of TiDB. diff --git a/troubleshoot-lock-conflicts.md b/troubleshoot-lock-conflicts.md index f12e7084b0c04..16646e062855f 100644 --- a/troubleshoot-lock-conflicts.md +++ b/troubleshoot-lock-conflicts.md @@ -204,7 +204,7 @@ Solutions: * If the above error occurs frequently, it is recommended to adjust the application logic. #### TTL manager has timed out - + The transaction execution time can not exceed the GC time limit. In addition, the TTL time of pessimistic transactions has an upper limit, whose default value is 1 hour. Therefore, a pessimistic transaction executed for more than 1 hour will fail to commit. This timeout threshold is controlled by the TiDB parameter [performance.max-txn-ttl](https://github.com/pingcap/tidb/blob/master/config/config.toml.example). When the execution time of a pessimistic transaction exceeds the TTL time, the following error message occurs in the TiDB log: @@ -215,7 +215,7 @@ TTL manager has timed out, pessimistic locks may expire, please commit or rollba Solutions: -* First, confirm whether the application logic can be optimized. For example, large transactions may trigger TiDB's transaction size limit, which can be split into multiple small transactions. +* First, confirm whether the application logic can be optimized. For example, large transactions may trigger TiDB's transaction size limit, which can be split into multiple small transactions. * Also, you can adjust the related parameters properly to meet the application transaction logic. #### Deadlock found when trying to get lock @@ -230,4 +230,148 @@ When a pessimistic transaction has a deadlock, one of the transactions must be t Solutions: -* The application needs to adjust transaction request logic when there too many deadlocks. +* If it is difficult to confirm the cause of the deadlock, for v5.1 and later versions, you are recommended to try to query the `INFORMATION_SCHEMA.DEADLOCKS` or `INFORMATION_SCHEMA.CLUSTER_DEADLOCKS` system table to get the information of deadlock waiting chain. For details, see the [Deadlock errors](#deadlock-errors) section and the [`DEADLOCKS` table](/information-schema/information-schema-deadlocks.md) document. +* If the deadlock occurs frequently, you need to adjust the transaction query logic in your application to reduce such occurrences. + +### Use Lock View to troubleshoot issues related to pessimistic locks + +Since v5.1, TiDB supports the Lock View feature. This feature has several system tables built in `information_schema` that provide more information about the pessimistic lock conflicts and pessimistic lock waitings. For the detailed introduction of these tables, see the following documents: + +* [`TIDB_TRX` and `CLUSTER_TIDB_TRX`](/information-schema/information-schema-tidb-trx.md): Provides information of all running transactions on the current TiDB node or in the entire cluster, including whether the transaction is in the lock-waiting state, the lock-waiting time, and the digests of statements that have been executed in the transaction. +* [`DATA_LOCK_WAITS`](/information-schema/information-schema-data-lock-waits.md): Provides the pessimistic lock-waiting information in TiKV, including the `start_ts` of the blocking and blocked transaction, the digest of the blocked SQL statement, and the key on which the waiting occurs. +* [`DEADLOCKS` and `CLUSTER_DEADLOCKS`](/information-schema/information-schema-deadlocks.md): Provides the information of several deadlock events that have recently occurred on the current TiDB node or in the entire cluster, including the waiting relationship among transactions in the deadlock loops, the digest of the statement currently being executed in the transaction, and the key on which the waiting occurs. + +> **Warning:** +> +> Currently, this is an experimental feature. The definition and behavior of the table structure might have major changes in future releases. + +The following sections show the examples of troubleshooting some issues using these tables. + +#### Deadlock errors + +To get the information of the recent deadlock errors, you can query the `DEADLOCKS` or `CLUSTER_DEADLOCKS` table. For example: + +{{< copyable "sql" >}} + +```sql +select * from information_schema.deadlocks; +``` + +```sql ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| DEADLOCK_ID | OCCUR_TIME | RETRYABLE | TRY_LOCK_TRX_ID | CURRENT_SQL_DIGEST | KEY | TRX_HOLDING_LOCK | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904707 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000002 | 425405959304904708 | +| 1 | 2021-06-04 08:22:38.765699 | 0 | 425405959304904708 | 22230766411edb40f27a68dadefc63c6c6970d5827f1e5e22fc97be2c4d8350d | 7480000000000000385F728000000000000001 | 425405959304904707 | ++-------------+----------------------------+-----------+--------------------+------------------------------------------------------------------+----------------------------------------+--------------------+ +``` + +The query result above shows the waiting relationship among multiple transactions in the deadlock error, the digest of the SQL statement currently being executed in each transaction, and the key on which the conflict occurs. + +You can get the text of the normalized SQL statement corresponding to the digest of the SQL statements executed recently from the `STATEMENTS_SUMMARY` or `STATEMENTS_SUMMARY_HISTORY` table. For details, see [`STATEMENTS_SUMMARY` and `STATEMENTS_SUMMARY_HISTORY` tables](/statement-summary-tables.md). You can also join the obtained results directly with the `DEADLOCKS` table. Note that the `STATEMENTS_SUMMARY` table might not contain the information of all SQL statements, so left join is used in the following example: + +{{< copyable "sql" >}} + +```sql +select l.deadlock_id, l.occur_time, l.try_lock_trx_id, l.trx_holding_lock, s.digest_text from information_schema.deadlocks as l left join information_schema.statements_summary as s on l.current_sql_digest = s.digest; +``` + +```sql ++-------------+----------------------------+--------------------+--------------------+-----------------------------------------+ +| deadlock_id | occur_time | try_lock_trx_id | trx_holding_lock | digest_text | ++-------------+----------------------------+--------------------+--------------------+-----------------------------------------+ +| 1 | 2021-06-04 08:22:38.765699 | 425405959304904707 | 425405959304904708 | update `t` set `v` = ? where `id` = ? ; | +| 1 | 2021-06-04 08:22:38.765699 | 425405959304904708 | 425405959304904707 | update `t` set `v` = ? where `id` = ? ; | ++-------------+----------------------------+--------------------+--------------------+-----------------------------------------+ +``` + +#### A few hot keys cause queueing locks + +The `DATA_LOCK_WAITS` system table provides the lock-waiting status on the TiKV nodes. When you query this table, TiDB automatically obtains the real-time lock-waiting information from all TiKV nodes. If a few hot keys are frequently locked and block many transactions, you can query the `DATA_LOCK_WAITS` table and aggregate the results by key to try to find the keys on which issues frequently occur: + +{{< copyable "sql" >}} + +```sql +select `key`, count(*) as `count` from information_schema.data_lock_waits group by `key` order by `count` desc; +``` + +```sql ++----------------------------------------+-------+ +| key | count | ++----------------------------------------+-------+ +| 7480000000000000415f728000000000000001 | 2 | +| 7480000000000000415f728000000000000002 | 1 | ++----------------------------------------+-------+ +``` + +To avoid contingency, you might need to make multiple queries. + +If you know the key that frequently has issues occurred, you can try to get the information of the transaction that tries to lock the key from the `TIDB_TRX` or `CLUSTER_TIDB_TRX` table. + +Note that the information displayed in the `TIDB_TRX` and `CLUSTER_TIDB_TRX` tables is also the information of the transactions that are running at the time the query is performed. These tables do not display the information of the completed transactions. If there is a large number of concurrent transactions, the result set of the query might also be large. You can use the `limit` clause or the `where` clause to filter out transactions with a long lock-waiting time. Note that when you join multiple tables in Lock View, the data in different tables might not be obtained at the same time, so the information in different tables might not be consistent. + +{{< copyable "sql" >}} + +```sql +select trx.* from information_schema.data_lock_waits as l left join information_schema.tidb_trx as trx on l.trx_id = trx.id where l.key = "7480000000000000415f728000000000000001"\G +``` + +```sql +*************************** 1. row *************************** + ID: 425496938634543111 + START_TIME: 2021-06-08 08:46:48.341000 +CURRENT_SQL_DIGEST: a4e28cc182bdd18288e2a34180499b9404cd0ba07e3cc34b6b3be7b7c2de7fe9 + STATE: LockWaiting +WAITING_START_TIME: 2021-06-08 08:46:48.388024 + MEM_BUFFER_KEYS: 1 + MEM_BUFFER_BYTES: 19 + SESSION_ID: 87 + USER: root + DB: test + ALL_SQL_DIGESTS: [0fdc781f19da1c6078c9de7eadef8a307889c001e05f107847bee4cfc8f3cdf3, a4e28cc182bdd18288e2a34180499b9404cd0 +ba07e3cc34b6b3be7b7c2de7fe9, a4e28cc182bdd18288e2a34180499b9404cd0ba07e3cc34b6b3be7b7c2de7fe9] +*************************** 2. row *************************** + ID: 425496940994101249 + START_TIME: 2021-06-08 08:46:57.342000 +CURRENT_SQL_DIGEST: a4e28cc182bdd18288e2a34180499b9404cd0ba07e3cc34b6b3be7b7c2de7fe9 + STATE: LockWaiting +WAITING_START_TIME: 2021-06-08 08:46:57.590060 + MEM_BUFFER_KEYS: 0 + MEM_BUFFER_BYTES: 0 + SESSION_ID: 85 + USER: root + DB: test + ALL_SQL_DIGESTS: [0fdc781f19da1c6078c9de7eadef8a307889c001e05f107847bee4cfc8f3cdf3, a4e28cc182bdd18288e2a34180499b9404cd0 +ba07e3cc34b6b3be7b7c2de7fe9] +2 rows in set (0.00 sec) +``` + +#### A transaction is blocked for a long time + +If a transaction is known to be blocked by another transaction (or multiple transactions) and the `start_ts` (transaction ID) of the current transaction is known, you can use the following method to obtain the information of the blocking transaction. Note that when you join multiple tables in Lock View, the data in different tables might not be obtained at the same time, so the information in different tables might not be consistent. + +{{< copyable "sql" >}} + +```sql +select l.key, trx.* from information_schema.data_lock_waits as l join information_schema.tidb_trx as trx on l.current_holding_trx_id = trx.id where l.trx_id = 425497223886536705\G +``` + +```sql +*************************** 1. row *************************** + key: 7480000000000000475f728000000000000002 + ID: 425497219115778059 + START_TIME: 2021-06-08 09:04:38.292000 +CURRENT_SQL_DIGEST: a4e28cc182bdd18288e2a34180499b9404cd0ba07e3cc34b6b3be7b7c2de7fe9 + STATE: LockWaiting +WAITING_START_TIME: 2021-06-08 09:04:38.336264 + MEM_BUFFER_KEYS: 1 + MEM_BUFFER_BYTES: 19 + SESSION_ID: 97 + USER: root + DB: test + ALL_SQL_DIGESTS: [0fdc781f19da1c6078c9de7eadef8a307889c001e05f107847bee4cfc8f3cdf3, a4e28cc182bdd18288e2a34180499b9404cd0 +ba07e3cc34b6b3be7b7c2de7fe9, a4e28cc182bdd18288e2a34180499b9404cd0ba07e3cc34b6b3be7b7c2de7fe9] +1 row in set (0.01 sec) +``` + +If the `start_ts` of the current transaction is unknown, you can try to find it out from the information in the `TIDB_TRX` / `CLUSTER_TIDB_TRX` table or in the [`PROCESSLIST` / `CLUSTER_PROCESSLIST`](/information-schema/information-schema-processlist.md) table.