diff --git a/dev/TOC.md b/dev/TOC.md index a34cde91c4ec5..6886d58f71b63 100644 --- a/dev/TOC.md +++ b/dev/TOC.md @@ -274,6 +274,7 @@ - [Understanding the Query Execution Plan](/dev/reference/performance/understanding-the-query-execution-plan.md) - [Introduction to Statistics](/dev/reference/performance/statistics.md) - [Optimizer Hints](/dev/reference/performance/optimizer-hints.md) + - [Follower Read](/dev/reference/performance/follower-read.md) - [Check the TiDB Cluster Status Using SQL Statements](/dev/reference/performance/check-cluster-status-using-sql-statements.md) - [Execution Plan Binding](/dev/reference/performance/execution-plan-bind.md) - [Statement Summary Table](/dev/reference/performance/statement-summary.md) diff --git a/dev/reference/performance/follower-read.md b/dev/reference/performance/follower-read.md new file mode 100644 index 0000000000000..36b4f2f2fde52 --- /dev/null +++ b/dev/reference/performance/follower-read.md @@ -0,0 +1,50 @@ +--- +title: Follower Read +summary: This document describes the use and implementation of Follower Read. +category: reference +--- + +# Follower Read + +When a read hotspot appears in a Region, the Region leader can become a read bottleneck for the entire system. In this situation, enabling the Follower Read feature can significantly reduce the load of the leader, and improve the throughput of the whole system by balancing the load among multiple followers. This document introduces the use and implementation mechanism of Follower Read. + +## Overview + +The Follower Read feature refers to using any follower replica of a Region to serve a read request under the premise of strongly consistent read. This feature improves the throughput of the TiDB cluster and reduces the load of the leader. It contains a series of load balancing mechanisms that offload TiKV read loads from the leader replica to the follower replica in a Region. TiKV's Follower Read implementation guarantees the linear consistency of single-row data reading. Combined with Snapshot Isolation in TiDB, this implementation also provides users with strongly consistent read. + +> **Note:** +> +> To achieve strongly consistent read, the follower node currently requires additional `ReadIndex` overhead. Therefore, the main benefits of Follower Read are to isolate read and write requests of the cluster and to increase overall read throughput. Regarding the latency of a single request, it requires one more interaction overhead with `ReadIndex` of the Raft protocol than the traditional leader reads. + +## Usage + +To enable TiDB's Follower Read feature, set the value of the `tidb_replica_read` session variable to `follower`: + +{{}} + +```sql +set @@tidb_replica_read = 'follower'; +``` + +Scope: SESSION + +Default: leader + +This variable is used to set the data read mode expected by the current session. + +- When the value of `tidb_replica_read` is set to `leader` or an empty string, TiDB maintains its original behavior and sends all read operations to the leader replica to perform. +- When the value of `tidb_replica_read` is set to `follower`, TiDB selects a follower replica of the Region to perform all read operations. + +## Implementation mechanism + +Before the Follower Read feature was introduced, TiDB applied the strong leader policy and submitted all read and write requests to the leader node of a Region to handle. Although TiKV can distribute Regions evenly on multiple physical nodes, for each Region, only the leader can provide external services. The other followers can do nothing to handle read requests but receive the data replicated from the leader at all times and prepare for voting to elect a leader in case of a failover. + +To allow data reading in the follower node without violating linear consistency or affecting Snapshot Isolation in TiDB, the follower node needs to use `ReadIndex` of the Raft protocol to ensure that the read request can read the latest data that has been committed on the leader. At the TiDB level, the Follower Read feature simply needs to send the read request of a Region to a follower replica based on the load balancing policy. + +### Strongly consistent read + +When the follower node processes a read request, it first uses `ReadIndex` of the Raft protocol to interact with the leader of the Region, to obtain the latest commit index of the current Raft group. After the latest commit index of the leader is applied locally to the follower, the processing of a read request starts. + +### Follower replica selection strategy + +Because the Follower Read feature guarantees linear consistency without affecting Snapshot Isolation, TiDB adopts the round-robin strategy to select the follower replica. Although TiKV can select any follower replica to handle any read request, considering the different replication speed among followers, if the load balancing granularity is too fine, it might cause significant fluctuation of latency. Currently, the granularity of the Follower Read load balancing policy is at the connection level. For a TiDB client connected to a specific Region, the selected follower is fixed, and is switched only when it fails or the scheduling policy is adjusted. diff --git a/v3.1/TOC.md b/v3.1/TOC.md index e6d01fefa5848..f4543747fc77d 100644 --- a/v3.1/TOC.md +++ b/v3.1/TOC.md @@ -273,6 +273,7 @@ - [Understanding the Query Execution Plan](/v3.1/reference/performance/understanding-the-query-execution-plan.md) - [Introduction to Statistics](/v3.1/reference/performance/statistics.md) - [Optimizer Hints](/v3.1/reference/performance/optimizer-hints.md) + - [Follower Read](/v3.1/reference/performance/follower-read.md) - [Check the TiDB Cluster Status Using SQL Statements](/v3.1/reference/performance/check-cluster-status-using-sql-statements.md) - [Execution Plan Binding](/v3.1/reference/performance/execution-plan-bind.md) - [Statement Summary Table](/v3.1/reference/performance/statement-summary.md) diff --git a/v3.1/reference/performance/follower-read.md b/v3.1/reference/performance/follower-read.md new file mode 100644 index 0000000000000..36b4f2f2fde52 --- /dev/null +++ b/v3.1/reference/performance/follower-read.md @@ -0,0 +1,50 @@ +--- +title: Follower Read +summary: This document describes the use and implementation of Follower Read. +category: reference +--- + +# Follower Read + +When a read hotspot appears in a Region, the Region leader can become a read bottleneck for the entire system. In this situation, enabling the Follower Read feature can significantly reduce the load of the leader, and improve the throughput of the whole system by balancing the load among multiple followers. This document introduces the use and implementation mechanism of Follower Read. + +## Overview + +The Follower Read feature refers to using any follower replica of a Region to serve a read request under the premise of strongly consistent read. This feature improves the throughput of the TiDB cluster and reduces the load of the leader. It contains a series of load balancing mechanisms that offload TiKV read loads from the leader replica to the follower replica in a Region. TiKV's Follower Read implementation guarantees the linear consistency of single-row data reading. Combined with Snapshot Isolation in TiDB, this implementation also provides users with strongly consistent read. + +> **Note:** +> +> To achieve strongly consistent read, the follower node currently requires additional `ReadIndex` overhead. Therefore, the main benefits of Follower Read are to isolate read and write requests of the cluster and to increase overall read throughput. Regarding the latency of a single request, it requires one more interaction overhead with `ReadIndex` of the Raft protocol than the traditional leader reads. + +## Usage + +To enable TiDB's Follower Read feature, set the value of the `tidb_replica_read` session variable to `follower`: + +{{}} + +```sql +set @@tidb_replica_read = 'follower'; +``` + +Scope: SESSION + +Default: leader + +This variable is used to set the data read mode expected by the current session. + +- When the value of `tidb_replica_read` is set to `leader` or an empty string, TiDB maintains its original behavior and sends all read operations to the leader replica to perform. +- When the value of `tidb_replica_read` is set to `follower`, TiDB selects a follower replica of the Region to perform all read operations. + +## Implementation mechanism + +Before the Follower Read feature was introduced, TiDB applied the strong leader policy and submitted all read and write requests to the leader node of a Region to handle. Although TiKV can distribute Regions evenly on multiple physical nodes, for each Region, only the leader can provide external services. The other followers can do nothing to handle read requests but receive the data replicated from the leader at all times and prepare for voting to elect a leader in case of a failover. + +To allow data reading in the follower node without violating linear consistency or affecting Snapshot Isolation in TiDB, the follower node needs to use `ReadIndex` of the Raft protocol to ensure that the read request can read the latest data that has been committed on the leader. At the TiDB level, the Follower Read feature simply needs to send the read request of a Region to a follower replica based on the load balancing policy. + +### Strongly consistent read + +When the follower node processes a read request, it first uses `ReadIndex` of the Raft protocol to interact with the leader of the Region, to obtain the latest commit index of the current Raft group. After the latest commit index of the leader is applied locally to the follower, the processing of a read request starts. + +### Follower replica selection strategy + +Because the Follower Read feature guarantees linear consistency without affecting Snapshot Isolation, TiDB adopts the round-robin strategy to select the follower replica. Although TiKV can select any follower replica to handle any read request, considering the different replication speed among followers, if the load balancing granularity is too fine, it might cause significant fluctuation of latency. Currently, the granularity of the Follower Read load balancing policy is at the connection level. For a TiDB client connected to a specific Region, the selected follower is fixed, and is switched only when it fails or the scheduling policy is adjusted.