diff --git a/TOC.md b/TOC.md index 115a16253f9e9..1a6bbb9499639 100644 --- a/TOC.md +++ b/TOC.md @@ -89,6 +89,7 @@ - [Identify Expensive Queries](/how-to/maintain/identify-abnormal-queries/identify-expensive-queries.md) + Scale - [Scale using Ansible](/how-to/scale/with-ansible.md) + - [Scale using TiUP](/how-to/scale/with-tiup.md) - [Scale a TiDB Cluster](/how-to/scale/horizontally.md) + Upgrade - [Upgrade Using TiUP (Recommended)](/how-to/upgrade/using-tiup.md) @@ -315,7 +316,7 @@ - [Use TiFlash](/reference/tiflash/use-tiflash.md) - [Maintain a TiFlash Cluster](/reference/tiflash/maintain.md) - [Monitor TiFlash](/reference/tiflash/monitor.md) - - [Scale TiFlash](/reference/tiflash/scale.md) + - [Scale TiFlash](/how-to/scale/with-tiup.md#2-scale-out-a-tiflash-node) - [Upgrade TiFlash Nodes](/reference/tiflash/upgrade.md) - [Configure TiFlash](/reference/tiflash/configuration.md) - [TiFlash Alert Rules](/reference/tiflash/alert-rules.md) diff --git a/how-to/scale/with-tiup.md b/how-to/scale/with-tiup.md new file mode 100644 index 0000000000000..6d4edaef1cfe0 --- /dev/null +++ b/how-to/scale/with-tiup.md @@ -0,0 +1,231 @@ +--- +title: Scale the TiDB Cluster Using TiUP +summary: +Category: how-to +--- + +# Scale the TiDB Cluster Using TiUP + +The capacity of a TiDB cluster can be increased or decreased without affecting the online services. + +This document describes how to scale the TiDB, TiKV, PD or TiFlash nodes using TiUP. If you have not installed TiUP, refer to the steps in [Install TiUP on the Control Machine](/how-to/upgrade/using-tiup.md#install-tiup-on-the-control-machine) and import the cluster into TiUP before you scale the TiDB cluster. + +To view the current cluster name list, run `tiup cluster list`. + +For example, if the original topology of the cluster is as follows: + +| Host IP | Service | +|:---|:----| +| 10.0.1.3 | TiDB + TiFlash | +| 10.0.1.4 | TiDB + PD | +| 10.0.1.5 | TiKV + Monitor | +| 10.0.1.1 | TiKV | +| 10.0.1.2 | TiKV | + +## Scale out a TiDB/TiKV/PD node + +If you want to add a TiDB node to the `10.0.1.5` host, take the following steps. + +> **Note:** +> +> You can take similar steps to add the TiKV or PD node. + +1. Configure the scale-out topology: + + > **Note:** + > + > * The port information is not required by default. + > * If multiple instances are deployed on a single machine, you need to allocate different ports for them. If the ports or directories have conflicts, you will receive a notification during deployment or scaling. + + Add the scale-out topology configuration in the `scale-out.yaml` file: + + {{< copyable "shell-regular" >}} + + ```shell + vi scale-out.yaml + ``` + + ``` + tidb_servers: + - host: 10.0.1.5 + ssh_port: 22 + port: 4000 + status_port: 10080 + ``` + + To view the whole configuration of the current cluster, run `tiup cluster edit-config `. The global configuration of `global` and `server_configs` also takes effect in `scale-out.yaml`. + + After the configuration, the current topology of the cluster is as follows: + + | Host IP | Service | + |:---|:----| + | 10.0.1.3 | TiDB + TiFlash | + | 10.0.1.4 | TiDB + PD | + | 10.0.1.5 | **TiDB** + TiKV + Monitor | + | 10.0.1.1 | TiKV | + | 10.0.1.2 | TiKV | + +2. Run the scale-out command: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster scale-out scale-out.yaml + ``` + + If you see the `Scaled cluster out successfully`, the scale-out operation is successfully completed. + +3. Check the cluster status: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster display + ``` + + Access the monitoring platform at using your browser to monitor the status of the cluster and the new node. + +## Scale out a TiFlash node + +If you want to add a TiFlash node to the `10.0.1.4` host, take the following steps. + +1. Add the node information to the `scale-out.yaml` file: + + Create the `scale-out.yaml` file to add the TiFlash node information. + + {{< copyable "" >}} + + ```ini + tiflash_servers: + - host: 10.0.1.4 + ``` + + Currently, you can only add IP but not domain name. + +2. Run the scale-out command: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster scale-out scale-out.yaml + ``` + +3. View the cluster status: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster display + ``` + + Access the monitoring platform at using your browser, and view the status of the cluster and the new node. + +## Scale in a TiDB/TiKV/PD node + +If you want to remove a TiKV node from the `10.0.1.5` host, take the following steps. + +> **Note:** +> +> You can take similar steps to remove the TiDB or PD node. + +1. View the node ID information: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster display + ``` + + ``` + Starting /root/.tiup/components/cluster/v0.4.6/cluster display + TiDB Cluster: + TiDB Version: v4.0.0-rc + ID Role Host Ports Status Data Dir Deploy Dir + -- ---- ---- ----- ------ -------- ---------- + 10.0.1.4:2379 pd 10.0.1.4 2379/2380 Healthy data/pd-2379 deploy/pd-2379 + 10.0.1.1:20160 tikv 10.0.1.1 20160/20180 Up data/tikv-20160 deploy/tikv-20160 + 10.0.1.2:20160 tikv 10.0.1.2 20160/20180 Up data/tikv-20160 deploy/tikv-20160 + 10.0.1.5:20160 tikv 10.0.1.5 20160/20180 Up data/tikv-20160 deploy/tikv-20160 + 10.0.1.3:4000 tidb 10.0.1.3 4000/10080 Up - deploy/tidb-4000 + 10.0.1.4:4000 tidb 10.0.1.4 4000/10080 Up - deploy/tidb-4000 + 10.0.1.5:4000 tidb 10.0.1.5 4000/10080 Up - deploy/tidb-4000 + 10.0.1.3:9000 tiflash 10.0.1.3 9000/8123/3930/20170/20292/8234 Up data/tiflash-9000 deploy/tiflash-9000 + 10.0.1.4:9000 tiflash 10.0.1.4 9000/8123/3930/20170/20292/8234 Up data/tiflash-9000 deploy/tiflash-9000 + 10.0.1.5:9290 prometheus 10.0.1.5 9290 Up data/prometheus-9290 deploy/prometheus-9290 + 10.0.1.5:3200 grafana 10.0.1.5 3200 Up - deploy/grafana-3200 + 10.0.1.5:9293 alertmanager 10.0.1.5 9293/9294 Up data/alertmanager-9293 deploy/alertmanager-9293 + ``` + +2. Run the scale-in command: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster scale-in --node 10.0.1.5:20160 + ``` + + The `--node` parameter is the ID of the node to be taken offline. + + If you see the `Scaled cluster in successfully`, the scale-in operation is successfully completed. + +3. Check the cluster status: + + The scale-in process takes some time. If the status of the node to be scaled in becomes `Tombstone`, that means the scale-in operation is successful. + + To check the scale-in status, run the following command: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster display + ``` + + The current topology is as follows: + + | Host IP | Service | + |:----|:----| + | 10.0.1.3 | TiDB + TiFlash | + | 10.0.1.4 | TiDB + PD + TiFlash | + | 10.0.1.5 | TiDB + Monitor **(TiKV is deleted)** | + | 10.0.1.1 | TiKV | + | 10.0.1.2 | TiKV | + + Access the monitoring platform at using your browser to monitor the status of the cluster. + +## Scale in a TiFlash node + +If you want to remove the TiFlash node from the `10.0.1.4` host, take the following steps. + +> **Note:** +> +> The scale-in process described in this section does not delete the data on the node that goes offline. If you need to bring the node back again, delete the data manually. + +1. Take the node offline: + + To take offline the node to be scaled in, refer to [Take a TiFlash node down](/reference/tiflash/maintain.md#take-a-tiflash-node-down). + +2. Check the node status: + + The scale-in process takes some time. + + You can use Grafana or pd-ctl to check whether the node has been successfully taken offline. + +3. Stop the TiFlash process: + + After the `store` corresponding to TiFlash disappears, or the `state_name` becomes `Tombstone`, execute the following command to stop the TiFlash process: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster scale-in --node 10.0.1.4:9000 + ``` + +4. View the cluster status: + + {{< copyable "shell-regular" >}} + + ```shell + tiup cluster display + ``` + + Access the monitoring platform at using your browser, and view the status of the cluster. diff --git a/reference/tiflash/deploy.md b/reference/tiflash/deploy.md index 1ae6a35a3ef31..837d53a6ed842 100644 --- a/reference/tiflash/deploy.md +++ b/reference/tiflash/deploy.md @@ -141,4 +141,4 @@ TiUP cluster is the deployment tool for TiDB 4.0 or later versions. It is recomm Currently, pd-ctl is not connected to TiUP cluster, you need to [manually download pd-ctl](https://download.pingcap.org/tidb-v4.0.0-rc-linux-amd64.tar.gz). -3. Refer to [Scale out a TiFlash node](/reference/tiflash/scale.md#scale-out-a-tiflash-node) and deploy TiFlash. +3. Refer to [Scale out a TiFlash node](/how-to/scale/with-tiup.md#2-scale-out-a-tiflash-node) and deploy TiFlash. diff --git a/reference/tiflash/maintain.md b/reference/tiflash/maintain.md index 60b582e742f4a..533923e18681e 100644 --- a/reference/tiflash/maintain.md +++ b/reference/tiflash/maintain.md @@ -32,7 +32,7 @@ There are two ways to check the TiFlash version: ## Take a TiFlash node down -Taking a TiFlash node down differs from [Scaling in a TiFlash node](/reference/tiflash/scale.md#scale-in-a-tiflash-node) in that the former doesn't remove the node in TiDB Ansible; instead, it just safely shuts down the TiFlash process. +Taking a TiFlash node down differs from [Scaling in a TiFlash node](/how-to/scale/with-tiup.md#scale-in-a-tiflash-node) in that the former doesn't remove the node in TiDB Ansible; instead, it just safely shuts down the TiFlash process. Follow the steps below to take a TiFlash node down: diff --git a/reference/tiflash/scale.md b/reference/tiflash/scale.md deleted file mode 100644 index f1fc3e79bc547..0000000000000 --- a/reference/tiflash/scale.md +++ /dev/null @@ -1,72 +0,0 @@ ---- -title: Scale the TiFlash Cluster -summary: Learn how to scale in and out nodes in the TiFlash cluster. -category: reference ---- - -# Scale the TiFlash Cluster - -This document describes how to scale in and out nodes in the TiFlash cluster. - -## Scale out a TiFlash node - -If you need to add a TiFlash node to the `172.19.0.104` host, take the following steps: - -1. Create the `scale-out.yaml` file to add the TiFlash node information: - - Currently, you can only add IP but not domain name. - - {{< copyable "" >}} - - ```ini - tiflash_servers: - - host: 172.19.0.104 - ``` - -2. Run the scale-out command: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster scale-out test scale-out.yaml - ``` - -3. View the cluster status: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster display test - ``` - -4. Access the monitoring platform using your browser, and view the status of the cluster and the new node. - -## Scale in a TiFlash node - -If you want to stop the TiFlash service on the `172.19.0.104` node, take the following steps: - -> **Note:** -> -> The offline process described in this section does not delete the data on the offline node. If you need to take the node online again, delete the data manually. - -1. Take down the node to be scaled in. See [Take a TiFlash node down](/reference/tiflash/maintain.md#take-a-tiflash-node-down) for details. - -2. Check whether the node has been offline successfully using Grafana or pd-ctl (the offline process takes some time). - -3. After the `store` corresponding to TiFlash disappears, or the `state_name` becomes `Tombstone`, execute the following command to shutdown the TiFlash process: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster scale-in test --node 172.19.0.104:9000 - ``` - -4. View the cluster status: - - {{< copyable "shell-regular" >}} - - ```shell - tiup cluster display test - ``` - -5. Access the monitoring platform using your browser, and view the status of the cluster.