-
Notifications
You must be signed in to change notification settings - Fork 710
how-to: add doc on how to scale with TiUP #2239
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
3b88faf
how-to: add doc on how to scale with TiUP
ran-huang 68f9d78
update anchor
ran-huang 0266ddc
Merge branch 'master' into scale-with-tiup
yikeke 9ec37fb
Apply suggestions from code review
ran-huang 0c93e77
remove numbers in heading
ran-huang 490f633
Merge branch 'master' into scale-with-tiup
yikeke 692d68d
update list format
ran-huang 063a174
Merge branch 'master' into scale-with-tiup
sre-bot File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,231 @@ | ||
| --- | ||
| title: Scale the TiDB Cluster Using TiUP | ||
| summary: | ||
| Category: how-to | ||
| --- | ||
|
|
||
| # Scale the TiDB Cluster Using TiUP | ||
|
|
||
| The capacity of a TiDB cluster can be increased or decreased without affecting the online services. | ||
|
|
||
| This document describes how to scale the TiDB, TiKV, PD or TiFlash nodes using TiUP. If you have not installed TiUP, refer to the steps in [Install TiUP on the Control Machine](/how-to/upgrade/using-tiup.md#install-tiup-on-the-control-machine) and import the cluster into TiUP before you scale the TiDB cluster. | ||
|
|
||
| To view the current cluster name list, run `tiup cluster list`. | ||
|
|
||
| For example, if the original topology of the cluster is as follows: | ||
|
|
||
| | Host IP | Service | | ||
| |:---|:----| | ||
| | 10.0.1.3 | TiDB + TiFlash | | ||
| | 10.0.1.4 | TiDB + PD | | ||
| | 10.0.1.5 | TiKV + Monitor | | ||
| | 10.0.1.1 | TiKV | | ||
| | 10.0.1.2 | TiKV | | ||
|
|
||
| ## Scale out a TiDB/TiKV/PD node | ||
|
|
||
| If you want to add a TiDB node to the `10.0.1.5` host, take the following steps. | ||
|
|
||
| > **Note:** | ||
| > | ||
| > You can take similar steps to add the TiKV or PD node. | ||
|
|
||
| 1. Configure the scale-out topology: | ||
|
|
||
| > **Note:** | ||
| > | ||
| > * The port information is not required by default. | ||
| > * If multiple instances are deployed on a single machine, you need to allocate different ports for them. If the ports or directories have conflicts, you will receive a notification during deployment or scaling. | ||
|
|
||
| Add the scale-out topology configuration in the `scale-out.yaml` file: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| vi scale-out.yaml | ||
| ``` | ||
|
|
||
| ``` | ||
| tidb_servers: | ||
| - host: 10.0.1.5 | ||
| ssh_port: 22 | ||
| port: 4000 | ||
| status_port: 10080 | ||
| ``` | ||
|
|
||
| To view the whole configuration of the current cluster, run `tiup cluster edit-config <cluster-name>`. The global configuration of `global` and `server_configs` also takes effect in `scale-out.yaml`. | ||
|
|
||
| After the configuration, the current topology of the cluster is as follows: | ||
|
|
||
| | Host IP | Service | | ||
| |:---|:----| | ||
| | 10.0.1.3 | TiDB + TiFlash | | ||
| | 10.0.1.4 | TiDB + PD | | ||
| | 10.0.1.5 | **TiDB** + TiKV + Monitor | | ||
| | 10.0.1.1 | TiKV | | ||
| | 10.0.1.2 | TiKV | | ||
|
|
||
| 2. Run the scale-out command: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster scale-out <cluster-name> scale-out.yaml | ||
| ``` | ||
|
|
||
| If you see the `Scaled cluster <cluster-name> out successfully`, the scale-out operation is successfully completed. | ||
|
|
||
| 3. Check the cluster status: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster display <cluster-name> | ||
| ``` | ||
|
|
||
| Access the monitoring platform at <http://10.0.1.5:3200> using your browser to monitor the status of the cluster and the new node. | ||
|
|
||
| ## Scale out a TiFlash node | ||
|
|
||
| If you want to add a TiFlash node to the `10.0.1.4` host, take the following steps. | ||
|
|
||
| 1. Add the node information to the `scale-out.yaml` file: | ||
|
|
||
| Create the `scale-out.yaml` file to add the TiFlash node information. | ||
|
|
||
| {{< copyable "" >}} | ||
|
|
||
| ```ini | ||
| tiflash_servers: | ||
| - host: 10.0.1.4 | ||
| ``` | ||
|
|
||
| Currently, you can only add IP but not domain name. | ||
|
|
||
| 2. Run the scale-out command: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster scale-out <cluster-name> scale-out.yaml | ||
| ``` | ||
|
|
||
| 3. View the cluster status: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster display <cluster-name> | ||
| ``` | ||
|
|
||
| Access the monitoring platform at <http://10.0.1.5:3200> using your browser, and view the status of the cluster and the new node. | ||
|
|
||
| ## Scale in a TiDB/TiKV/PD node | ||
|
|
||
| If you want to remove a TiKV node from the `10.0.1.5` host, take the following steps. | ||
|
|
||
| > **Note:** | ||
| > | ||
| > You can take similar steps to remove the TiDB or PD node. | ||
|
|
||
| 1. View the node ID information: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster display <cluster-name> | ||
| ``` | ||
|
|
||
| ``` | ||
| Starting /root/.tiup/components/cluster/v0.4.6/cluster display <cluster-name> | ||
| TiDB Cluster: <cluster-name> | ||
| TiDB Version: v4.0.0-rc | ||
| ID Role Host Ports Status Data Dir Deploy Dir | ||
| -- ---- ---- ----- ------ -------- ---------- | ||
| 10.0.1.4:2379 pd 10.0.1.4 2379/2380 Healthy data/pd-2379 deploy/pd-2379 | ||
| 10.0.1.1:20160 tikv 10.0.1.1 20160/20180 Up data/tikv-20160 deploy/tikv-20160 | ||
| 10.0.1.2:20160 tikv 10.0.1.2 20160/20180 Up data/tikv-20160 deploy/tikv-20160 | ||
| 10.0.1.5:20160 tikv 10.0.1.5 20160/20180 Up data/tikv-20160 deploy/tikv-20160 | ||
| 10.0.1.3:4000 tidb 10.0.1.3 4000/10080 Up - deploy/tidb-4000 | ||
| 10.0.1.4:4000 tidb 10.0.1.4 4000/10080 Up - deploy/tidb-4000 | ||
| 10.0.1.5:4000 tidb 10.0.1.5 4000/10080 Up - deploy/tidb-4000 | ||
| 10.0.1.3:9000 tiflash 10.0.1.3 9000/8123/3930/20170/20292/8234 Up data/tiflash-9000 deploy/tiflash-9000 | ||
| 10.0.1.4:9000 tiflash 10.0.1.4 9000/8123/3930/20170/20292/8234 Up data/tiflash-9000 deploy/tiflash-9000 | ||
| 10.0.1.5:9290 prometheus 10.0.1.5 9290 Up data/prometheus-9290 deploy/prometheus-9290 | ||
| 10.0.1.5:3200 grafana 10.0.1.5 3200 Up - deploy/grafana-3200 | ||
| 10.0.1.5:9293 alertmanager 10.0.1.5 9293/9294 Up data/alertmanager-9293 deploy/alertmanager-9293 | ||
| ``` | ||
|
|
||
| 2. Run the scale-in command: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster scale-in <cluster-name> --node 10.0.1.5:20160 | ||
| ``` | ||
|
|
||
| The `--node` parameter is the ID of the node to be taken offline. | ||
|
|
||
| If you see the `Scaled cluster <cluster-name> in successfully`, the scale-in operation is successfully completed. | ||
|
|
||
| 3. Check the cluster status: | ||
|
|
||
| The scale-in process takes some time. If the status of the node to be scaled in becomes `Tombstone`, that means the scale-in operation is successful. | ||
|
|
||
| To check the scale-in status, run the following command: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster display <cluster-name> | ||
| ``` | ||
|
|
||
| The current topology is as follows: | ||
|
|
||
| | Host IP | Service | | ||
| |:----|:----| | ||
| | 10.0.1.3 | TiDB + TiFlash | | ||
| | 10.0.1.4 | TiDB + PD + TiFlash | | ||
| | 10.0.1.5 | TiDB + Monitor **(TiKV is deleted)** | | ||
| | 10.0.1.1 | TiKV | | ||
| | 10.0.1.2 | TiKV | | ||
|
|
||
| Access the monitoring platform at <http://10.0.1.5:3200> using your browser to monitor the status of the cluster. | ||
|
|
||
| ## Scale in a TiFlash node | ||
|
|
||
| If you want to remove the TiFlash node from the `10.0.1.4` host, take the following steps. | ||
|
|
||
| > **Note:** | ||
| > | ||
| > The scale-in process described in this section does not delete the data on the node that goes offline. If you need to bring the node back again, delete the data manually. | ||
|
|
||
| 1. Take the node offline: | ||
|
|
||
| To take offline the node to be scaled in, refer to [Take a TiFlash node down](/reference/tiflash/maintain.md#take-a-tiflash-node-down). | ||
|
|
||
| 2. Check the node status: | ||
|
|
||
| The scale-in process takes some time. | ||
|
|
||
| You can use Grafana or pd-ctl to check whether the node has been successfully taken offline. | ||
|
|
||
| 3. Stop the TiFlash process: | ||
|
|
||
| After the `store` corresponding to TiFlash disappears, or the `state_name` becomes `Tombstone`, execute the following command to stop the TiFlash process: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster scale-in <cluster-name> --node 10.0.1.4:9000 | ||
| ``` | ||
|
|
||
| 4. View the cluster status: | ||
|
|
||
| {{< copyable "shell-regular" >}} | ||
|
|
||
| ```shell | ||
| tiup cluster display <cluster-name> | ||
| ``` | ||
|
|
||
| Access the monitoring platform at <http://10.0.1.5:3200> using your browser, and view the status of the cluster. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
summary: xxx
category: how-to
aliases: ['/docs/stable/how-to/scale/with-tiup/']
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@ran-huang Please open a PR to fix the meta info, and we can merge the PR after the doc special week.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK