From 5afb7ba3fb42538314bce6e779c15492269314f6 Mon Sep 17 00:00:00 2001 From: Keke Yi <40977455+yikeke@users.noreply.github.com> Date: Mon, 6 Jul 2020 10:45:53 +0800 Subject: [PATCH 1/2] cherry pick #3000 to release-2.1 Signed-off-by: ti-srebot --- .markdownlint.yaml | 2 +- deploy-test-cluster-using-docker-compose.md | 185 ++++++++++++++++++++ test-deployment-using-docker.md | 8 + 3 files changed, 194 insertions(+), 1 deletion(-) create mode 100644 deploy-test-cluster-using-docker-compose.md diff --git a/.markdownlint.yaml b/.markdownlint.yaml index 178e1f7fd65b0..4ee86212623c4 100644 --- a/.markdownlint.yaml +++ b/.markdownlint.yaml @@ -1,4 +1,4 @@ -comment: 'Rules for PingCAP docs' +comment: 'Markdownlint Rules for PingCAP docs' # disable all by default default: false diff --git a/deploy-test-cluster-using-docker-compose.md b/deploy-test-cluster-using-docker-compose.md new file mode 100644 index 0000000000000..2b17f671aeb54 --- /dev/null +++ b/deploy-test-cluster-using-docker-compose.md @@ -0,0 +1,185 @@ +--- +title: TiDB Docker Compose Deployment +summary: Use Docker Compose to quickly deploy a TiDB testing cluster. +category: how-to +aliases: ['/docs/dev/deploy-test-cluster-using-docker-compose/','/docs/dev/how-to/get-started/deploy-tidb-from-docker-compose/'] +--- + +# TiDB Docker Compose Deployment + +> **Warning:** +> +> The Docker Compose deployment method provided in this document is no longer maintained. If you want to test TiDB, it is recommended to refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md) for deployment. For **production environment**, **do not use** Docker Compose for deployment, but [deploy TiDB with TiUP](/production-deployment-using-tiup.md) or [TiDB Operator in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/deploy-tidb-operator). + +This document describes how to quickly deploy a TiDB testing cluster with a single command using [Docker Compose](https://docs.docker.com/compose/overview). + +With Docker Compose, you can use a YAML file to configure application services in multiple containers. Then, with a single command, you can create and start all the services from your configuration. + +## Prerequisites + +Make sure you have installed the following items on your machine: + +- [Git](https://git-scm.com/downloads) +- [Docker Compose](https://docs.docker.com/compose/install/) +- [MySQL Client](https://dev.mysql.com/downloads/mysql/) + +## Deploy TiDB using Docker Compose + +1. Download `tidb-docker-compose`. + + ```bash + git clone https://github.com/pingcap/tidb-docker-compose.git + ``` + +2. Change the directory to tidb-docker-compose and get the latest TiDB Docker Images: + + ```bash + cd tidb-docker-compose && docker-compose pull + ``` + +3. Start the TiDB cluster: + + ```bash + docker-compose up -d + ``` + +4. Use the MySQL client to connect to TiDB to read and write data: + + ``` + mysql -h 127.0.0.1 -P 4000 -u root + ``` + +## Monitor the cluster + +After you have successfully deployed a TiDB cluster, you can now monitor the TiDB cluster using one of the following methods: + +- Use Grafana to view the status of the cluster via [http://localhost:3000](http://localhost:3000) with the default account name and password: `admin` and `admin`. +- Use [TiDB-Vision](https://github.com/pingcap/tidb-vision), a cluster visualization tool, to see data transfer and load-balancing inside your cluster via [http://localhost:8010](http://localhost:8010). + +## Customize the cluster + +After the deployment is completed, the following components are deployed by default: + +- 3 PD instances, 3 TiKV instances, 1 TiDB instance +- Monitoring components: Prometheus, Pushgateway, Grafana +- Data visualization component: tidb-vision + +To customize the cluster, you can edit the `docker-compose.yml` file directly. It is recommended to generate `docker-compose.yml` using the [Helm](https://helm.sh) template engine, because manual editing is tedious and error-prone. + +1. Install Helm. + + [Helm](https://helm.sh) can be used as a template rendering engine. Make sure your Helm version >= 2.9.0 and < 3.0.0. + To use Helm, you only need to download its binary file: + + ```bash + curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash + ``` + + For macOS, you can also install Helm using the following command in Homebrew: + + ```bash + brew install helm@2 + brew link --force helm@2 + ``` + +2. Download `tidb-docker-compose`. + + ```bash + git clone https://github.com/pingcap/tidb-docker-compose.git + ``` + +3. Customize the cluster. + + ```bash + cd tidb-docker-compose + vim compose/values.yaml # custom the cluster size, docker image, port mapping and so on + ``` + + You can modify the configuration in `values.yaml`, such as the cluster size, TiDB image version, and so on. + + [tidb-vision](https://github.com/pingcap/tidb-vision) is the data visualization interface of the TiDB cluster, used to visually display the PD scheduling on TiKV data. If you do not need this component, comment out the `tidbVision` section. + + For PD, TiKV, TiDB and tidb-vision, you can build Docker images from GitHub source code or local files for development and testing. + + - To build PD, TiKV or TiDB images from the locally compiled binary file, you need to comment out the `image` field and copy the compiled binary file to the corresponding `pd/bin/pd-server`, `tikv/bin/tikv-server`, `tidb/bin/tidb-server`. + - To build the tidb-vision image from local, you need to comment out the `image` field and copy the tidb-vision project to `tidb-vision/tidb-vision`. + +4. Generate the `docker-compose.yml` file. + + ```bash + helm template compose > generated-docker-compose.yml + ``` + +5. Create and start the cluster using the generated `docker-compose.yml` file. + + ```bash + docker-compose -f generated-docker-compose.yml pull # Get the latest Docker images + docker-compose -f generated-docker-compose.yml up -d + ``` + +6. Access the cluster. + + ```bash + mysql -h 127.0.0.1 -P 4000 -u root + ``` + + Access the Grafana monitoring interface: + + - Default address: + - Default account name: admin + - Default password: admin + + If tidb-vision is enabled, you can access the cluster data visualization interface: . + +## Access the Spark shell and load TiSpark + +Insert some sample data to the TiDB cluster: + +```bash +$ docker-compose exec tispark-master bash +$ cd /opt/spark/data/tispark-sample-data +$ mysql -h tidb -P 4000 -u root < dss.ddl +``` + +After the sample data is loaded into the TiDB cluster, you can access the Spark shell using `docker-compose exec tispark-master /opt/spark/bin/spark-shell`. + +```bash +$ docker-compose exec tispark-master /opt/spark/bin/spark-shell +... +Spark context available as 'sc' (master = local[*], app id = local-1527045927617). +Spark session available as 'spark'. +Welcome to + ____ __ + / __/__ ___ _____/ /__ + _\ \/ _ \/ _ `/ __/ '_/ + /___/ .__/\_,_/_/ /_/\_\ version 2.1.1 + /_/ + +Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_172) +Type in expressions to have them evaluated. +Type :help for more information. + +scala> import org.apache.spark.sql.TiContext +... +scala> val ti = new TiContext(spark) +... +scala> ti.tidbMapDatabase("TPCH_001") +... +scala> spark.sql("select count(*) from lineitem").show ++--------+ +|count(1)| ++--------+ +| 60175| ++--------+ +``` + +You can also access Spark with Python or R using the following commands: + +``` +docker-compose exec tispark-master /opt/spark/bin/pyspark +docker-compose exec tispark-master /opt/spark/bin/sparkR +``` + +For more details about TiSpark, see [TiSpark Quick Start Guide](/get-started-with-tispark.md). + +Here is [a 5-minute tutorial](https://pingcap.com/blog/how_to_spin_up_an_htap_database_in_5_minutes_with_tidb_tispark/) for macOS users that shows how to spin up a standard TiDB cluster using Docker Compose on your local computer. diff --git a/test-deployment-using-docker.md b/test-deployment-using-docker.md index 321c18baf5aba..f8fb1f39b4f12 100644 --- a/test-deployment-using-docker.md +++ b/test-deployment-using-docker.md @@ -7,7 +7,15 @@ aliases: ['/docs/v2.1/test-deployment-using-docker/','/docs/v2.1/how-to/deploy/o # Deploy TiDB Using Docker +<<<<<<< HEAD This page shows you how to manually deploy a multi-node TiDB cluster on multiple machines using Docker. +======= +> **Warning:** +> +> The Docker deployment method provided in this document is no longer maintained. If you want to test TiDB, it is recommended to refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md) for deployment. For **production environment**, **do not use** Docker Compose for deployment, but [deploy TiDB with TiUP](/production-deployment-using-tiup.md) or [TiDB Operator in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/deploy-tidb-operator). + +This document shows you how to manually deploy a multi-node TiDB cluster on multiple machines using Docker. +>>>>>>> 83ba8b0... add deprecated warnings for outdated deployment methods (#3000) To learn more, see [TiDB architecture](/architecture.md) and [Software and Hardware Recommendations](/hardware-and-software-requirements.md). From a3f0773d8dbe92ab9a2a7cb75733721b785de69e Mon Sep 17 00:00:00 2001 From: yikeke Date: Mon, 6 Jul 2020 10:55:03 +0800 Subject: [PATCH 2/2] resolve conflicts --- deploy-test-cluster-using-docker-compose.md | 185 -------------------- test-deployment-using-docker.md | 6 +- 2 files changed, 1 insertion(+), 190 deletions(-) delete mode 100644 deploy-test-cluster-using-docker-compose.md diff --git a/deploy-test-cluster-using-docker-compose.md b/deploy-test-cluster-using-docker-compose.md deleted file mode 100644 index 2b17f671aeb54..0000000000000 --- a/deploy-test-cluster-using-docker-compose.md +++ /dev/null @@ -1,185 +0,0 @@ ---- -title: TiDB Docker Compose Deployment -summary: Use Docker Compose to quickly deploy a TiDB testing cluster. -category: how-to -aliases: ['/docs/dev/deploy-test-cluster-using-docker-compose/','/docs/dev/how-to/get-started/deploy-tidb-from-docker-compose/'] ---- - -# TiDB Docker Compose Deployment - -> **Warning:** -> -> The Docker Compose deployment method provided in this document is no longer maintained. If you want to test TiDB, it is recommended to refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md) for deployment. For **production environment**, **do not use** Docker Compose for deployment, but [deploy TiDB with TiUP](/production-deployment-using-tiup.md) or [TiDB Operator in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/deploy-tidb-operator). - -This document describes how to quickly deploy a TiDB testing cluster with a single command using [Docker Compose](https://docs.docker.com/compose/overview). - -With Docker Compose, you can use a YAML file to configure application services in multiple containers. Then, with a single command, you can create and start all the services from your configuration. - -## Prerequisites - -Make sure you have installed the following items on your machine: - -- [Git](https://git-scm.com/downloads) -- [Docker Compose](https://docs.docker.com/compose/install/) -- [MySQL Client](https://dev.mysql.com/downloads/mysql/) - -## Deploy TiDB using Docker Compose - -1. Download `tidb-docker-compose`. - - ```bash - git clone https://github.com/pingcap/tidb-docker-compose.git - ``` - -2. Change the directory to tidb-docker-compose and get the latest TiDB Docker Images: - - ```bash - cd tidb-docker-compose && docker-compose pull - ``` - -3. Start the TiDB cluster: - - ```bash - docker-compose up -d - ``` - -4. Use the MySQL client to connect to TiDB to read and write data: - - ``` - mysql -h 127.0.0.1 -P 4000 -u root - ``` - -## Monitor the cluster - -After you have successfully deployed a TiDB cluster, you can now monitor the TiDB cluster using one of the following methods: - -- Use Grafana to view the status of the cluster via [http://localhost:3000](http://localhost:3000) with the default account name and password: `admin` and `admin`. -- Use [TiDB-Vision](https://github.com/pingcap/tidb-vision), a cluster visualization tool, to see data transfer and load-balancing inside your cluster via [http://localhost:8010](http://localhost:8010). - -## Customize the cluster - -After the deployment is completed, the following components are deployed by default: - -- 3 PD instances, 3 TiKV instances, 1 TiDB instance -- Monitoring components: Prometheus, Pushgateway, Grafana -- Data visualization component: tidb-vision - -To customize the cluster, you can edit the `docker-compose.yml` file directly. It is recommended to generate `docker-compose.yml` using the [Helm](https://helm.sh) template engine, because manual editing is tedious and error-prone. - -1. Install Helm. - - [Helm](https://helm.sh) can be used as a template rendering engine. Make sure your Helm version >= 2.9.0 and < 3.0.0. - To use Helm, you only need to download its binary file: - - ```bash - curl https://raw.githubusercontent.com/kubernetes/helm/master/scripts/get | bash - ``` - - For macOS, you can also install Helm using the following command in Homebrew: - - ```bash - brew install helm@2 - brew link --force helm@2 - ``` - -2. Download `tidb-docker-compose`. - - ```bash - git clone https://github.com/pingcap/tidb-docker-compose.git - ``` - -3. Customize the cluster. - - ```bash - cd tidb-docker-compose - vim compose/values.yaml # custom the cluster size, docker image, port mapping and so on - ``` - - You can modify the configuration in `values.yaml`, such as the cluster size, TiDB image version, and so on. - - [tidb-vision](https://github.com/pingcap/tidb-vision) is the data visualization interface of the TiDB cluster, used to visually display the PD scheduling on TiKV data. If you do not need this component, comment out the `tidbVision` section. - - For PD, TiKV, TiDB and tidb-vision, you can build Docker images from GitHub source code or local files for development and testing. - - - To build PD, TiKV or TiDB images from the locally compiled binary file, you need to comment out the `image` field and copy the compiled binary file to the corresponding `pd/bin/pd-server`, `tikv/bin/tikv-server`, `tidb/bin/tidb-server`. - - To build the tidb-vision image from local, you need to comment out the `image` field and copy the tidb-vision project to `tidb-vision/tidb-vision`. - -4. Generate the `docker-compose.yml` file. - - ```bash - helm template compose > generated-docker-compose.yml - ``` - -5. Create and start the cluster using the generated `docker-compose.yml` file. - - ```bash - docker-compose -f generated-docker-compose.yml pull # Get the latest Docker images - docker-compose -f generated-docker-compose.yml up -d - ``` - -6. Access the cluster. - - ```bash - mysql -h 127.0.0.1 -P 4000 -u root - ``` - - Access the Grafana monitoring interface: - - - Default address: - - Default account name: admin - - Default password: admin - - If tidb-vision is enabled, you can access the cluster data visualization interface: . - -## Access the Spark shell and load TiSpark - -Insert some sample data to the TiDB cluster: - -```bash -$ docker-compose exec tispark-master bash -$ cd /opt/spark/data/tispark-sample-data -$ mysql -h tidb -P 4000 -u root < dss.ddl -``` - -After the sample data is loaded into the TiDB cluster, you can access the Spark shell using `docker-compose exec tispark-master /opt/spark/bin/spark-shell`. - -```bash -$ docker-compose exec tispark-master /opt/spark/bin/spark-shell -... -Spark context available as 'sc' (master = local[*], app id = local-1527045927617). -Spark session available as 'spark'. -Welcome to - ____ __ - / __/__ ___ _____/ /__ - _\ \/ _ \/ _ `/ __/ '_/ - /___/ .__/\_,_/_/ /_/\_\ version 2.1.1 - /_/ - -Using Scala version 2.11.8 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_172) -Type in expressions to have them evaluated. -Type :help for more information. - -scala> import org.apache.spark.sql.TiContext -... -scala> val ti = new TiContext(spark) -... -scala> ti.tidbMapDatabase("TPCH_001") -... -scala> spark.sql("select count(*) from lineitem").show -+--------+ -|count(1)| -+--------+ -| 60175| -+--------+ -``` - -You can also access Spark with Python or R using the following commands: - -``` -docker-compose exec tispark-master /opt/spark/bin/pyspark -docker-compose exec tispark-master /opt/spark/bin/sparkR -``` - -For more details about TiSpark, see [TiSpark Quick Start Guide](/get-started-with-tispark.md). - -Here is [a 5-minute tutorial](https://pingcap.com/blog/how_to_spin_up_an_htap_database_in_5_minutes_with_tidb_tispark/) for macOS users that shows how to spin up a standard TiDB cluster using Docker Compose on your local computer. diff --git a/test-deployment-using-docker.md b/test-deployment-using-docker.md index f8fb1f39b4f12..a9a361d77586a 100644 --- a/test-deployment-using-docker.md +++ b/test-deployment-using-docker.md @@ -7,15 +7,11 @@ aliases: ['/docs/v2.1/test-deployment-using-docker/','/docs/v2.1/how-to/deploy/o # Deploy TiDB Using Docker -<<<<<<< HEAD -This page shows you how to manually deploy a multi-node TiDB cluster on multiple machines using Docker. -======= > **Warning:** > -> The Docker deployment method provided in this document is no longer maintained. If you want to test TiDB, it is recommended to refer to [Quick Start Guide for the TiDB Database Platform](/quick-start-with-tidb.md) for deployment. For **production environment**, **do not use** Docker Compose for deployment, but [deploy TiDB with TiUP](/production-deployment-using-tiup.md) or [TiDB Operator in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/v1.1/deploy-tidb-operator). +> The Docker deployment method provided in this document is no longer maintained. If you want to test TiDB, it is recommended to refer to [Quick Start Guide for the TiDB Database Platform](https://docs.pingcap.com/tidb/dev/quick-start-with-tidb) for deployment. For **production environment**, **do not use** Docker for deployment, but [deploy TiDB with TiDB Ansible](/online-deployment-using-ansible.md) or [TiDB Operator in Kubernetes](https://docs.pingcap.com/tidb-in-kubernetes/v1.0/deploy-tidb-operator). This document shows you how to manually deploy a multi-node TiDB cluster on multiple machines using Docker. ->>>>>>> 83ba8b0... add deprecated warnings for outdated deployment methods (#3000) To learn more, see [TiDB architecture](/architecture.md) and [Software and Hardware Recommendations](/hardware-and-software-requirements.md).