diff --git a/TOC.md b/TOC.md index e006bb5eab728..5020e61f131d9 100644 --- a/TOC.md +++ b/TOC.md @@ -287,6 +287,9 @@ - [Error Handling](/tidb-binlog/handle-tidb-binlog-errors.md) - [FAQ](/tidb-binlog/tidb-binlog-faq.md) + Tools + - [Overview](/ecosystem-tool-user-guide.md) + - [Use Cases](/ecosystem-tool-user-case.md) + - [Download](/download-ecosystem-tools.md) - [Mydumper](/mydumper-overview.md) - [Syncer](/syncer-overview.md) - [Loader](/loader-overview.md) @@ -312,7 +315,6 @@ - [PD Recover](/pd-recover.md) - [TiKV Control](/tikv-control.md) - [TiDB Control](/tidb-control.md) - - [Download](/download-ecosystem-tools.md) + FAQs - [TiDB FAQs](/faq/tidb-faq.md) - [TiDB Lightning FAQs](/tidb-lightning/tidb-lightning-faq.md) diff --git a/ecosystem-tool-user-case.md b/ecosystem-tool-user-case.md new file mode 100644 index 0000000000000..953abac056df0 --- /dev/null +++ b/ecosystem-tool-user-case.md @@ -0,0 +1,39 @@ +--- +title: TiDB Ecosystem Tools Use Cases +summary: Learn the common use cases of TiDB ecosystem tools and how to choose the tools. +category: reference +--- + +# TiDB Ecosystem Tools Use Cases + +This document introduces the common use cases of TiDB ecosystem tools and how to choose the right tool for your scenario. + +## Import data from CSV to TiDB + +If you need to import the compatible CSV files exported by other tools to TiDB, use [TiDB Lightning](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md). + +## Import full data from MySQL/Aurora + +If you need to import full data from MySQL or Aurora, use [Dumpling](/export-or-backup-using-dumpling.md) first to export data as SQL dump files, and then use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) to import data into the TiDB cluster. + +## Migrate data from MySQL/Aurora + +If you need to migrate both full data and incremental data from MySQL/Aurora, use [TiDB Data Migration](https://pingcap.com/docs/tidb-data-migration/stable/overview/) (DM) to perform the full and incremental data migration. + +If the full data volume is large (at the TB level), you can first use [Dumpling](/export-or-backup-using-dumpling.md) and [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) to perform the full data migration, and then use DM to perform the incremental data migration. + +## Back up and restore TiDB cluster + +If you need to back up a TiDB cluster, use [Dumpling](/export-or-backup-using-dumpling.md). + +If you need to restore data to a TiDB cluster, use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md). + +## Migrate data from TiDB + +If you need to migrate data from a TiDB cluster to MySQL or to another TiDB cluster, use [Dumpling](/export-or-backup-using-dumpling.md) to export full data from TiDB as SQL dump files, and then use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) to import data to MySQL or another TiDB cluster. + +If you also need to migrate incremental data, use [TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md). + +## TiDB incremental data subscription + +If you need to subscribe to TiDB's incremental changes, use [TiDB Binlog](/tidb-binlog/binlog-slave-client.md). diff --git a/ecosystem-tool-user-guide.md b/ecosystem-tool-user-guide.md new file mode 100644 index 0000000000000..5ee85782efa9b --- /dev/null +++ b/ecosystem-tool-user-guide.md @@ -0,0 +1,81 @@ +--- +title: TiDB Ecosystem Tools Overview +category: reference +aliases: ['/docs/v2.1/reference/tools/user-guide/','/docs/v2.1/how-to/migrate/from-mysql/','/docs/v2.1/how-to/migrate/incrementally-from-mysql/','/docs/v2.1/how-to/migrate/overview/'] +--- + +# TiDB Ecosystem Tools Overview + +This document introduces the functionalities of TiDB ecosystem tools and their relationship. + +## Full data export + +[Dumpling](/export-or-backup-using-dumpling.md) is a tool for the logical full data export from MySQL or TiDB. + +The following are the basics of Dumpling: + +- Input: MySQL/TiDB cluster +- Output: SQL/CSV file +- Supported TiDB versions: all versions +- Kubernetes support: No + +## Full data import + +[TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) (Lightning) is a tool used for the full import of large amounts of data into a TiDB cluster. Currently, TiDB Lightning supports reading SQL dump exported via Dumpling or CSV data source. + +TiDB Lightning supports two modes: + +- `importer`: This mode uses tikv-importer as the backend, which is usually for importing a large amount of data (at the TB level). During the import, the cluster cannot provide services. +- `tidb`: This mode uses TiDB/MySQL as the backend, which is slower than the `importer` mode but can be performed online. It also supports importing data to MySQL. + +The following are the basics of TiDB Lightning: + +- Input data source: + - The output file of Dumpling + - Other compatible CSV file +- Supported TiDB versions: v2.1 or later +- Kubernetes support: Yes. See [Quickly restore data into a TiDB cluster in Kubernetes using TiDB Lightning](https://pingcap.com/docs/tidb-in-kubernetes/stable/restore-data-using-tidb-lightning/) for details. + +> **Note:** +> +> The Loader tool is no longer maintained. For scenarios related to Loader, it is recommended that you use the [`tidb` mode of TiDB Lighting](/tidb-lightning/tidb-lightning-overview.md) instead. + +## Backup and restore + +[Dumpling](/export-or-backup-using-dumpling.md) can be used to back up the TiDB cluster to SQL/CSV files. See [Full data export](#full-data-export). + +[TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) can be used to restore the SQL/CSV files exported by Dumpling to the TiDB cluster. See [Full data import](#full-data-import). + +## Incremental data replication + +[TiDB Binlog](/tidb-binlog/tidb-binlog-overview.md) is a tool that collects binlog for TiDB clusters and provides near real-time sync and backup. It can be used for incremental data replication between TiDB clusters, such as making a TiDB cluster the secondary cluster of the primary TiDB cluster. + +The following are the basics of TiDB Binlog: + +- Input: TiDB cluster +- Output: TiDB cluster, MySQL, Kafka or incremental backup files +- Supported TiDB versions: v2.1 or later +- Kubernetes support: Yes. See [TiDB Binlog Cluster Operations](https://pingcap.com/docs/tidb-in-kubernetes/stable/deploy-tidb-binlog/) and [TiDB Binlog Drainer Configurations in Kubernetes](https://pingcap.com/docs/tidb-in-kubernetes/stable/configure-tidb-binlog-drainer/) for details. + +## Data migration + +[TiDB Data Migration](https://pingcap.com/docs/tidb-data-migration/stable/) (DM) is an integrated data replication task management platform that supports the full data migration and the incremental data migration from MySQL/MariaDB to TiDB. + +The following are the basics of DM: + +- Input: MySQL/MariaDB +- Output: TiDB cluster +- Supported TiDB versions: all versions +- Kubernetes support: No, under development + +If the data volume is below the TB level, it is recommended to migrate data from MySQL/MariaDB to TiDB directly using DM. The migration process includes the full data import and export and the incremental data replication. + +If the data volume is at the TB level, take the following steps: + +1. Use [Dumpling](/export-or-backup-using-dumpling.md) to export the full data from MySQL/MariaDB. +2. Use [TiDB Lightning](/tidb-lightning/tidb-lightning-overview.md) to import the data exported in Step 1 to the TiDB cluster. +3. Use DM to migrate the incremental data from MySQL/MariaDB to TiDB. + +> **Note:** +> +> The Syncer tool is no longer maintained. For scenarios related to Syncer, it is recommended that you use DM's incremental task mode instead.