From d78a6367e9d2d99b11dad4bee2538c9c4532c500 Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Tue, 16 Jun 2020 20:17:03 +0800 Subject: [PATCH 1/3] cherry pick #2774 to release-4.0 Signed-off-by: ti-srebot --- migration-overview.md | 64 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) create mode 100644 migration-overview.md diff --git a/migration-overview.md b/migration-overview.md new file mode 100644 index 0000000000000..9c962997d0d22 --- /dev/null +++ b/migration-overview.md @@ -0,0 +1,64 @@ +--- +title: Migration Overview +summary: This document describes how to migrate data from databases or data formats (CSV/SQL). +category: reference +--- + +# Migration Overview + +This document describes how to migrate data to TiDB, including migrating data from MySQL and from CSV/SQL files. + +## Migrate from MySQL to TiDB + +To migrate data from MySQL to TiDB, it is recommended to use one of the following methods: + +- [Use Mydumper and TiDB Lightning](#use-mydumper-and-tidb-lightning-full-data) to migrate full data. +- [Use TiDB Data Migration (DM)](#use-dm) to migrate full and incremental data. + +### Use Mydumper and TiDB Lightning (full data) + +#### Scenarios + +You can use Mydumper and TiDB Lightning to migrate full data when the data size is greater than 1 TB. If you need to replicate incremental data, it is recommended to [use DM](#use-dm) to create an incremental replication task. + +#### Migration method + +1. Use Mydumper to export the full MySQL data. +2. Use TiDB Lightning to import the full data to TiDB. For details, refer to [Migrate from Mydumper Files](/migrate-from-mysql-mydumper-files.md). + +### Use DM + +#### Scenarios + +You can use DM to migrate full MySQL data and to replicate incremental data. It is suggested that the size of the full data is less than 1 TB. Otherwise, it is recommended to use Mydumper and TiDB Lightning to import the full data, and then use DM to replicate the incremental data. + +#### Migration method + +For details, refer to [Migrate from MySQL (Amazon Aurora)](/migrate-from-aurora-mysql-database.md) + +## Migrate data from files to TiDB + +You can migrate data from CSV/SQL files to TiDB. + +### Migrate data from CSV files to TiDB + +#### Scenarios + +You can migrate data from heterogeneous databases that are not compatible with the MySQL protocol to TiDB. + +#### Migration method + +1. Export full data to CSV files. +2. Import CSV files to TiDB using one of the following methods: + + - Use TiDB Lightning. + + Its import speed is fast. It is recommended to use TiDB Lightning in the case of large amounts of data in CSV files. For details, refer to [TiDB Lightning CSV Support](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md). + + - Use the `LOAD DATA` statement. + + Execute the `LOAD DATA` statement in TiDB to import CSV files. This is more convenient, but if an error or interruption occurs during the import, manual intervention is required to check the consistency and integrity of the data. Therefore, it is **not recommended** to use this method in the production environment. For details, refer to [LOAD DATA](/sql-statements/sql-statement-load-data.md). + +### Migrate data from SQL files to TiDB + +Use Mydumper and TiDB Lightning to migrate data from SQL files to TiDB. For details, refer to [Use Mydumper and TiDB Lightning](#use-mydumper-and-tidb-lightning-full-data). From 8d15aea4b82ba3ce398a1824a6d3cbf3b1d8d57f Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Wed, 17 Jun 2020 16:30:50 +0800 Subject: [PATCH 2/3] Update TOC.md --- TOC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TOC.md b/TOC.md index c2f06821fefd2..3d278d777d08e 100644 --- a/TOC.md +++ b/TOC.md @@ -41,7 +41,7 @@ + [v3.0 TPC-C Performance Test](/benchmark/v3.0-performance-benchmarking-with-tpcc.md) + [Interaction Test on Online Workloads and `ADD INDEX`](/benchmark/online-workloads-and-add-index-operations.md) + Migrate - + [Migration Tools Guide](/ecosystem-tool-user-guide.md) + + [Migration Overview](/migration-overview.md) + Migrate from MySQL + [Migrate from Mydumper Files](/migrate-from-mysql-mydumper-files.md) + [Migrate from Amazon Aurora MySQL Using DM](/migrate-from-aurora-mysql-database.md) From f88cffd31c2824aad012e2fbf4ad06fb327e6d81 Mon Sep 17 00:00:00 2001 From: toutdesuite Date: Wed, 17 Jun 2020 16:40:16 +0800 Subject: [PATCH 3/3] Update TOC.md --- TOC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/TOC.md b/TOC.md index 3d278d777d08e..e1ae575a6d668 100644 --- a/TOC.md +++ b/TOC.md @@ -41,7 +41,7 @@ + [v3.0 TPC-C Performance Test](/benchmark/v3.0-performance-benchmarking-with-tpcc.md) + [Interaction Test on Online Workloads and `ADD INDEX`](/benchmark/online-workloads-and-add-index-operations.md) + Migrate - + [Migration Overview](/migration-overview.md) + + [Overview](/migration-overview.md) + Migrate from MySQL + [Migrate from Mydumper Files](/migrate-from-mysql-mydumper-files.md) + [Migrate from Amazon Aurora MySQL Using DM](/migrate-from-aurora-mysql-database.md)