-
Notifications
You must be signed in to change notification settings - Fork 710
Data migration: Add migration-overview.md (#2774) #2915
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
ti-srebot
merged 4 commits into
pingcap:release-4.0
from
ti-srebot:release-4.0-48dd73b180ff
Jun 17, 2020
Merged
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| --- | ||
| title: Migration Overview | ||
| summary: This document describes how to migrate data from databases or data formats (CSV/SQL). | ||
| category: reference | ||
| --- | ||
|
|
||
| # Migration Overview | ||
|
|
||
| This document describes how to migrate data to TiDB, including migrating data from MySQL and from CSV/SQL files. | ||
|
|
||
| ## Migrate from MySQL to TiDB | ||
|
|
||
| To migrate data from MySQL to TiDB, it is recommended to use one of the following methods: | ||
|
|
||
| - [Use Mydumper and TiDB Lightning](#use-mydumper-and-tidb-lightning-full-data) to migrate full data. | ||
| - [Use TiDB Data Migration (DM)](#use-dm) to migrate full and incremental data. | ||
|
|
||
| ### Use Mydumper and TiDB Lightning (full data) | ||
|
|
||
| #### Scenarios | ||
|
|
||
| You can use Mydumper and TiDB Lightning to migrate full data when the data size is greater than 1 TB. If you need to replicate incremental data, it is recommended to [use DM](#use-dm) to create an incremental replication task. | ||
|
|
||
| #### Migration method | ||
|
|
||
| 1. Use Mydumper to export the full MySQL data. | ||
| 2. Use TiDB Lightning to import the full data to TiDB. For details, refer to [Migrate from Mydumper Files](/migrate-from-mysql-mydumper-files.md). | ||
|
|
||
| ### Use DM | ||
|
|
||
| #### Scenarios | ||
|
|
||
| You can use DM to migrate full MySQL data and to replicate incremental data. It is suggested that the size of the full data is less than 1 TB. Otherwise, it is recommended to use Mydumper and TiDB Lightning to import the full data, and then use DM to replicate the incremental data. | ||
|
|
||
| #### Migration method | ||
|
|
||
| For details, refer to [Migrate from MySQL (Amazon Aurora)](/migrate-from-aurora-mysql-database.md) | ||
|
|
||
| ## Migrate data from files to TiDB | ||
|
|
||
| You can migrate data from CSV/SQL files to TiDB. | ||
|
|
||
| ### Migrate data from CSV files to TiDB | ||
|
|
||
| #### Scenarios | ||
|
|
||
| You can migrate data from heterogeneous databases that are not compatible with the MySQL protocol to TiDB. | ||
|
|
||
| #### Migration method | ||
|
|
||
| 1. Export full data to CSV files. | ||
| 2. Import CSV files to TiDB using one of the following methods: | ||
|
|
||
| - Use TiDB Lightning. | ||
|
|
||
| Its import speed is fast. It is recommended to use TiDB Lightning in the case of large amounts of data in CSV files. For details, refer to [TiDB Lightning CSV Support](/tidb-lightning/migrate-from-csv-using-tidb-lightning.md). | ||
|
|
||
| - Use the `LOAD DATA` statement. | ||
|
|
||
| Execute the `LOAD DATA` statement in TiDB to import CSV files. This is more convenient, but if an error or interruption occurs during the import, manual intervention is required to check the consistency and integrity of the data. Therefore, it is **not recommended** to use this method in the production environment. For details, refer to [LOAD DATA](/sql-statements/sql-statement-load-data.md). | ||
|
|
||
| ### Migrate data from SQL files to TiDB | ||
|
|
||
| Use Mydumper and TiDB Lightning to migrate data from SQL files to TiDB. For details, refer to [Use Mydumper and TiDB Lightning](#use-mydumper-and-tidb-lightning-full-data). | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.