From 98813c008afa848226006b1f544663eabef7c458 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Wed, 14 Apr 2021 15:24:32 +0200 Subject: [PATCH 1/2] dumpling-overview: Suggest 256 MB or less as file size This is because lightning is [suggesting this](https://github.com/pingcap/tidb-lightning/blob/39dac927300b350f939da529f5328160d34d7727/lightning/mydump/region.go#L279-L284) for non-CSV files. --- dumpling-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dumpling-overview.md b/dumpling-overview.md index a7b388a55ee83..863eb9e7d7649 100644 --- a/dumpling-overview.md +++ b/dumpling-overview.md @@ -237,7 +237,7 @@ Examples: The exported file is stored in the `./export-` directory by default. Commonly used options are as follows: - `-o` is used to select the directory where the exported files are stored. -- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). +- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). It is advised to keep this 256 MB or less if you plan to use lightning to load this into a TiDB instance. - `-r` option is used to specify the maximum number of records (or the number of rows in the database) for a single file. When it is enabled, Dumpling enables concurrency in the table to improve the speed of exporting large tables. With the above options specified, Dumpling can have a higher degree of parallelism. From 6f2352874670925f83d839a5fc5f765d977ebf87 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Dani=C3=ABl=20van=20Eeden?= Date: Fri, 16 Apr 2021 08:28:17 +0200 Subject: [PATCH 2/2] Update dumpling-overview.md Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- dumpling-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dumpling-overview.md b/dumpling-overview.md index 863eb9e7d7649..d7440a2d6a936 100644 --- a/dumpling-overview.md +++ b/dumpling-overview.md @@ -237,7 +237,7 @@ Examples: The exported file is stored in the `./export-` directory by default. Commonly used options are as follows: - `-o` is used to select the directory where the exported files are stored. -- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). It is advised to keep this 256 MB or less if you plan to use lightning to load this into a TiDB instance. +- `-F` option is used to specify the maximum size of a single file (the unit here is `MiB`; inputs like `5GiB` or `8KB` are also acceptable). It is recommended to keep its value to 256 MiB or less if you plan to use TiDB Lightning to load this file into a TiDB instance. - `-r` option is used to specify the maximum number of records (or the number of rows in the database) for a single file. When it is enabled, Dumpling enables concurrency in the table to improve the speed of exporting large tables. With the above options specified, Dumpling can have a higher degree of parallelism.