loader: move loader faq to loader overview#3270
Merged
Merged
Conversation
TomShawn
added
translation/from-docs-cn
ran-huang
reviewed
Jul 16, 2020
|
|
||
| ### Reasons | ||
|
|
||
| * Both MySQL client and MySQL/TiDB Server have `max_allowed_packet` quotas. If any of the `max_allowed_packet` quotas is violated, the client receives a corresponding error message. Currently, the latest version of Syncer, Loader, DM, and TiDB Server all have a default `max_allowed_packet` quota of `64M`. |
Contributor
There was a problem hiding this comment.
Suggested change
| * Both MySQL client and MySQL/TiDB Server have `max_allowed_packet` quotas. If any of the `max_allowed_packet` quotas is violated, the client receives a corresponding error message. Currently, the latest version of Syncer, Loader, DM, and TiDB Server all have a default `max_allowed_packet` quota of `64M`. | |
| * Both MySQL client and MySQL/TiDB Server have `max_allowed_packet` quotas. If any of the `max_allowed_packet` quotas is violated, the client receives a corresponding error message. Currently, the latest version of Loader and TiDB Server all have a default `max_allowed_packet` quota of `64M`. |
|
|
||
| * Both MySQL client and MySQL/TiDB Server have `max_allowed_packet` quotas. If any of the `max_allowed_packet` quotas is violated, the client receives a corresponding error message. Currently, the latest version of Syncer, Loader, DM, and TiDB Server all have a default `max_allowed_packet` quota of `64M`. | ||
| * Please use the latest version, or the latest stable version of the tool. See the [download page](/download-ecosystem-tools.md). | ||
| * The full data import processing module in Loader or DM does not support splitting `dump sqls` files. |
Contributor
There was a problem hiding this comment.
A large chunk of information is missing.
* Loader 的全量数据导入处理模块不支持对 dump sqls 文件进行切分，原因是 Mydumper 采用了最简单的编码实现，正如 Mydumper 代码注释 `/* Poor man's data dump code */` 所言。如果在 Loader 实现文件切分，那么需要在 `TiDB parser` 基础上实现一个完备的解析器才能正确的处理数据切分，但是随之会带来以下的问题：
* 工作量大
* 复杂度高，不容易保证正确性
* 性能的极大降低
|
|
||
| ``` | ||
| Row bigger than statement_size for xxx | ||
| `` |
|
|
||
| * If a single row of a wide table exceeds 64M, you need to modify and enable the following two configurations: | ||
| * Execute `set @@global.max_allowed_packet=134217728` (`134217728 = 128M`) in TiDB Server. | ||
| * Add the configuration like `max-allowed-packet=128M` to the Loader configuration file or DB configuration in DM task configuration file, and then restart the progress or task. |
Contributor
There was a problem hiding this comment.
Suggested change
| * Add the configuration like `max-allowed-packet=128M` to the Loader configuration file or DB configuration in DM task configuration file, and then restart the progress or task. | |
| * Add `max-allowed-packet=128M` to db configuration in the Loader configuration file according to your situation, and then restart the progress or task. |
Contributor
Author
|
@ran-huang PTAL again, thanks! |
ran-huang
reviewed
Jul 16, 2020
ran-huang
approved these changes
Jul 16, 2020
Contributor
Author
|
@WangXiangUSTC PTAL |
ti-srebot
approved these changes
Jul 22, 2020
ti-srebot
pushed a commit
to ti-srebot/docs
that referenced
this pull request
Jul 22, 2020
Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
5 tasks
Contributor
|
cherry pick to release-4.0 in PR #3364 |
9 tasks
This was referenced Aug 31, 2020
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is changed, added or deleted? (Required)
get startedfor tidb-binlogloader-overview.mdWhich TiDB version(s) do your changes apply to? (Required)
What is the related PR or file link(s)?