From cc45a9a53a820c20f2661cba1324aee86cc63f0a Mon Sep 17 00:00:00 2001 From: JoyinQin <56883733+Joyinqin@users.noreply.github.com> Date: Sun, 6 Sep 2020 15:54:43 +0800 Subject: [PATCH 1/5] br: add use constraint with CDC and drainer --- br/backup-and-restore-faq.md | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/br/backup-and-restore-faq.md b/br/backup-and-restore-faq.md index 1e157d2c91b66..73af75f05da9b 100644 --- a/br/backup-and-restore-faq.md +++ b/br/backup-and-restore-faq.md @@ -41,17 +41,17 @@ Running BR with the root access might fail due to the disk permission, because t ## What should I do to handle the `Io(Os...)` error? Almost all of these problems are system call errors that occur when TiKV writes data to the disk. You can check the mounting method and the file system of the backup directory, and try to back up data to another folder or another hard disk. -  + For example, you might encounter the `Code: 22(invalid argument)` error when backing up data to the network disk built by `samba`. ## What should I do to handle the `rpc error: code = Unavailable desc =...` error occurred in BR? - + This error might occur when the capacity of the cluster to restore (using BR) is insufficient. You can further confirm the cause by checking the monitoring metrics of this cluster or the TiKV log. To handle this issue, you can try to scale out the cluster resources, reduce the concurrency during restore, and enable the `RATE_LIMIT` option. ## Where are the backed up files stored when I use `local` storage? -  + When you use `local` storage, `backupmeta` is generated on the node where BR is running, and backup files are generated on the Leader nodes of each Region. ## How about the size of the backup data? Are there replicas of the backup? @@ -59,3 +59,11 @@ When you use `local` storage, `backupmeta` is generated on the node where BR is During data backup, backup files are generated on the Leader nodes of each Region. The size of the backup is equal to the data size, with no redundant replicas. Therefore, the total data size is approximately the total number of TiKV data divided by the number of replicas. However, if you want to restore data from local storage, the number of replicas is equal to that of the TiKV nodes, because each TiKV must have access to all backup files. + +## What should I pay attention to when BR restores data to the upstream clusters of TiCDC/Drainer? + ++ **The data restored by BR cannot be replicate to the downstream**. Because BR directly imports SST files, and downstream clusters currently cannot obtain upstream SST files. + ++ Restored data that cannot be replicated to downstream might cause TiCDC/Drainer to abnormally execute DDL; therefore, if you must perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored by BR to the TiCDC/Drainer block list. + +You can use [`filter.rules`](https://github.com/pingcap/ticdc/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) item to complete the configuration of TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) item to complete the configuration of Drainer. \ No newline at end of file From d0f6d366c2eac714afb28341b5414179fe5a8222 Mon Sep 17 00:00:00 2001 From: JoyinQin <56883733+Joyinqin@users.noreply.github.com> Date: Sun, 6 Sep 2020 16:02:33 +0800 Subject: [PATCH 2/5] fix errors --- br/backup-and-restore-faq.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/br/backup-and-restore-faq.md b/br/backup-and-restore-faq.md index 73af75f05da9b..c70e3f0b34670 100644 --- a/br/backup-and-restore-faq.md +++ b/br/backup-and-restore-faq.md @@ -60,9 +60,9 @@ During data backup, backup files are generated on the Leader nodes of each Regio However, if you want to restore data from local storage, the number of replicas is equal to that of the TiKV nodes, because each TiKV must have access to all backup files. -## What should I pay attention to when BR restores data to the upstream clusters of TiCDC/Drainer? +## What should I pay attention to when BR restores data to the upstream cluster of TiCDC/Drainer? -+ **The data restored by BR cannot be replicate to the downstream**. Because BR directly imports SST files, and downstream clusters currently cannot obtain upstream SST files. ++ **The data restored by BR cannot be replicated to the downstream**. Because BR directly imports SST files, and the downstream cluster currently cannot obtain upstream SST files. + Restored data that cannot be replicated to downstream might cause TiCDC/Drainer to abnormally execute DDL; therefore, if you must perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored by BR to the TiCDC/Drainer block list. From f069305b0419f4b01d9f8a0a9ec62c27239d9d57 Mon Sep 17 00:00:00 2001 From: JoyinQ <56883733+Joyinqin@users.noreply.github.com> Date: Mon, 7 Sep 2020 19:29:34 +0800 Subject: [PATCH 3/5] Apply suggestions from code review Co-authored-by: TomShawn <41534398+TomShawn@users.noreply.github.com> --- br/backup-and-restore-faq.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/br/backup-and-restore-faq.md b/br/backup-and-restore-faq.md index c70e3f0b34670..523f042b7200d 100644 --- a/br/backup-and-restore-faq.md +++ b/br/backup-and-restore-faq.md @@ -60,10 +60,10 @@ During data backup, backup files are generated on the Leader nodes of each Regio However, if you want to restore data from local storage, the number of replicas is equal to that of the TiKV nodes, because each TiKV must have access to all backup files. -## What should I pay attention to when BR restores data to the upstream cluster of TiCDC/Drainer? +## What should I do when BR restores data to the upstream cluster of TiCDC/Drainer? -+ **The data restored by BR cannot be replicated to the downstream**. Because BR directly imports SST files, and the downstream cluster currently cannot obtain upstream SST files. ++ **The data restored using BR cannot be replicated to the downstream**. This is because BR directly imports SST files but the downstream cluster currently cannot obtain these files from the upstream. + Restored data that cannot be replicated to downstream might cause TiCDC/Drainer to abnormally execute DDL; therefore, if you must perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored by BR to the TiCDC/Drainer block list. -You can use [`filter.rules`](https://github.com/pingcap/ticdc/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) item to complete the configuration of TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) item to complete the configuration of Drainer. \ No newline at end of file +You can use [`filter.rules`](https://github.com/pingcap/ticdc/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) to configure the block list for TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) to configure the block list for Drainer. From e36790e2eb0ea8f769cef9d8abfd72cd2db94f5a Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 8 Sep 2020 15:19:48 +0800 Subject: [PATCH 4/5] Update br/backup-and-restore-faq.md --- br/backup-and-restore-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-faq.md b/br/backup-and-restore-faq.md index 523f042b7200d..09284f5900f69 100644 --- a/br/backup-and-restore-faq.md +++ b/br/backup-and-restore-faq.md @@ -64,6 +64,6 @@ However, if you want to restore data from local storage, the number of replicas + **The data restored using BR cannot be replicated to the downstream**. This is because BR directly imports SST files but the downstream cluster currently cannot obtain these files from the upstream. -+ Restored data that cannot be replicated to downstream might cause TiCDC/Drainer to abnormally execute DDL; therefore, if you must perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored by BR to the TiCDC/Drainer block list. ++ Data from BR before v4.0.3 might cause unexcepted DDL executions; therefore, if you need to perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored using BR to the TiCDC/Drainer block list. You can use [`filter.rules`](https://github.com/pingcap/ticdc/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) to configure the block list for TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) to configure the block list for Drainer. From a09169819e7dce08a3b0a763b490222c66868de6 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Tue, 8 Sep 2020 15:21:01 +0800 Subject: [PATCH 5/5] Update br/backup-and-restore-faq.md --- br/backup-and-restore-faq.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/br/backup-and-restore-faq.md b/br/backup-and-restore-faq.md index 09284f5900f69..c05c81123e68b 100644 --- a/br/backup-and-restore-faq.md +++ b/br/backup-and-restore-faq.md @@ -64,6 +64,6 @@ However, if you want to restore data from local storage, the number of replicas + **The data restored using BR cannot be replicated to the downstream**. This is because BR directly imports SST files but the downstream cluster currently cannot obtain these files from the upstream. -+ Data from BR before v4.0.3 might cause unexcepted DDL executions; therefore, if you need to perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored using BR to the TiCDC/Drainer block list. ++ Data from BR before v4.0.3 might cause unexcepted DDL executions. Therefore, if you need to perform restore on the upstream cluster of TiCDC/Drainer, add all tables restored using BR to the TiCDC/Drainer block list. You can use [`filter.rules`](https://github.com/pingcap/ticdc/blob/7c3c2336f98153326912f3cf6ea2fbb7bcc4a20c/cmd/changefeed.toml#L16) to configure the block list for TiCDC and use [`syncer.ignore-table`](/tidb-binlog/tidb-binlog-configuration-file.md#ignore-table) to configure the block list for Drainer.