[Bug](backup) publish will always timeout after doris upgraded from 1.2 to 2.0 #37076
Description
Search before asking
- I had searched in the issues and found no similar issues.
Version
1.2|2.0
What's Wrong?
After upgrading from version 1.2 to version 2.0, we found that all tables writes failed during the publish phase.
E.g. Do insert on a old table upgraded from 1.2, it will encountered publish timeout(default 10s):
MySQL [report]> INSERT INTO t_jishu_silian (id, jflg, compid, empid, jishu, bilv1, bilv2, dayin, dayout) VALUES('598690564658692098', 1, '585500694313795584', NULL, NULL, 0.60000, 0.00000, NULL, NULL);
Query OK, 1 row affected (10.03 sec)
{'label':'insert_1fd3d610d1ec489b_b0621f9b4ae7aac5', 'status':'COMMITTED', 'txnId':'2009'}
After reviewing, we found that as long as backup and restore operations were performed on the original 1.2 version cluster, it would lead to publish failures after upgrading to version 2.0.
The reason is that during the restore operation in version 1.2, when using snapshot to replace the old data file, the tablet metadata used the information in the backup file, then it is modified according to the real meta in FE, but the table/partition ID of the tablet metadata was not modified.
Therefore, after the restore operation, the table/partition ID in the tablet meta is from the backup file, rather than the real meta in FE.
After upgrade to 2.0, the table/partition ID between tablet meta in BE and real meta in FE is mismatch, which will cause the publish failures.
What You Expected?
To completely fix this issue, we need to take a few steps:
- Fix the backup and restore missing modifying table/partition ID of version 1.2.
- Add a feature of checking and correcting table/partition ID in 1.2/2.0, to solve the problem that table/partition ID mismatch after upgrading.
How to Reproduce?
1、Preform a restore operation in a 1.2 version cluster.
2、upgrade cluster from version 1.2 to 2.0.
3、Do insert on a table which has been done a restore operation in step 1.
4、publish timeout
Anything Else?
No response
Are you willing to submit PR?
- Yes I am willing to submit a PR!
Code of Conduct
- I agree to follow this project's Code of Conduct