Skip to content

[hotfix](external) Fixed External meta replay check #47928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
zy-kkk merged 1 commit into from
Feb 14, 2025
Merged

[hotfix](external) Fixed External meta replay check #47928

zy-kkk merged 1 commit into from
Feb 14, 2025

Conversation

@zy-kkk
Copy link
Member

@zy-kkk zy-kkk commented Feb 14, 2025

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #47603

Problem Summary:

When checking if a table fetched from meta has a remote name, we should use tmpmap

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zy-kkk zy-kkk force-pushed the fix_external_meta_0214 branch from e1a4644 to d375d2a Compare February 14, 2025 09:33
@zy-kkk
Copy link
Member Author

zy-kkk commented Feb 14, 2025

run buildall

@zy-kkk zy-kkk added usercase Important user case type label dev/2.1.x dev/3.0.x labels Feb 14, 2025
@zy-kkk zy-kkk force-pushed the fix_external_meta_0214 branch from d375d2a to 90fd3f8 Compare February 14, 2025 09:36
@zy-kkk
Copy link
Member Author

zy-kkk commented Feb 14, 2025

run buildall

This was referenced Feb 14, 2025
Merged
Merged
@doris-robot
Copy link

doris-robot commented Feb 14, 2025

TPC-H: Total hot run time: 31364 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 90fd3f8ca5c71aa915e4ef6d0b148f64d78b521c, data reload: false

------ Round 1 ----------------------------------
q1	17572	5139	5081	5081
q2	2049	291	170	170
q3	10418	1220	722	722
q4	10209	1002	526	526
q5	7520	2354	2317	2317
q6	190	163	132	132
q7	899	743	582	582
q8	9287	1281	1063	1063
q9	4811	4660	4655	4655
q10	6821	2304	1877	1877
q11	479	283	253	253
q12	343	346	218	218
q13	17757	3676	3127	3127
q14	222	218	203	203
q15	510	447	456	447
q16	626	622	591	591
q17	584	855	344	344
q18	6517	6290	6097	6097
q19	1449	934	539	539
q20	316	317	188	188
q21	2765	2275	1918	1918
q22	362	331	314	314
Total cold run time: 101706 ms
Total hot run time: 31364 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5111	5098	5097	5097
q2	256	329	231	231
q3	2195	2709	2284	2284
q4	1476	1848	1386	1386
q5	4197	4108	4187	4108
q6	207	161	125	125
q7	1851	1792	1688	1688
q8	2532	2615	2498	2498
q9	7147	7122	7095	7095
q10	3008	3194	2780	2780
q11	571	518	487	487
q12	737	751	628	628
q13	3499	3781	3275	3275
q14	294	303	287	287
q15	495	470	466	466
q16	635	673	631	631
q17	1136	1598	1325	1325
q18	7649	7298	7416	7298
q19	824	850	933	850
q20	1952	1984	1888	1888
q21	5390	4950	4835	4835
q22	600	565	542	542
Total cold run time: 51762 ms
Total hot run time: 49804 ms

@doris-robot
Copy link

doris-robot commented Feb 14, 2025

TPC-DS: Total hot run time: 188978 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 90fd3f8ca5c71aa915e4ef6d0b148f64d78b521c, data reload: false

query1	1316	933	930	930
query2	6210	1839	1790	1790
query3	10969	4388	4303	4303
query4	54416	25632	22972	22972
query5	5306	538	490	490
query6	353	183	178	178
query7	4957	506	294	294
query8	319	253	235	235
query9	6011	2519	2511	2511
query10	418	302	244	244
query11	15298	15003	14845	14845
query12	164	104	103	103
query13	1067	492	383	383
query14	9903	6202	6648	6202
query15	203	191	186	186
query16	6998	648	480	480
query17	1046	748	573	573
query18	1519	404	342	342
query19	201	211	162	162
query20	126	117	124	117
query21	219	126	109	109
query22	4393	4353	4324	4324
query23	33891	33436	33322	33322
query24	5659	2475	2469	2469
query25	477	482	407	407
query26	659	286	164	164
query27	1697	511	331	331
query28	2827	2407	2374	2374
query29	563	563	448	448
query30	219	192	159	159
query31	887	908	853	853
query32	76	69	61	61
query33	450	355	301	301
query34	762	833	495	495
query35	798	816	756	756
query36	972	1042	943	943
query37	132	99	76	76
query38	4354	4474	4317	4317
query39	1571	1474	1421	1421
query40	200	117	105	105
query41	52	51	48	48
query42	126	108	105	105
query43	519	509	479	479
query44	1293	793	808	793
query45	188	170	167	167
query46	878	1053	645	645
query47	1844	1890	1807	1807
query48	393	436	319	319
query49	675	517	402	402
query50	727	736	426	426
query51	4311	4298	4251	4251
query52	113	115	102	102
query53	239	273	183	183
query54	474	481	410	410
query55	87	82	81	81
query56	260	297	269	269
query57	1169	1180	1148	1148
query58	254	250	261	250
query59	2843	2821	2686	2686
query60	286	275	281	275
query61	124	114	116	114
query62	760	771	689	689
query63	235	194	194	194
query64	1916	1017	678	678
query65	3374	3235	3150	3150
query66	765	390	289	289
query67	16058	15640	15446	15446
query68	5527	761	498	498
query69	521	316	278	278
query70	1250	1171	1125	1125
query71	467	301	280	280
query72	5958	3576	3711	3576
query73	1248	731	349	349
query74	9205	8928	8683	8683
query75	3191	3120	2694	2694
query76	3913	1159	742	742
query77	534	378	291	291
query78	10013	10084	9324	9324
query79	1675	794	592	592
query80	671	524	444	444
query81	499	273	232	232
query82	197	173	94	94
query83	174	171	149	149
query84	292	94	78	78
query85	740	336	311	311
query86	347	304	285	285
query87	4317	4546	4352	4352
query88	2950	2184	2157	2157
query89	410	313	309	309
query90	1833	189	192	189
query91	135	136	109	109
query92	76	62	54	54
query93	2074	999	565	565
query94	664	410	270	270
query95	338	267	257	257
query96	487	569	277	277
query97	2770	2880	2771	2771
query98	228	206	204	204
query99	1295	1402	1258	1258
Total cold run time: 291568 ms
Total hot run time: 188978 ms

@doris-robot
Copy link

doris-robot commented Feb 14, 2025

ClickBench: Total hot run time: 30.26 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 90fd3f8ca5c71aa915e4ef6d0b148f64d78b521c, data reload: false

query1	0.03	0.04	0.05
query2	0.07	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.10	0.10
query5	0.42	0.40	0.39
query6	1.20	0.64	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.60	0.52	0.52
query10	0.57	0.57	0.57
query11	0.15	0.10	0.10
query12	0.14	0.11	0.12
query13	0.61	0.59	0.61
query14	2.72	2.69	2.74
query15	0.93	0.86	0.84
query16	0.38	0.37	0.37
query17	1.03	1.00	1.06
query18	0.21	0.19	0.19
query19	1.92	1.83	1.97
query20	0.02	0.01	0.02
query21	15.36	0.90	0.55
query22	0.74	1.23	0.74
query23	14.80	1.38	0.64
query24	8.44	3.45	0.50
query25	0.36	0.31	0.09
query26	0.76	0.19	0.13
query27	0.05	0.04	0.05
query28	6.64	0.77	0.42
query29	12.53	3.98	3.30
query30	0.25	0.08	0.05
query31	2.83	0.57	0.38
query32	3.22	0.54	0.47
query33	2.98	3.06	3.13
query34	15.80	5.06	4.52
query35	4.53	4.47	4.50
query36	0.66	0.49	0.49
query37	0.09	0.07	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.12
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.03	0.02	0.03
Total cold run time: 103.34 s
Total hot run time: 30.26 s

dataroaring pushed a commit that referenced this pull request Feb 14, 2025
@zy-kkk
branch-3.0: [hotfix](external) Fixed External meta replay check (#47930)
44fb456 
cherry-pick from #47928
@github-actions
Copy link
Contributor

github-actions bot commented Feb 14, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 14, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Feb 14, 2025

PR approved by anyone and no changes requested.

morningman pushed a commit that referenced this pull request Feb 14, 2025
@zy-kkk
branch-2.1: [hotfix](external) Fixed External meta replay check (#47931)
156a2e3 
cherry-pick from #47928
@zy-kkk zy-kkk merged commit db80419 into apache:master Feb 14, 2025
29 of 30 checks passed
@zy-kkk zy-kkk deleted the fix_external_meta_0214 branch February 14, 2025 15:24
morningman pushed a commit to morningman/doris that referenced this pull request Feb 20, 2025
@zy-kkk @morningman
branch-2.1: [hotfix](external) Fixed External meta replay check (apac…
150d707 
…he#47931)

cherry-pick from apache#47928
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
@zy-kkk @lzyy2024
[hotfix](external) Fixed External meta replay check (apache#47928)
2e950d6 
Related PR: apache#47603

Problem Summary:

When checking if a table fetched from meta has a remote name, we should use tmpmap
deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Apr 30, 2025
@morningman
[hotfix](external) Fixed External meta replay check apache#47928
c713ed2
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@zy-kkk @koarz
[hotfix](external) Fixed External meta replay check (apache#47928)
47e9dd0 
Related PR: apache#47603

Problem Summary:

When checking if a table fetched from meta has a remote name, we should use tmpmap
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morningman morningman morningman approved these changes

@CalvinKirs CalvinKirs CalvinKirs approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.4-merged p0_b p0_r reviewed usercase Important user case type label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@zy-kkk @Thearas @doris-robot @morningman @CalvinKirs @wm1581066 @dataroaring