Skip to content

branch-3.0: [fix](nereids) fix sql cache bug and some tests #46443 #46506

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 2 commits into from
Jan 7, 2025
Merged

branch-3.0: [fix](nereids) fix sql cache bug and some tests #46443 #46506

924060929 merged 2 commits into from
Jan 7, 2025

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Jan 7, 2025

Cherry-picked from #46443

@924060929
[fix](nereids) fix sql cache bug and some tests (#46443)
f507231 
1. use retry to fix unstable test `colocate_union_numbers`,
`prune_bucket_with_bucket_shuffle_join`
2. fix failed test `explain`, this bug only exists in master branch,
introduced by #40202
3. fix sql cache bug which use stale cache after drop table and
create(table id changed), test in `parse_sql_from_sql_cache`, introduced
by #33262
4. regression test add `foreachFrontends`, `foreachBackends`, `retry`
function
@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring closed this Jan 7, 2025
@dataroaring dataroaring reopened this Jan 7, 2025
@Thearas
Copy link
Contributor

Thearas commented Jan 7, 2025

run buildall

@924060929
Copy link
Contributor

924060929 commented Jan 7, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jan 7, 2025

TPC-H: Total hot run time: 41026 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0ba7da02c52c509571aba69cb54aa39d4d858bd8, data reload: false

------ Round 1 ----------------------------------
q1	17591	7445	7280	7280
q2	2082	170	158	158
q3	10563	1151	1134	1134
q4	10471	793	675	675
q5	7937	2888	2916	2888
q6	238	146	149	146
q7	988	627	621	621
q8	9794	1970	2006	1970
q9	6672	6447	6374	6374
q10	7056	2275	2305	2275
q11	462	267	269	267
q12	419	217	207	207
q13	17787	2990	3025	2990
q14	242	220	214	214
q15	558	525	520	520
q16	698	628	598	598
q17	973	528	605	528
q18	7382	6662	6687	6662
q19	1388	1047	1028	1028
q20	445	193	191	191
q21	4030	3402	3298	3298
q22	1081	1002	1008	1002
Total cold run time: 108857 ms
Total hot run time: 41026 ms

----- Round 2, with runtime_filter_mode=off -----
q1	7357	7300	7244	7244
q2	332	234	226	226
q3	2996	2925	3055	2925
q4	2113	1823	1815	1815
q5	5700	5690	5696	5690
q6	222	134	134	134
q7	2191	1795	1857	1795
q8	3347	3532	3513	3513
q9	8795	8858	8836	8836
q10	3628	3572	3594	3572
q11	590	499	488	488
q12	786	591	634	591
q13	7682	3159	3144	3144
q14	306	261	271	261
q15	568	505	509	505
q16	706	648	689	648
q17	1793	1574	1552	1552
q18	7756	7417	7348	7348
q19	1646	1446	1499	1446
q20	2018	1785	1792	1785
q21	5524	5264	5112	5112
q22	1128	1020	1010	1010
Total cold run time: 67184 ms
Total hot run time: 59640 ms

@doris-robot
Copy link

doris-robot commented Jan 7, 2025

TPC-DS: Total hot run time: 191406 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0ba7da02c52c509571aba69cb54aa39d4d858bd8, data reload: false

query1	990	372	364	364
query2	6510	2040	2016	2016
query3	6712	216	226	216
query4	34233	23425	23558	23425
query5	4361	459	437	437
query6	264	178	198	178
query7	4629	312	318	312
query8	287	215	226	215
query9	9354	2700	2696	2696
query10	480	257	272	257
query11	18092	15146	15218	15146
query12	150	110	105	105
query13	1652	427	424	424
query14	10310	6928	6969	6928
query15	255	175	178	175
query16	7733	464	475	464
query17	1659	576	552	552
query18	2056	315	324	315
query19	359	167	152	152
query20	117	107	111	107
query21	215	108	105	105
query22	4620	4440	4338	4338
query23	34466	33959	34016	33959
query24	11275	2811	2933	2811
query25	674	393	396	393
query26	1373	164	169	164
query27	2812	335	335	335
query28	7930	2453	2443	2443
query29	873	438	430	430
query30	327	161	169	161
query31	1026	806	816	806
query32	94	55	59	55
query33	768	280	286	280
query34	949	498	516	498
query35	840	703	701	701
query36	1101	943	937	937
query37	137	72	77	72
query38	4061	3868	3823	3823
query39	1474	1400	1422	1400
query40	281	105	97	97
query41	52	47	49	47
query42	116	103	97	97
query43	525	477	481	477
query44	1288	808	803	803
query45	185	165	178	165
query46	1134	714	696	696
query47	1943	1839	1844	1839
query48	463	370	376	370
query49	1179	381	383	381
query50	809	417	412	412
query51	7339	7041	7083	7041
query52	103	94	91	91
query53	257	185	184	184
query54	1277	470	468	468
query55	85	81	75	75
query56	261	233	250	233
query57	1242	1124	1094	1094
query58	243	205	216	205
query59	3241	3090	3046	3046
query60	283	254	254	254
query61	113	111	107	107
query62	875	660	676	660
query63	226	194	187	187
query64	5372	664	644	644
query65	3303	3167	3174	3167
query66	1467	316	309	309
query67	15935	15768	15591	15591
query68	4454	588	574	574
query69	425	277	266	266
query70	1145	1144	1098	1098
query71	338	259	261	259
query72	6402	3983	4123	3983
query73	750	345	346	345
query74	10164	9107	8974	8974
query75	3345	2594	2665	2594
query76	2668	1143	1009	1009
query77	400	275	278	275
query78	10302	9519	9646	9519
query79	1087	601	596	596
query80	624	421	441	421
query81	518	245	240	240
query82	206	121	114	114
query83	167	146	153	146
query84	245	76	82	76
query85	990	300	284	284
query86	311	300	286	286
query87	4462	4283	4274	4274
query88	3327	2437	2385	2385
query89	382	297	292	292
query90	2005	185	186	185
query91	179	156	171	156
query92	58	52	49	49
query93	1036	547	548	547
query94	794	294	291	291
query95	351	256	257	256
query96	605	285	282	282
query97	3315	3201	3223	3201
query98	211	218	196	196
query99	1535	1298	1318	1298
Total cold run time: 297616 ms
Total hot run time: 191406 ms

@doris-robot
Copy link

doris-robot commented Jan 7, 2025

ClickBench: Total hot run time: 33.71 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0ba7da02c52c509571aba69cb54aa39d4d858bd8, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.03	0.04
query3	0.23	0.06	0.07
query4	1.63	0.10	0.10
query5	0.53	0.52	0.53
query6	1.13	0.74	0.73
query7	0.02	0.01	0.01
query8	0.03	0.04	0.03
query9	0.56	0.51	0.51
query10	0.54	0.56	0.55
query11	0.15	0.11	0.11
query12	0.14	0.12	0.13
query13	0.61	0.61	0.59
query14	3.04	2.92	2.90
query15	0.90	0.83	0.83
query16	0.38	0.39	0.38
query17	1.04	1.00	1.02
query18	0.23	0.22	0.22
query19	2.01	1.94	1.86
query20	0.02	0.01	0.01
query21	15.37	0.61	0.59
query22	2.84	2.71	1.99
query23	17.21	0.89	1.14
query24	2.68	1.23	2.38
query25	0.25	0.15	0.10
query26	0.52	0.13	0.14
query27	0.06	0.05	0.04
query28	9.94	1.10	1.08
query29	12.61	3.32	3.28
query30	0.25	0.06	0.07
query31	2.88	0.40	0.40
query32	3.23	0.46	0.47
query33	3.02	3.02	3.02
query34	16.96	4.54	4.48
query35	4.62	4.56	4.57
query36	0.68	0.50	0.51
query37	0.09	0.07	0.06
query38	0.05	0.03	0.03
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.04	0.03	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.9 s
Total hot run time: 33.71 s

@924060929 924060929 merged commit 6551775 into branch-3.0 Jan 7, 2025
23 of 25 checks passed
@github-actions github-actions bot deleted the auto-pick-46443-branch-3.0 branch January 7, 2025 06:29
csun5285 pushed a commit to csun5285/doris that referenced this pull request Mar 28, 2025
@morrySnow
[fix](nereids) fix sql cache bug and some tests apache#46443 apache#4…
900a7e6 
…6537 (apache#3821)

pick from apache doris' branch-3.0

- apache#46506
- apache#46555
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@Thearas @924060929 @doris-robot @dataroaring