Skip to content

[fix](nereids) fix bug in StatsCalculator.disableJoinReorderIfStatsInvalid when rowCount is not aviable #53834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
englefly merged 1 commit into from
Aug 7, 2025
Merged

[fix](nereids) fix bug in StatsCalculator.disableJoinReorderIfStatsInvalid when rowCount is not aviable #53834

englefly merged 1 commit into from
Aug 7, 2025

Conversation

@englefly
Copy link
Contributor

@englefly englefly commented Jul 24, 2025

What problem does this PR solve?

this bug is introduced by pr #41790.

when any table row count is not available, disable join reorder.
this is implemented in pr #40349 , but pr #41790 breaks this function.

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly englefly changed the title [fix](nereids) fix bug in StatsCalculator.disableJoinReorderIfStatsInvalid when rowCount==-1 [fix](nereids) fix bug in StatsCalculator.disableJoinReorderIfStatsInvalid when rowCount is not aviable Jul 24, 2025
morrySnow
morrySnow reviewed Jul 24, 2025
View reviewed changes
Copy link
Contributor

@morrySnow morrySnow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add test case to ensure it could work as expected

@englefly
Copy link
Contributor Author

englefly commented Jul 24, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 24, 2025

TPC-H: Total hot run time: 33850 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit efb3e5245f88ac5cd3894794e8a8aa71b310c220, data reload: false

------ Round 1 ----------------------------------
q1	17579	5727	5125	5125
q2	1937	297	177	177
q3	10310	1249	696	696
q4	10250	984	517	517
q5	8286	2406	2296	2296
q6	186	157	128	128
q7	885	732	610	610
q8	9300	1287	1141	1141
q9	6771	5039	5070	5039
q10	6860	2370	1962	1962
q11	459	296	270	270
q12	339	347	220	220
q13	17759	3709	3078	3078
q14	245	240	211	211
q15	550	488	479	479
q16	420	423	375	375
q17	599	848	368	368
q18	7477	7139	7108	7108
q19	1482	934	557	557
q20	332	339	230	230
q21	3836	3102	2303	2303
q22	1068	1041	960	960
Total cold run time: 106930 ms
Total hot run time: 33850 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5224	5142	5085	5085
q2	237	340	224	224
q3	2185	2662	2278	2278
q4	1377	1786	1321	1321
q5	4190	4457	4493	4457
q6	221	194	136	136
q7	2049	1942	1724	1724
q8	2620	2635	2481	2481
q9	7396	7222	7296	7222
q10	3132	3246	2848	2848
q11	592	515	501	501
q12	700	775	601	601
q13	3654	3960	3458	3458
q14	277	289	260	260
q15	513	480	490	480
q16	450	483	443	443
q17	1162	1577	1395	1395
q18	7940	7742	7312	7312
q19	767	839	950	839
q20	1949	1988	1825	1825
q21	4904	4532	4453	4453
q22	1076	1076	1026	1026
Total cold run time: 52615 ms
Total hot run time: 50369 ms

@doris-robot
Copy link

doris-robot commented Jul 24, 2025

TPC-DS: Total hot run time: 187491 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit efb3e5245f88ac5cd3894794e8a8aa71b310c220, data reload: false

query1	991	390	421	390
query2	6517	1687	1634	1634
query3	6763	216	221	216
query4	26552	23272	23362	23272
query5	4429	612	479	479
query6	319	218	211	211
query7	4631	514	281	281
query8	277	220	210	210
query9	8601	2856	2854	2854
query10	443	336	276	276
query11	15645	14939	14796	14796
query12	157	114	125	114
query13	1662	542	420	420
query14	8629	5824	5849	5824
query15	211	196	159	159
query16	7388	650	486	486
query17	1217	731	656	656
query18	2006	437	328	328
query19	199	198	165	165
query20	124	120	115	115
query21	215	122	105	105
query22	4059	4413	4163	4163
query23	33847	33108	33008	33008
query24	8090	2403	2369	2369
query25	567	536	403	403
query26	1231	264	154	154
query27	2761	498	343	343
query28	4335	2221	2175	2175
query29	780	563	445	445
query30	281	214	191	191
query31	903	800	761	761
query32	87	78	69	69
query33	565	433	337	337
query34	776	841	518	518
query35	839	810	722	722
query36	980	1001	896	896
query37	115	99	80	80
query38	4179	4158	4095	4095
query39	1488	1419	1396	1396
query40	221	128	117	117
query41	60	57	53	53
query42	122	109	110	109
query43	504	485	466	466
query44	1329	846	843	843
query45	174	172	168	168
query46	847	1005	633	633
query47	1778	1813	1763	1763
query48	370	418	312	312
query49	721	474	390	390
query50	631	698	404	404
query51	5520	5443	5551	5443
query52	117	110	107	107
query53	225	258	184	184
query54	604	604	533	533
query55	97	86	86	86
query56	306	327	326	326
query57	1198	1215	1150	1150
query58	288	268	270	268
query59	2563	2623	2658	2623
query60	346	336	372	336
query61	125	125	123	123
query62	795	741	649	649
query63	236	194	189	189
query64	4352	1030	727	727
query65	4250	4152	4177	4152
query66	1170	423	353	353
query67	15595	15572	15420	15420
query68	8284	943	558	558
query69	480	331	289	289
query70	1242	1169	1159	1159
query71	453	325	304	304
query72	5735	4770	4923	4770
query73	763	655	354	354
query74	9104	9092	8987	8987
query75	3820	3096	2591	2591
query76	3720	1127	719	719
query77	798	382	331	331
query78	9995	10059	9268	9268
query79	2485	829	599	599
query80	633	541	480	480
query81	484	262	223	223
query82	443	138	114	114
query83	289	256	234	234
query84	299	101	81	81
query85	768	359	321	321
query86	345	321	290	290
query87	4465	4355	4222	4222
query88	3140	2247	2238	2238
query89	384	318	285	285
query90	1925	227	221	221
query91	137	138	107	107
query92	85	70	67	67
query93	1574	994	634	634
query94	686	400	295	295
query95	401	321	317	317
query96	486	579	287	287
query97	2688	2738	2616	2616
query98	244	214	209	209
query99	1455	1419	1278	1278
Total cold run time: 275213 ms
Total hot run time: 187491 ms

@doris-robot
Copy link

doris-robot commented Jul 24, 2025

ClickBench: Total hot run time: 32.25 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit efb3e5245f88ac5cd3894794e8a8aa71b310c220, data reload: false

query1	0.04	0.03	0.04
query2	0.07	0.04	0.04
query3	0.25	0.08	0.07
query4	1.62	0.10	0.11
query5	0.41	0.44	0.39
query6	1.16	0.65	0.67
query7	0.03	0.02	0.01
query8	0.04	0.03	0.04
query9	0.60	0.53	0.52
query10	0.58	0.57	0.57
query11	0.16	0.10	0.11
query12	0.15	0.12	0.12
query13	0.63	0.62	0.62
query14	0.81	0.82	0.84
query15	0.91	0.87	0.86
query16	0.39	0.39	0.40
query17	1.03	1.08	1.05
query18	0.22	0.20	0.20
query19	1.91	1.84	1.81
query20	0.02	0.01	0.01
query21	15.39	0.88	0.55
query22	0.76	1.19	0.76
query23	14.83	1.40	0.62
query24	6.66	1.70	0.62
query25	0.50	0.18	0.07
query26	0.64	0.16	0.15
query27	0.05	0.06	0.05
query28	8.79	0.94	0.43
query29	12.61	4.09	3.30
query30	3.10	3.04	2.96
query31	2.87	0.59	0.39
query32	3.25	0.55	0.47
query33	3.11	3.14	3.06
query34	15.94	5.43	4.81
query35	4.90	4.85	4.82
query36	0.73	0.52	0.49
query37	0.10	0.06	0.07
query38	0.06	0.04	0.03
query39	0.04	0.03	0.02
query40	0.17	0.14	0.13
query41	0.08	0.02	0.03
query42	0.04	0.02	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.68 s
Total hot run time: 32.25 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 24, 2025

FE UT Coverage Report

Increment line coverage 54.55% (12/22) 🎉
Increment coverage report
Complete coverage report

@englefly
Copy link
Contributor Author

englefly commented Jul 30, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Jul 30, 2025

TPC-H: Total hot run time: 33857 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7fd81941720a3cb49cf8f3fcefa6724ff4aa76bd, data reload: false

------ Round 1 ----------------------------------
q1	17609	5713	5370	5370
q2	1937	289	195	195
q3	10327	1277	697	697
q4	10214	1000	522	522
q5	7485	2288	2264	2264
q6	175	161	129	129
q7	880	757	614	614
q8	9299	1268	1083	1083
q9	6750	5029	5059	5029
q10	6898	2355	1961	1961
q11	460	297	261	261
q12	339	363	224	224
q13	17793	3445	2980	2980
q14	242	250	235	235
q15	526	460	454	454
q16	413	439	363	363
q17	537	792	375	375
q18	7754	6976	7021	6976
q19	1730	976	551	551
q20	339	312	220	220
q21	2735	2959	2355	2355
q22	1035	1040	999	999
Total cold run time: 105477 ms
Total hot run time: 33857 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5679	5759	5796	5759
q2	233	309	217	217
q3	2073	2571	2200	2200
q4	1287	1705	1325	1325
q5	4072	4456	4573	4456
q6	221	175	139	139
q7	1969	1926	1720	1720
q8	2536	2489	2506	2489
q9	7340	7186	7364	7186
q10	3163	3389	2811	2811
q11	554	652	500	500
q12	675	763	670	670
q13	3391	3653	3249	3249
q14	296	311	290	290
q15	501	468	439	439
q16	456	465	442	442
q17	1160	1469	1367	1367
q18	7973	8050	7597	7597
q19	10360	934	1012	934
q20	1990	1984	1774	1774
q21	14844	4256	4293	4256
q22	1028	1030	959	959
Total cold run time: 71801 ms
Total hot run time: 50779 ms

@doris-robot
Copy link

doris-robot commented Jul 30, 2025

TPC-DS: Total hot run time: 162737 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7fd81941720a3cb49cf8f3fcefa6724ff4aa76bd, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1030	377	410	377
query2	6526	1748	1657	1657
query3	6739	234	216	216
query4	26988	24167	23604	23604
query5	4314	621	530	530
query6	307	234	242	234
query7	4647	529	298	298
query8	288	239	220	220
query9	8631	3114	3147	3114
query10	483	343	294	294
query11	15940	14975	14642	14642
query12	194	137	136	136
query13	1672	568	423	423
query14	9939	7661	7681	7661
query15	209	214	181	181
query16	7205	632	431	431
query17	1244	743	616	616
query18	2131	446	308	308
query19	208	205	182	182
query20	150	144	138	138
query21	221	123	107	107
query22	3928	3996	4044	3996
query23	34451	33973	34158	33973
query24	7008	2392	2422	2392
query25	488	501	427	427
query26	710	291	159	159
query27	2024	479	355	355
query28	2751	2263	2287	2263
query29	626	600	483	483
query30	287	235	208	208
query31	875	780	721	721
query32	90	79	78	78
query33	544	407	402	402
query34	800	853	520	520
query35	803	822	761	761
query36	1017	1032	946	946
query37	140	110	95	95
query38	3987	4019	3931	3931
query39	1438	1406	1389	1389
query40	249	148	138	138
query41	70	61	58	58
query42	141	125	133	125
query43	494	494	482	482
query44	1383	866	862	862
query45	205	192	186	186
query46	952	1085	682	682
query47	1798	1837	1758	1758
query48	408	452	314	314
query49	686	510	417	417
query50	667	690	407	407
query51	4202	4181	4185	4181
query52	130	135	125	125
query53	253	307	212	212
query54	664	663	566	566
query55	93	88	95	88
query56	374	359	384	359
query57	1214	1261	1150	1150
query58	329	334	329	329
query59	2620	2610	2613	2610
query60	424	398	387	387
query61	128	121	118	118
query62	744	750	650	650
query63	253	212	218	212
query64	2511	1109	778	778
query65	4180	4089	4140	4089
query66	969	454	355	355
query67	query68	17980	631	612	612
query69	992	306	292	292
query70	1371	1060	1141	1060
query71	715	346	339	339
query72	9184	2227	2408	2227
query73	3838	633	350	350
query74	9096	8934	8868	8868
query75	7725	3107	2642	2642
query76	8740	1208	772	772
query77	1149	400	329	329
query78	query79	16786	581	571	571
query80	3152	562	472	472
query81	573	294	244	244
query82	557	151	118	118
query83	458	279	265	265
query84	295	99	91	91
query85	1698	372	332	332
query86	366	302	297	297
query87	4273	4352	4198	4198
query88	5000	2262	2199	2199
query89	535	376	316	316
query90	2658	237	230	230
query91	142	134	108	108
query92	87	71	67	67
query93	6712	938	661	661
query94	1172	389	286	286
query95	394	324	317	317
query96	490	586	282	282
query97	2682	2726	2589	2589
query98	239	226	222	222
query99	1547	1372	1275	1275
Total cold run time: 297780 ms
Total hot run time: 162737 ms

@doris-robot
Copy link

doris-robot commented Jul 30, 2025

ClickBench: Total hot run time: 32.74 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7fd81941720a3cb49cf8f3fcefa6724ff4aa76bd, data reload: false

query1	0.04	0.04	0.04
query2	0.07	0.04	0.05
query3	0.24	0.07	0.07
query4	1.63	0.11	0.11
query5	0.41	0.42	0.42
query6	1.17	0.66	0.67
query7	0.03	0.02	0.02
query8	0.05	0.04	0.03
query9	0.57	0.46	0.46
query10	0.52	0.53	0.52
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.65	0.64	0.64
query14	0.96	1.01	1.17
query15	0.91	0.86	0.90
query16	0.39	0.40	0.39
query17	1.07	1.08	1.04
query18	0.21	0.21	0.20
query19	2.04	1.82	1.94
query20	0.01	0.01	0.01
query21	15.36	0.87	0.56
query22	0.78	1.18	0.69
query23	14.89	1.21	0.63
query24	6.65	1.79	0.38
query25	0.38	0.15	0.19
query26	0.57	0.17	0.16
query27	0.05	0.06	0.06
query28	9.38	0.83	0.45
query29	12.58	3.84	3.37
query30	3.03	2.98	2.98
query31	2.81	0.57	0.40
query32	3.23	0.58	0.49
query33	3.02	3.19	3.15
query34	16.14	5.36	4.95
query35	4.88	4.92	4.92
query36	0.71	0.52	0.51
query37	0.09	0.07	0.06
query38	0.06	0.04	0.05
query39	0.03	0.02	0.02
query40	0.17	0.14	0.14
query41	0.08	0.03	0.03
query42	0.03	0.04	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.24 s
Total hot run time: 32.74 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 30, 2025

FE UT Coverage Report

Increment line coverage 54.55% (12/22) 🎉
Increment coverage report
Complete coverage report

@englefly englefly force-pushed the disable-join-reorder-notablerow branch from 6cf6c99 to fd0d658 Compare July 31, 2025 06:45
@englefly
Copy link
Contributor Author

englefly commented Jul 31, 2025

run buildall

@hello-stephen
Copy link
Contributor

hello-stephen commented Jul 31, 2025

FE UT Coverage Report

Increment line coverage 54.55% (12/22) 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

doris-robot commented Jul 31, 2025

TPC-H: Total hot run time: 34006 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fd0d658da383e244f50e2def270ece21124f443e, data reload: false

------ Round 1 ----------------------------------
q1	17788	5711	5547	5547
q2	1952	300	182	182
q3	10665	1322	723	723
q4	10341	1035	502	502
q5	10016	2266	2380	2266
q6	202	167	133	133
q7	895	770	625	625
q8	9322	1327	1129	1129
q9	7020	5027	5160	5027
q10	6946	2356	1959	1959
q11	470	281	267	267
q12	356	368	227	227
q13	17802	3531	2993	2993
q14	253	244	216	216
q15	545	482	483	482
q16	455	445	362	362
q17	568	840	357	357
q18	8473	6969	6977	6969
q19	1465	1009	538	538
q20	319	327	230	230
q21	3498	3065	2296	2296
q22	1043	1047	976	976
Total cold run time: 110394 ms
Total hot run time: 34006 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5887	5838	5805	5805
q2	270	321	218	218
q3	2093	2603	2200	2200
q4	1331	1741	1532	1532
q5	4523	4318	4461	4318
q6	215	180	157	157
q7	1977	1809	1890	1809
q8	2622	2424	2434	2424
q9	7367	7486	7415	7415
q10	3175	3385	2884	2884
q11	533	508	496	496
q12	707	765	625	625
q13	3400	3905	3179	3179
q14	293	317	270	270
q15	492	448	439	439
q16	465	488	449	449
q17	1176	1546	1554	1546
q18	8011	7806	7905	7806
q19	10397	968	1057	968
q20	1885	1951	1767	1767
q21	9614	4290	4236	4236
q22	1075	1028	974	974
Total cold run time: 67508 ms
Total hot run time: 51517 ms

@doris-robot
Copy link

doris-robot commented Jul 31, 2025

TPC-DS: Total hot run time: 162920 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fd0d658da383e244f50e2def270ece21124f443e, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1017	377	421	377
query2	6521	1817	1723	1723
query3	6747	227	220	220
query4	27206	23263	22828	22828
query5	4424	658	512	512
query6	320	237	219	219
query7	4629	514	296	296
query8	286	254	229	229
query9	8608	3184	3168	3168
query10	487	347	296	296
query11	15765	15073	14850	14850
query12	190	140	134	134
query13	1680	568	425	425
query14	9617	7707	7825	7707
query15	212	204	173	173
query16	7998	629	482	482
query17	1646	821	643	643
query18	2109	476	328	328
query19	300	215	206	206
query20	152	142	143	142
query21	226	128	118	118
query22	4216	4073	3970	3970
query23	34379	34172	34538	34172
query24	8178	2393	2399	2393
query25	544	506	445	445
query26	888	291	166	166
query27	2633	501	349	349
query28	4231	2365	2309	2309
query29	683	595	481	481
query30	297	232	198	198
query31	892	802	730	730
query32	109	75	79	75
query33	571	411	391	391
query34	800	834	514	514
query35	796	850	750	750
query36	1016	1047	938	938
query37	142	106	89	89
query38	3973	3972	3946	3946
query39	1433	1382	1403	1382
query40	237	145	131	131
query41	76	69	65	65
query42	142	123	132	123
query43	519	525	485	485
query44	1478	870	875	870
query45	208	182	181	181
query46	953	1055	665	665
query47	1789	1832	1757	1757
query48	398	428	316	316
query49	708	502	422	422
query50	659	696	419	419
query51	4119	4119	4182	4119
query52	128	131	120	120
query53	261	294	213	213
query54	632	654	578	578
query55	91	87	88	87
query56	352	362	358	358
query57	1187	1214	1135	1135
query58	342	330	335	330
query59	2587	2651	2619	2619
query60	407	398	383	383
query61	123	119	118	118
query62	785	720	661	661
query63	250	209	213	209
query64	3495	1097	773	773
query65	4257	4115	4093	4093
query66	976	449	356	356
query67	query68	17810	590	583	583
query69	998	295	278	278
query70	1420	1159	1127	1127
query71	693	388	322	322
query72	9200	2308	2348	2308
query73	3915	630	353	353
query74	8919	8709	8801	8709
query75	7578	3124	2685	2685
query76	8540	1232	777	777
query77	1169	412	332	332
query78	query79	18454	866	853	853
query80	2648	469	484	469
query81	591	242	244	242
query82	582	133	112	112
query83	345	277	261	261
query84	296	103	81	81
query85	1700	394	335	335
query86	360	330	269	269
query87	4282	4293	4224	4224
query88	5371	2226	2238	2226
query89	507	360	328	328
query90	2549	238	240	238
query91	153	138	117	117
query92	89	71	66	66
query93	7063	974	665	665
query94	1164	395	284	284
query95	434	384	331	331
query96	494	586	281	281
query97	2679	2702	2609	2609
query98	241	223	222	222
query99	1557	1364	1273	1273
Total cold run time: 304881 ms
Total hot run time: 162920 ms

@doris-robot
Copy link

doris-robot commented Jul 31, 2025

ClickBench: Total hot run time: 32.77 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fd0d658da383e244f50e2def270ece21124f443e, data reload: false

query1	0.05	0.04	0.04
query2	0.08	0.04	0.04
query3	0.24	0.07	0.08
query4	1.62	0.11	0.11
query5	0.42	0.42	0.41
query6	1.18	0.67	0.67
query7	0.03	0.02	0.02
query8	0.05	0.03	0.03
query9	0.56	0.47	0.47
query10	0.51	0.53	0.53
query11	0.15	0.10	0.11
query12	0.15	0.11	0.11
query13	0.64	0.63	0.64
query14	0.93	1.11	1.05
query15	0.94	0.89	0.89
query16	0.38	0.40	0.38
query17	1.05	1.08	1.07
query18	0.23	0.21	0.20
query19	2.03	1.94	1.95
query20	0.02	0.01	0.02
query21	15.37	0.85	0.55
query22	0.75	1.12	0.61
query23	15.11	1.18	0.62
query24	6.46	1.98	0.34
query25	0.29	0.21	0.14
query26	0.62	0.16	0.14
query27	0.06	0.05	0.06
query28	9.62	0.85	0.44
query29	12.57	3.82	3.33
query30	3.06	3.08	2.98
query31	2.81	0.57	0.40
query32	3.25	0.57	0.50
query33	3.01	3.13	3.24
query34	15.99	5.30	4.94
query35	4.95	5.02	4.97
query36	0.71	0.53	0.51
query37	0.10	0.08	0.08
query38	0.06	0.05	0.04
query39	0.04	0.02	0.02
query40	0.18	0.13	0.14
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 106.42 s
Total hot run time: 32.77 s

@englefly englefly force-pushed the disable-join-reorder-notablerow branch from fd0d658 to 10da22b Compare August 6, 2025 06:55
@englefly
Copy link
Contributor Author

englefly commented Aug 6, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Aug 6, 2025

TPC-H: Total hot run time: 33766 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 10da22b7261d58c6326a09206345cff3973e379e, data reload: false

------ Round 1 ----------------------------------
q1	17592	5257	5377	5257
q2	1924	289	200	200
q3	10341	1444	697	697
q4	10295	1092	535	535
q5	9515	2345	2317	2317
q6	220	167	134	134
q7	893	786	621	621
q8	9332	1298	1075	1075
q9	7058	5076	5040	5040
q10	6928	2365	1996	1996
q11	483	281	266	266
q12	355	364	233	233
q13	17770	3438	3011	3011
q14	232	253	216	216
q15	531	470	467	467
q16	412	440	378	378
q17	592	823	360	360
q18	7183	7045	6925	6925
q19	1971	983	553	553
q20	334	324	214	214
q21	3372	2474	2251	2251
q22	1068	1043	1020	1020
Total cold run time: 108401 ms
Total hot run time: 33766 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5563	5287	5288	5287
q2	230	310	220	220
q3	2082	2547	2197	2197
q4	1282	1720	1336	1336
q5	4207	4494	4179	4179
q6	216	171	154	154
q7	2042	2176	1812	1812
q8	2498	2497	2382	2382
q9	7672	7157	7475	7157
q10	3194	3422	2809	2809
q11	562	528	497	497
q12	693	767	645	645
q13	3419	3693	3232	3232
q14	316	309	270	270
q15	486	470	457	457
q16	629	497	455	455
q17	1156	1495	1353	1353
q18	8224	7560	7664	7560
q19	17404	872	828	828
q20	2011	1984	1866	1866
q21	13997	4229	4288	4229
q22	1076	1030	992	992
Total cold run time: 78959 ms
Total hot run time: 49917 ms

@doris-robot
Copy link

doris-robot commented Aug 6, 2025

TPC-DS: Total hot run time: 161575 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 10da22b7261d58c6326a09206345cff3973e379e, data reload: false

reason	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 22:43:56	2023-12-26 22:44:01	NULL	utf-8	NULL	NULL	
============================================
query1	1009	389	425	389
query2	6550	1780	1720	1720
query3	6744	228	223	223
query4	26778	23696	23107	23107
query5	4310	607	536	536
query6	350	233	238	233
query7	4637	517	289	289
query8	265	235	211	211
query9	8622	2939	2910	2910
query10	470	326	299	299
query11	15798	14908	14736	14736
query12	189	139	133	133
query13	1652	551	404	404
query14	8479	5926	5829	5829
query15	201	196	164	164
query16	7139	649	454	454
query17	951	763	618	618
query18	1977	430	324	324
query19	219	205	186	186
query20	143	144	137	137
query21	213	126	112	112
query22	3961	4144	4021	4021
query23	34835	34535	34454	34454
query24	8258	2365	2412	2365
query25	578	517	463	463
query26	1243	289	157	157
query27	2736	497	373	373
query28	4397	2323	2316	2316
query29	821	600	503	503
query30	301	240	193	193
query31	895	789	695	695
query32	92	77	79	77
query33	569	415	379	379
query34	811	839	520	520
query35	819	901	755	755
query36	1017	1050	928	928
query37	139	116	95	95
query38	3989	3941	3932	3932
query39	1461	1380	1366	1366
query40	243	152	140	140
query41	70	63	60	60
query42	143	125	132	125
query43	510	493	474	474
query44	1414	881	861	861
query45	209	195	196	195
query46	967	1068	690	690
query47	1850	1832	1796	1796
query48	410	437	331	331
query49	723	536	446	446
query50	656	704	428	428
query51	4181	4176	4165	4165
query52	129	131	120	120
query53	277	289	220	220
query54	660	669	567	567
query55	96	94	91	91
query56	367	361	377	361
query57	1218	1249	1127	1127
query58	349	347	346	346
query59	2707	2691	2562	2562
query60	409	403	383	383
query61	124	127	129	127
query62	811	729	650	650
query63	249	223	215	215
query64	4368	1111	893	893
query65	4267	4179	4146	4146
query66	1186	453	338	338
query67	query68	20213	1135	790	790
query69	1007	308	290	290
query70	1412	1105	1104	1104
query71	684	358	321	321
query72	9112	2342	4780	2342
query73	4609	772	357	357
query74	8997	8970	8866	8866
query75	7171	3200	2681	2681
query76	8487	1243	843	843
query77	1179	414	337	337
query78	query79	17731	615	598	598
query80	2165	589	490	490
query81	541	270	222	222
query82	462	151	115	115
query83	350	330	268	268
query84	297	97	84	84
query85	1659	411	345	345
query86	365	308	297	297
query87	4218	4214	4183	4183
query88	5643	2209	2202	2202
query89	519	368	315	315
query90	2618	232	232	232
query91	171	140	113	113
query92	85	79	67	67
query93	6870	1004	656	656
query94	1074	410	290	290
query95	407	327	333	327
query96	505	593	277	277
query97	2728	2746	2597	2597
query98	269	239	226	226
query99	1556	1433	1277	1277
Total cold run time: 305315 ms
Total hot run time: 161575 ms

@doris-robot
Copy link

doris-robot commented Aug 6, 2025

ClickBench: Total hot run time: 33.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 10da22b7261d58c6326a09206345cff3973e379e, data reload: false

query1	0.04	0.03	0.03
query2	0.09	0.04	0.05
query3	0.24	0.08	0.07
query4	1.61	0.11	0.11
query5	0.41	0.42	0.41
query6	1.17	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.04	0.04
query9	0.57	0.48	0.48
query10	0.51	0.53	0.53
query11	0.16	0.11	0.10
query12	0.15	0.11	0.11
query13	0.66	0.63	0.64
query14	0.94	1.22	1.02
query15	0.93	0.89	0.88
query16	0.39	0.39	0.40
query17	1.09	1.07	1.08
query18	0.23	0.21	0.22
query19	2.04	1.88	1.90
query20	0.02	0.02	0.02
query21	15.37	0.84	0.56
query22	0.78	1.20	0.84
query23	14.73	1.22	0.63
query24	6.48	1.95	1.18
query25	0.54	0.24	0.13
query26	0.57	0.14	0.13
query27	0.07	0.06	0.05
query28	10.33	0.80	0.44
query29	12.59	3.85	3.29
query30	3.05	3.06	3.04
query31	2.81	0.57	0.40
query32	3.24	0.58	0.52
query33	3.09	3.16	3.26
query34	15.94	5.44	4.99
query35	4.99	4.97	5.04
query36	0.68	0.53	0.50
query37	0.10	0.08	0.07
query38	0.06	0.06	0.04
query39	0.03	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.03	0.02
query42	0.04	0.03	0.04
query43	0.04	0.04	0.03
Total cold run time: 107.09 s
Total hot run time: 33.88 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Aug 6, 2025

FE UT Coverage Report

Increment line coverage 52.17% (12/23) 🎉
Increment coverage report
Complete coverage report

@github-actions
Copy link
Contributor

github-actions bot commented Aug 7, 2025

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Aug 7, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Aug 7, 2025

PR approved by anyone and no changes requested.

@englefly englefly merged commit 7ebe2b3 into apache:master Aug 7, 2025
26 of 28 checks passed
@englefly englefly deleted the disable-join-reorder-notablerow branch August 7, 2025 02:30
This was referenced Aug 7, 2025
Open
Merged
Merged
freemandealer pushed a commit to freemandealer/doris that referenced this pull request Aug 7, 2025
@englefly @freemandealer
[fix](nereids) fix bug in StatsCalculator.disableJoinReorderIfStatsIn…
51965a9 
…valid when rowCount is not aviable (apache#53834)

### What problem does this PR solve?
this bug is introduced by pr apache#41790.

when any table row count is not available, disable join reorder.
this is implemented in pr apache#40349 , but pr apache#41790 breaks this function.
@morrySnow morrySnow mentioned this pull request Aug 9, 2025
Merged
morrySnow pushed a commit that referenced this pull request Aug 9, 2025
@englefly
branch-3.1: [fix](nereids) fix bug in StatsCalculator.disableJoinReor…
2e27b81 
…derIfStatsInvalid when rowCount is not aviable #53834 (#54428)

picked from #53834
yiguolei pushed a commit that referenced this pull request Aug 14, 2025
@englefly
branch-2.1[fix](nereids) fix bug in StatsCalculator.disableJoinReorde…
26b0cef 
…rIfStatsInvalid when rowCount is not aviable #53834 (#54426)

### What problem does this PR solve?
Pick #53834
Issue Number: close #xxx

Related PR: #xxx
dataroaring pushed a commit that referenced this pull request Aug 14, 2025
@englefly
branch-3.0 [fix](nereids) fix bug in StatsCalculator.disableJoinReord…
43b104d 
…erIfStatsInvalid when rowCount is not aviable #53834 (#54427)

### What problem does this PR solve?
pick #53834 
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
@zddr zddr mentioned this pull request Aug 26, 2025
Merged
morrySnow pushed a commit that referenced this pull request Aug 26, 2025
@zddr
branch-3.1: [case](mtmv)fix case mv_ssb_test #53834 (#55279)
b445f53 
picked case mv_ssb_test from #53834
@gavinchou gavinchou mentioned this pull request Sep 1, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@morrySnow morrySnow morrySnow left review comments

@starocean999 starocean999 starocean999 approved these changes

@Jibing-Li Jibing-Li Jibing-Li approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.12-merged dev/3.0.8-merged dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@englefly @hello-stephen @doris-robot @starocean999 @Jibing-Li @morrySnow @yiguolei @dataroaring