Skip to content

Conversation

@keanji-x
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@keanji-x
Copy link
Contributor Author

run buildall

@keanji-x
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38322 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1f6a5bc4ddcbd040e7d3addb1a2ac92402b2cce, data reload: false

------ Round 1 ----------------------------------
q1	17599	4420	4235	4235
q2	2015	187	187	187
q3	10449	1209	1168	1168
q4	10199	720	794	720
q5	7538	2726	2646	2646
q6	223	133	128	128
q7	1013	609	580	580
q8	9205	2078	2050	2050
q9	7917	6600	6524	6524
q10	8510	3541	3536	3536
q11	465	241	235	235
q12	463	223	212	212
q13	17936	2923	2875	2875
q14	262	234	232	232
q15	521	496	479	479
q16	508	378	380	378
q17	964	662	665	662
q18	7418	6999	6729	6729
q19	6146	1447	1504	1447
q20	692	313	302	302
q21	3461	2744	2700	2700
q22	361	299	297	297
Total cold run time: 113865 ms
Total hot run time: 38322 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4333	4220	4221	4220
q2	363	253	260	253
q3	2967	2788	2746	2746
q4	1908	1636	1641	1636
q5	5349	5345	5315	5315
q6	211	123	125	123
q7	2242	1877	1858	1858
q8	3216	3342	3332	3332
q9	8562	8594	8608	8594
q10	4056	3931	3988	3931
q11	615	492	490	490
q12	785	633	648	633
q13	16432	3172	3142	3142
q14	324	276	289	276
q15	521	504	501	501
q16	491	450	461	450
q17	1825	1603	1501	1501
q18	8158	7978	7822	7822
q19	1674	1597	1620	1597
q20	2043	1857	1809	1809
q21	7146	4982	5004	4982
q22	545	476	479	476
Total cold run time: 73766 ms
Total hot run time: 55687 ms

@keanji-x
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38283 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0a9deaebce47c8fc8fdd9a02ce29b616182e1f47, data reload: false

------ Round 1 ----------------------------------
q1	17624	4308	4253	4253
q2	2026	188	186	186
q3	10459	1168	1176	1168
q4	10185	811	832	811
q5	7527	2647	2628	2628
q6	209	129	128	128
q7	984	605	593	593
q8	9203	2041	2016	2016
q9	7970	6566	6503	6503
q10	8528	3517	3486	3486
q11	461	232	226	226
q12	433	220	217	217
q13	17984	2909	2942	2909
q14	278	237	237	237
q15	510	474	486	474
q16	520	397	389	389
q17	952	627	730	627
q18	7285	6677	6622	6622
q19	7636	1545	1528	1528
q20	694	321	314	314
q21	3456	2666	2682	2666
q22	353	303	302	302
Total cold run time: 115277 ms
Total hot run time: 38283 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4323	4244	4214	4214
q2	375	271	254	254
q3	2967	2740	2763	2740
q4	1824	1548	1577	1548
q5	5238	5282	5271	5271
q6	209	124	123	123
q7	2211	1851	1869	1851
q8	3231	3327	3306	3306
q9	8505	8474	8611	8474
q10	4067	3927	4052	3927
q11	609	477	497	477
q12	793	614	621	614
q13	16141	3227	3174	3174
q14	320	289	298	289
q15	532	477	467	467
q16	484	466	445	445
q17	1834	1533	1555	1533
q18	8076	7961	7658	7658
q19	1720	1649	1588	1588
q20	1972	1802	1840	1802
q21	5119	5109	5013	5013
q22	549	451	474	451
Total cold run time: 71099 ms
Total hot run time: 55219 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183501 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0a9deaebce47c8fc8fdd9a02ce29b616182e1f47, data reload: false

query1	876	377	1123	377
query2	6205	2770	2709	2709
query3	6765	215	216	215
query4	39153	21914	21306	21306
query5	4169	390	389	389
query6	248	202	180	180
query7	4035	293	288	288
query8	240	196	189	189
query9	5754	2297	2253	2253
query10	359	238	237	237
query11	14714	14211	14155	14155
query12	148	88	89	88
query13	991	352	354	352
query14	9232	6924	7035	6924
query15	236	173	182	173
query16	7152	264	249	249
query17	1877	564	535	535
query18	1805	274	266	266
query19	329	149	144	144
query20	94	86	90	86
query21	201	121	126	121
query22	4972	4846	4774	4774
query23	33750	32966	33346	32966
query24	11443	3018	3012	3012
query25	558	369	395	369
query26	1190	156	158	156
query27	3092	361	385	361
query28	7461	2043	2062	2043
query29	883	607	611	607
query30	319	178	174	174
query31	934	757	753	753
query32	63	54	55	54
query33	567	253	244	244
query34	1031	489	498	489
query35	853	716	726	716
query36	1038	922	921	921
query37	133	71	73	71
query38	3426	3356	3367	3356
query39	1640	1581	1709	1581
query40	180	128	126	126
query41	47	48	44	44
query42	107	97	95	95
query43	589	534	544	534
query44	1431	722	730	722
query45	282	282	271	271
query46	1086	752	733	733
query47	1986	1926	1947	1926
query48	366	292	299	292
query49	905	391	386	386
query50	770	387	391	387
query51	6837	6706	6779	6706
query52	98	93	89	89
query53	357	277	280	277
query54	260	229	236	229
query55	75	72	71	71
query56	243	229	246	229
query57	1275	1182	1163	1163
query58	217	213	218	213
query59	3749	3290	3114	3114
query60	239	237	235	235
query61	96	91	91	91
query62	593	445	438	438
query63	298	278	278	278
query64	4049	4044	3788	3788
query65	3060	3028	3067	3028
query66	736	323	317	317
query67	15579	14891	14832	14832
query68	7158	529	534	529
query69	584	307	308	307
query70	1256	1216	1195	1195
query71	473	275	269	269
query72	6499	2748	2570	2570
query73	818	316	319	316
query74	6622	6580	6419	6419
query75	3494	2708	2607	2607
query76	4230	1123	1062	1062
query77	630	243	338	243
query78	10893	10105	10207	10105
query79	8020	530	507	507
query80	1824	416	416	416
query81	528	241	227	227
query82	1328	95	95	95
query83	196	166	165	165
query84	260	78	83	78
query85	1020	259	258	258
query86	455	302	309	302
query87	3455	3283	3223	3223
query88	5785	2334	2274	2274
query89	531	369	378	369
query90	1901	172	171	171
query91	183	93	97	93
query92	55	44	45	44
query93	6738	501	485	485
query94	1019	172	178	172
query95	373	1213	286	286
query96	623	257	256	256
query97	3089	2926	2957	2926
query98	232	215	216	215
query99	1156	820	865	820
Total cold run time: 304874 ms
Total hot run time: 183501 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.07 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0a9deaebce47c8fc8fdd9a02ce29b616182e1f47, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.22	0.06	0.05
query4	1.66	0.07	0.06
query5	0.51	0.49	0.48
query6	1.47	0.72	0.73
query7	0.02	0.01	0.02
query8	0.05	0.05	0.04
query9	0.53	0.50	0.50
query10	0.55	0.55	0.54
query11	0.16	0.11	0.12
query12	0.14	0.12	0.12
query13	0.60	0.59	0.58
query14	0.74	0.78	0.78
query15	0.83	0.81	0.80
query16	0.36	0.35	0.36
query17	0.93	1.03	1.02
query18	0.26	0.19	0.26
query19	1.74	1.74	1.82
query20	0.01	0.01	0.01
query21	15.40	0.65	0.64
query22	4.82	6.44	2.59
query23	18.29	1.38	1.25
query24	1.87	0.19	0.22
query25	0.14	0.08	0.08
query26	0.25	0.16	0.16
query27	0.07	0.07	0.08
query28	13.49	0.99	0.99
query29	12.56	3.22	3.28
query30	0.26	0.06	0.07
query31	2.85	0.36	0.38
query32	3.28	0.46	0.46
query33	2.88	2.86	2.80
query34	17.24	4.51	4.50
query35	4.46	4.50	4.52
query36	0.65	0.46	0.46
query37	0.19	0.16	0.15
query38	0.15	0.14	0.15
query39	0.04	0.04	0.03
query40	0.17	0.14	0.14
query41	0.09	0.05	0.04
query42	0.05	0.05	0.05
query43	0.04	0.04	0.03
Total cold run time: 110.14 s
Total hot run time: 31.07 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0a9deaebce47c8fc8fdd9a02ce29b616182e1f47 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

@keanji-x keanji-x changed the title [fix](Nereids): move adjust_null in rewrite cte sub-tree topic [fix](Nereids): add whole tree rewriter when root is not CTEAnchor Apr 16, 2024
@keanji-x
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39137 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b926fd225108b098a4e24ecb26d1d25292d98abf, data reload: false

------ Round 1 ----------------------------------
q1	17688	4408	4251	4251
q2	2015	190	192	190
q3	10444	1198	1147	1147
q4	10202	734	793	734
q5	7578	2694	2662	2662
q6	217	131	132	131
q7	1007	590	582	582
q8	9223	2066	2042	2042
q9	7499	6655	6621	6621
q10	8622	3541	3563	3541
q11	449	230	227	227
q12	529	211	211	211
q13	18040	2994	3188	2994
q14	283	258	234	234
q15	538	486	489	486
q16	576	401	412	401
q17	973	648	734	648
q18	8221	7326	7184	7184
q19	10490	1564	1512	1512
q20	1079	311	306	306
q21	3394	2729	2731	2729
q22	369	309	304	304
Total cold run time: 119436 ms
Total hot run time: 39137 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4416	4283	4251	4251
q2	479	261	268	261
q3	2992	2712	2723	2712
q4	1872	1630	1614	1614
q5	5305	5310	5339	5310
q6	205	123	123	123
q7	2224	1881	1853	1853
q8	3188	3344	3357	3344
q9	8581	8582	8949	8582
q10	4056	3930	3883	3883
q11	605	503	489	489
q12	789	642	673	642
q13	17653	3111	3194	3111
q14	321	302	287	287
q15	538	465	495	465
q16	519	442	486	442
q17	1806	1511	1508	1508
q18	8100	7919	7679	7679
q19	2638	1572	1624	1572
q20	2106	1838	1856	1838
q21	6598	4958	4810	4810
q22	551	495	489	489
Total cold run time: 75542 ms
Total hot run time: 55265 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183976 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b926fd225108b098a4e24ecb26d1d25292d98abf, data reload: false

query1	889	360	356	356
query2	6185	2828	2315	2315
query3	6657	202	195	195
query4	23727	21291	21292	21291
query5	4170	404	401	401
query6	276	184	176	176
query7	4580	286	280	280
query8	244	195	188	188
query9	8446	2305	2282	2282
query10	416	241	252	241
query11	14672	14138	14257	14138
query12	135	90	86	86
query13	1663	358	350	350
query14	9562	7890	7857	7857
query15	269	187	189	187
query16	8239	252	249	249
query17	2010	559	548	548
query18	2104	273	266	266
query19	327	145	143	143
query20	88	83	84	83
query21	202	128	127	127
query22	4981	4874	4834	4834
query23	34046	33399	33229	33229
query24	11269	2995	3026	2995
query25	628	377	380	377
query26	1272	158	156	156
query27	2261	373	393	373
query28	6917	2083	2022	2022
query29	864	631	613	613
query30	249	193	185	185
query31	976	735	775	735
query32	91	52	55	52
query33	747	249	250	249
query34	1184	481	501	481
query35	831	740	733	733
query36	1051	953	913	913
query37	126	69	77	69
query38	3552	3369	3319	3319
query39	1643	1569	1561	1561
query40	174	127	122	122
query41	47	43	43	43
query42	105	94	105	94
query43	561	549	550	549
query44	1229	748	741	741
query45	286	262	266	262
query46	1126	774	745	745
query47	2048	1987	1919	1919
query48	370	300	287	287
query49	835	382	385	382
query50	794	393	382	382
query51	6926	6761	6717	6717
query52	103	89	90	89
query53	344	282	272	272
query54	304	236	225	225
query55	75	69	71	69
query56	263	225	224	224
query57	1234	1117	1136	1117
query58	234	204	201	201
query59	3287	3410	3123	3123
query60	258	232	228	228
query61	106	102	100	100
query62	597	434	441	434
query63	303	278	276	276
query64	5055	3952	3762	3762
query65	3111	3050	3006	3006
query66	748	333	326	326
query67	15585	15183	14849	14849
query68	9092	557	556	556
query69	552	301	296	296
query70	1441	1091	1194	1091
query71	490	260	264	260
query72	6599	2656	2401	2401
query73	1604	320	319	319
query74	6804	6478	6445	6445
query75	4246	2613	2619	2613
query76	5018	896	1038	896
query77	586	257	252	252
query78	10931	10192	10145	10145
query79	7569	508	511	508
query80	1223	454	440	440
query81	505	236	242	236
query82	237	95	93	93
query83	191	170	160	160
query84	254	84	84	84
query85	933	269	253	253
query86	329	298	285	285
query87	3449	3288	3275	3275
query88	4827	2307	2315	2307
query89	487	367	380	367
query90	1896	181	180	180
query91	122	95	96	95
query92	52	45	45	45
query93	5900	503	493	493
query94	1117	174	175	174
query95	376	294	294	294
query96	610	259	256	256
query97	3150	3002	2906	2906
query98	248	224	212	212
query99	1050	840	851	840
Total cold run time: 295545 ms
Total hot run time: 183976 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.52 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit b926fd225108b098a4e24ecb26d1d25292d98abf, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.04
query3	0.24	0.05	0.05
query4	1.68	0.06	0.07
query5	0.48	0.49	0.50
query6	1.48	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.05	0.04
query9	0.54	0.51	0.49
query10	0.54	0.57	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.12
query13	0.61	0.59	0.58
query14	0.77	0.78	0.77
query15	0.82	0.80	0.80
query16	0.37	0.35	0.35
query17	0.95	0.99	1.02
query18	0.22	0.26	0.24
query19	1.84	1.70	1.77
query20	0.01	0.01	0.01
query21	15.41	0.66	0.65
query22	4.14	7.54	2.16
query23	18.30	1.41	1.28
query24	1.95	0.24	0.21
query25	0.15	0.08	0.08
query26	0.26	0.16	0.15
query27	0.08	0.08	0.08
query28	13.31	1.01	0.98
query29	12.61	3.29	3.20
query30	0.27	0.06	0.05
query31	2.88	0.38	0.38
query32	3.27	0.46	0.46
query33	2.79	2.85	2.81
query34	16.97	4.42	4.40
query35	4.48	4.45	4.45
query36	0.65	0.47	0.46
query37	0.18	0.15	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.17	0.15	0.14
query41	0.09	0.05	0.04
query42	0.05	0.05	0.04
query43	0.04	0.03	0.04
Total cold run time: 109.27 s
Total hot run time: 30.52 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit b926fd225108b098a4e24ecb26d1d25292d98abf with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          57 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       14.1 seconds inserted 10000000 Rows, about 709K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 17, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Dec 27, 2024
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 3, 2025
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Jan 16, 2025
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Feb 6, 2025
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request Feb 13, 2025
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2025
morrySnow added a commit to morrySnow/incubator-doris that referenced this pull request May 22, 2025
morrySnow added a commit that referenced this pull request May 28, 2025
### What problem does this PR solve?

Related PR: #33591

Problem Summary:

when CTEAnchoer not exists in plan, cost based rewrite job could not get
correct cost from applied cbo rewrite rule plan.
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
### What problem does this PR solve?

Related PR: apache#33591

Problem Summary:

when CTEAnchoer not exists in plan, cost based rewrite job could not get
correct cost from applied cbo rewrite rule plan.
liutang123 pushed a commit to liutang123/doris that referenced this pull request Oct 11, 2025
### What problem does this PR solve?

Related PR: apache#33591

Problem Summary:

when CTEAnchoer not exists in plan, cost based rewrite job could not get
correct cost from applied cbo rewrite rule plan.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.11-merged dev/2.1.3-merged reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants