Skip to content

Conversation

@starocean999
Copy link
Contributor

Proposed changes

pick from master #33784

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@github-actions github-actions bot added area/planner Issues or PRs related to the query planner kind/test labels Apr 17, 2024
@doris-robot
Copy link

TPC-H: Total hot run time: 50301 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 09022a8f126df921655cff42fec6727aac63495d, data reload: false

------ Round 1 ----------------------------------
q1	17573	4338	4306	4306
q2	2030	147	136	136
q3	10290	1880	1950	1880
q4	10098	1284	1279	1279
q5	8501	3983	3935	3935
q6	226	120	119	119
q7	2051	1592	1577	1577
q8	9266	2723	2724	2723
q9	10764	11173	10896	10896
q10	8655	3490	3437	3437
q11	429	239	244	239
q12	468	289	294	289
q13	18359	3942	4015	3942
q14	345	320	335	320
q15	514	460	453	453
q16	684	591	588	588
q17	1133	941	958	941
q18	7328	6929	6941	6929
q19	1716	1573	1534	1534
q20	510	317	307	307
q21	4463	4097	4083	4083
q22	510	388	392	388
Total cold run time: 115913 ms
Total hot run time: 50301 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4287	4252	4272	4252
q2	323	218	220	218
q3	4136	4192	4134	4134
q4	2751	2753	2747	2747
q5	7270	7206	7120	7120
q6	233	116	114	114
q7	3252	2850	2815	2815
q8	4309	4464	4406	4406
q9	17061	16905	16868	16868
q10	4221	4275	4224	4224
q11	757	673	731	673
q12	1016	845	856	845
q13	4215	3791	3745	3745
q14	464	431	418	418
q15	496	453	457	453
q16	748	719	706	706
q17	3777	3742	3746	3742
q18	8769	8868	8777	8777
q19	1693	1715	1657	1657
q20	2434	2132	2113	2113
q21	8520	8557	8393	8393
q22	1036	929	898	898
Total cold run time: 81768 ms
Total hot run time: 79318 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 200368 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 09022a8f126df921655cff42fec6727aac63495d, data reload: false

query1	931	393	374	374
query2	6523	2142	2050	2050
query3	6913	201	200	200
query4	20747	18104	17934	17934
query5	19718	6483	6454	6454
query6	289	229	228	228
query7	4148	303	293	293
query8	254	245	233	233
query9	3132	2704	2611	2611
query10	394	306	293	293
query11	11296	10692	10626	10626
query12	119	73	73	73
query13	5572	656	607	607
query14	17597	13512	13277	13277
query15	354	221	243	221
query16	6470	265	254	254
query17	1744	1442	878	878
query18	2317	398	400	398
query19	204	139	142	139
query20	76	76	74	74
query21	184	96	94	94
query22	5286	4899	4936	4899
query23	32373	31879	31959	31879
query24	6935	6483	6429	6429
query25	500	418	419	418
query26	527	160	161	160
query27	1897	296	288	288
query28	6092	2268	2256	2256
query29	2896	2628	2782	2628
query30	241	163	159	159
query31	884	720	742	720
query32	66	59	61	59
query33	397	249	248	248
query34	845	474	465	465
query35	1084	876	914	876
query36	1304	1163	1188	1163
query37	94	62	58	58
query38	3070	2913	2903	2903
query39	1372	1320	1344	1320
query40	192	97	95	95
query41	35	32	32	32
query42	77	88	82	82
query43	725	645	562	562
query44	1121	718	716	716
query45	234	226	230	226
query46	1229	961	955	955
query47	1964	1750	1669	1669
query48	985	665	656	656
query49	609	357	368	357
query50	861	610	611	610
query51	4735	4706	4618	4618
query52	86	72	76	72
query53	437	309	304	304
query54	2605	2459	2479	2459
query55	84	74	80	74
query56	211	192	214	192
query57	1171	1091	1044	1044
query58	207	217	195	195
query59	3388	3187	3283	3187
query60	201	185	203	185
query61	83	84	80	80
query62	815	454	459	454
query63	469	325	325	325
query64	2511	1482	1341	1341
query65	3611	3545	3544	3544
query66	810	354	358	354
query67	16457	15215	15458	15215
query68	8473	653	676	653
query69	565	363	352	352
query70	1488	1315	1472	1315
query71	390	297	300	297
query72	6572	3395	3384	3384
query73	737	324	324	324
query74	6313	5827	5933	5827
query75	4829	3739	3683	3683
query76	5035	1151	1219	1151
query77	708	263	242	242
query78	13254	11877	11615	11615
query79	10334	648	655	648
query80	870	376	386	376
query81	503	227	228	227
query82	1326	98	99	98
query83	159	130	127	127
query84	255	65	66	65
query85	706	273	271	271
query86	325	283	288	283
query87	3264	3030	2968	2968
query88	5166	2350	2334	2334
query89	414	281	286	281
query90	1884	213	196	196
query91	148	113	116	113
query92	59	49	46	46
query93	6528	595	586	586
query94	676	195	206	195
query95	1110	1059	1060	1059
query96	654	325	336	325
query97	6460	6393	6416	6393
query98	184	170	166	166
query99	3057	990	926	926
Total cold run time: 313204 ms
Total hot run time: 200368 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.75 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 09022a8f126df921655cff42fec6727aac63495d, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.04
query4	1.79	0.06	0.06
query5	0.54	0.52	0.52
query6	1.27	0.61	0.62
query7	0.01	0.01	0.00
query8	0.05	0.02	0.02
query9	0.54	0.48	0.49
query10	0.54	0.53	0.53
query11	0.12	0.08	0.09
query12	0.13	0.09	0.10
query13	0.62	0.61	0.60
query14	0.77	0.79	0.79
query15	0.78	0.75	0.78
query16	0.36	0.35	0.36
query17	1.02	1.00	0.98
query18	0.25	0.23	0.22
query19	1.87	1.75	1.79
query20	0.02	0.01	0.01
query21	15.47	0.57	0.56
query22	2.08	2.37	1.55
query23	16.82	1.08	1.10
query24	5.50	0.99	1.12
query25	0.46	0.16	0.08
query26	0.68	0.16	0.17
query27	0.05	0.03	0.04
query28	8.15	0.72	0.79
query29	12.59	2.39	2.35
query30	0.62	0.51	0.52
query31	2.80	0.40	0.39
query32	3.35	0.49	0.51
query33	3.08	3.09	3.04
query34	15.29	4.79	4.80
query35	4.86	4.82	4.85
query36	1.05	1.02	1.01
query37	0.05	0.05	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.02
query40	0.17	0.14	0.14
query41	0.06	0.01	0.01
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 104.25 s
Total hot run time: 30.75 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 09022a8f126df921655cff42fec6727aac63495d with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.7 seconds inserted 10000000 Rows, about 483K ops/s

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50163 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit eb5a08757aaeff763ba1ba4fc08e33f9c93be7f4, data reload: false

------ Round 1 ----------------------------------
q1	17533	4318	4291	4291
q2	2022	151	140	140
q3	10280	1906	1949	1906
q4	10102	1269	1334	1269
q5	8406	3981	4007	3981
q6	235	122	122	122
q7	2038	1608	1590	1590
q8	9306	2709	2692	2692
q9	10763	11112	10841	10841
q10	8649	3500	3540	3500
q11	418	229	240	229
q12	464	298	303	298
q13	18372	3956	4092	3956
q14	343	328	315	315
q15	500	451	452	451
q16	687	586	598	586
q17	1143	967	985	967
q18	7220	6822	6934	6822
q19	1707	1561	1522	1522
q20	510	318	289	289
q21	4423	4152	4009	4009
q22	494	400	387	387
Total cold run time: 115615 ms
Total hot run time: 50163 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4340	4278	4317	4278
q2	324	223	228	223
q3	4142	4135	4149	4135
q4	2741	2736	2737	2736
q5	7283	7183	7170	7170
q6	231	120	117	117
q7	3250	2848	2829	2829
q8	4348	4470	4475	4470
q9	17006	16940	16926	16926
q10	4258	4286	4252	4252
q11	771	699	692	692
q12	1024	839	850	839
q13	5432	3722	3756	3722
q14	470	422	424	422
q15	500	463	446	446
q16	761	711	687	687
q17	3946	3815	3861	3815
q18	8838	8736	8829	8736
q19	1698	1710	1628	1628
q20	2358	2100	2102	2100
q21	8520	8438	8558	8438
q22	1071	972	982	972
Total cold run time: 83312 ms
Total hot run time: 79633 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 201474 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit eb5a08757aaeff763ba1ba4fc08e33f9c93be7f4, data reload: false

query1	928	390	378	378
query2	6515	2377	2142	2142
query3	6920	199	197	197
query4	21176	17983	17847	17847
query5	19717	6509	6472	6472
query6	283	212	229	212
query7	4149	297	289	289
query8	234	235	228	228
query9	3131	2724	2666	2666
query10	411	307	315	307
query11	11397	10734	10739	10734
query12	124	72	69	69
query13	5575	651	636	636
query14	17619	13636	13567	13567
query15	368	227	242	227
query16	6476	274	252	252
query17	1730	1448	881	881
query18	2319	398	396	396
query19	204	147	149	147
query20	74	78	75	75
query21	186	91	91	91
query22	5258	5070	4971	4971
query23	32671	31762	31875	31762
query24	7053	6494	6440	6440
query25	514	437	410	410
query26	523	159	155	155
query27	1911	288	293	288
query28	6190	2266	2235	2235
query29	2970	2686	2759	2686
query30	242	158	159	158
query31	902	758	706	706
query32	65	61	57	57
query33	386	254	250	250
query34	842	480	477	477
query35	1085	896	913	896
query36	1302	1210	1165	1165
query37	89	61	57	57
query38	3067	2959	2931	2931
query39	1360	1320	1313	1313
query40	201	98	92	92
query41	36	32	32	32
query42	88	86	89	86
query43	635	665	533	533
query44	1134	726	731	726
query45	236	226	226	226
query46	1244	992	979	979
query47	1947	1901	1681	1681
query48	982	674	643	643
query49	614	364	362	362
query50	858	597	594	594
query51	4774	4659	4685	4659
query52	86	83	89	83
query53	438	320	323	320
query54	2649	2453	2444	2444
query55	84	80	78	78
query56	199	208	186	186
query57	1188	1078	1060	1060
query58	216	198	192	192
query59	3377	3289	3252	3252
query60	216	181	206	181
query61	83	85	81	81
query62	840	487	475	475
query63	474	336	332	332
query64	2543	1423	1446	1423
query65	3681	3555	3580	3555
query66	788	354	356	354
query67	16218	15636	17098	15636
query68	8572	671	674	671
query69	557	317	346	317
query70	1815	1332	1394	1332
query71	376	288	322	288
query72	6540	3480	3420	3420
query73	726	323	326	323
query74	6290	5911	5801	5801
query75	5073	3777	3731	3731
query76	5217	1178	1179	1178
query77	880	251	258	251
query78	12614	11618	11497	11497
query79	9226	645	624	624
query80	1370	382	386	382
query81	484	241	227	227
query82	1674	94	99	94
query83	171	132	129	129
query84	253	70	68	68
query85	853	276	275	275
query86	334	329	311	311
query87	3213	3005	3013	3005
query88	4797	2359	2343	2343
query89	482	287	301	287
query90	1913	206	191	191
query91	155	112	118	112
query92	56	50	51	50
query93	6494	582	594	582
query94	716	208	204	204
query95	1134	1062	1077	1062
query96	634	343	324	324
query97	6421	6349	6421	6349
query98	181	183	173	173
query99	2986	838	842	838
Total cold run time: 314002 ms
Total hot run time: 201474 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit eb5a08757aaeff763ba1ba4fc08e33f9c93be7f4, data reload: false

query1	0.03	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.79	0.07	0.06
query5	0.53	0.52	0.52
query6	1.24	0.62	0.63
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.53	0.48	0.47
query10	0.53	0.55	0.53
query11	0.12	0.08	0.09
query12	0.11	0.10	0.09
query13	0.62	0.61	0.61
query14	0.78	0.78	0.81
query15	0.78	0.77	0.76
query16	0.37	0.37	0.36
query17	1.00	1.05	0.99
query18	0.21	0.27	0.24
query19	1.94	1.83	1.85
query20	0.01	0.01	0.01
query21	15.49	0.58	0.56
query22	2.05	2.99	1.55
query23	17.29	0.98	0.86
query24	5.93	0.99	1.41
query25	0.42	0.10	0.05
query26	0.63	0.16	0.17
query27	0.05	0.05	0.04
query28	6.82	0.76	0.78
query29	12.72	2.51	2.51
query30	0.60	0.52	0.52
query31	2.81	0.41	0.38
query32	3.36	0.49	0.49
query33	3.07	3.08	3.05
query34	15.26	4.80	4.80
query35	4.85	4.85	4.85
query36	1.06	1.00	1.01
query37	0.05	0.04	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.14	0.14
query41	0.07	0.02	0.01
query42	0.02	0.01	0.01
query43	0.02	0.01	0.02
Total cold run time: 103.71 s
Total hot run time: 30.87 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit eb5a08757aaeff763ba1ba4fc08e33f9c93be7f4 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       20.9 seconds inserted 10000000 Rows, about 478K ops/s

@xiaokang xiaokang marked this pull request as draft April 17, 2024 13:30
@xiaokang
Copy link
Contributor

master pr is draft

@starocean999 starocean999 marked this pull request as ready for review April 18, 2024 01:16
@starocean999 starocean999 merged commit 4954b1c into apache:branch-2.0 Apr 18, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/planner Issues or PRs related to the query planner kind/test

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants