Skip to content

Conversation

@morrySnow
Copy link
Contributor

pick from master #39499

since legacy planner will cost more time to plan, fallback will be worse than throw exception directly

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50258 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18d85649193d7b5c537769c7bea14703ae1d9012, data reload: false

------ Round 1 ----------------------------------
q1	18015	4427	4396	4396
q2	2073	198	202	198
q3	10240	1918	1962	1918
q4	10132	1284	1358	1284
q5	8448	4101	3862	3862
q6	262	143	146	143
q7	2081	1656	1655	1655
q8	9589	2775	2761	2761
q9	13903	10344	10252	10252
q10	8679	3526	3534	3526
q11	428	277	267	267
q12	503	339	339	339
q13	18359	3958	4046	3958
q14	384	339	356	339
q15	559	503	499	499
q16	702	610	616	610
q17	1156	957	902	902
q18	7286	6870	6854	6854
q19	1771	1606	1591	1591
q20	539	344	336	336
q21	4452	4223	4134	4134
q22	533	440	434	434
Total cold run time: 120094 ms
Total hot run time: 50258 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4379	4291	4310	4291
q2	360	264	267	264
q3	4186	4129	4179	4129
q4	2771	2758	2779	2758
q5	7209	7098	7109	7098
q6	263	137	144	137
q7	3283	2886	2874	2874
q8	4400	4514	4520	4514
q9	16894	16839	16861	16839
q10	4298	4253	4252	4252
q11	796	734	709	709
q12	1066	883	871	871
q13	7169	3729	3771	3729
q14	480	448	444	444
q15	544	502	498	498
q16	772	708	711	708
q17	3821	3939	3831	3831
q18	8804	8744	8849	8744
q19	1790	1758	1699	1699
q20	2410	2188	2159	2159
q21	8518	8546	8610	8546
q22	1074	957	997	957
Total cold run time: 85287 ms
Total hot run time: 80051 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 209042 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18d85649193d7b5c537769c7bea14703ae1d9012, data reload: false

query1	975	425	409	409
query2	6765	2272	2197	2197
query3	6950	227	222	222
query4	20882	18091	18266	18091
query5	19945	6672	6813	6672
query6	319	257	276	257
query7	4186	321	335	321
query8	469	466	471	466
query9	3239	2835	2765	2765
query10	477	370	369	369
query11	11374	10774	10933	10774
query12	154	110	106	106
query13	5659	709	718	709
query14	17949	13826	13301	13301
query15	393	271	244	244
query16	6824	334	308	308
query17	1652	1468	940	940
query18	2346	481	480	480
query19	255	182	184	182
query20	115	114	116	114
query21	236	147	155	147
query22	5024	5054	5016	5016
query23	32689	31822	32072	31822
query24	7096	6557	6602	6557
query25	567	492	468	468
query26	562	189	189	189
query27	1793	337	334	334
query28	6079	2435	2393	2393
query29	2916	2911	2879	2879
query30	285	210	207	207
query31	1025	851	834	834
query32	103	91	90	90
query33	498	345	329	329
query34	920	516	542	516
query35	1160	942	988	942
query36	1361	1154	1261	1154
query37	119	91	90	90
query38	3081	2975	2976	2975
query39	1478	1427	1434	1427
query40	250	148	148	148
query41	156	159	153	153
query42	106	112	111	111
query43	759	653	642	642
query44	1178	794	793	793
query45	279	262	270	262
query46	1270	1022	1025	1022
query47	1876	1738	1775	1738
query48	1044	760	742	742
query49	846	584	609	584
query50	938	708	688	688
query51	4819	4744	4744	4744
query52	119	112	131	112
query53	514	379	374	374
query54	2744	2528	2519	2519
query55	115	88	104	88
query56	291	308	293	293
query57	1264	1247	1205	1205
query58	308	293	307	293
query59	3598	3482	3277	3277
query60	287	271	291	271
query61	160	161	162	161
query62	792	647	529	529
query63	544	402	417	402
query64	2600	1615	1546	1546
query65	3655	3583	3609	3583
query66	1242	819	834	819
query67	15453	15362	15853	15362
query68	9167	710	708	708
query69	650	425	408	408
query70	1685	1611	1474	1474
query71	472	363	371	363
query72	6638	3547	3534	3534
query73	778	363	367	363
query74	6351	5899	5908	5899
query75	5503	3834	3802	3802
query76	5626	1235	1239	1235
query77	1089	429	420	420
query78	12741	12706	12171	12171
query79	13094	684	692	684
query80	945	548	546	546
query81	538	277	273	273
query82	1154	129	130	129
query83	269	215	214	214
query84	283	98	99	98
query85	856	413	410	410
query86	390	325	324	324
query87	3288	3106	3063	3063
query88	6135	2513	2518	2513
query89	472	341	334	334
query90	2023	255	255	255
query91	201	165	157	157
query92	88	84	87	84
query93	6316	637	619	619
query94	776	254	240	240
query95	1153	1094	1099	1094
query96	662	346	356	346
query97	6462	6515	6454	6454
query98	228	215	210	210
query99	2739	861	966	861
Total cold run time: 323899 ms
Total hot run time: 209042 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.84 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 18d85649193d7b5c537769c7bea14703ae1d9012, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.03	0.03
query3	0.25	0.06	0.05
query4	1.78	0.07	0.08
query5	0.53	0.52	0.54
query6	1.33	0.62	0.64
query7	0.01	0.01	0.02
query8	0.04	0.03	0.02
query9	0.52	0.49	0.48
query10	0.55	0.55	0.54
query11	0.13	0.10	0.09
query12	0.13	0.10	0.10
query13	0.63	0.62	0.63
query14	0.77	0.79	0.80
query15	0.78	0.78	0.77
query16	0.37	0.39	0.36
query17	1.01	1.03	1.03
query18	0.23	0.25	0.26
query19	1.97	1.88	1.86
query20	0.02	0.02	0.01
query21	15.46	0.55	0.57
query22	2.40	2.03	1.60
query23	17.46	1.00	1.01
query24	5.28	1.31	1.33
query25	0.39	0.10	0.05
query26	0.64	0.16	0.17
query27	0.04	0.04	0.04
query28	7.20	0.86	0.85
query29	12.76	2.34	2.30
query30	0.79	0.71	0.74
query31	2.82	0.41	0.40
query32	3.34	0.52	0.51
query33	3.10	3.12	3.07
query34	15.25	4.81	4.80
query35	4.86	4.87	4.86
query36	1.09	1.02	1.03
query37	0.08	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.03	0.03
query40	0.19	0.16	0.18
query41	0.08	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.04
Total cold run time: 104.55 s
Total hot run time: 31.84 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 18d85649193d7b5c537769c7bea14703ae1d9012 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       21.7 seconds inserted 10000000 Rows, about 460K ops/s

…che#39499)

pick from master apache#39499

since legacy planner will cost more time to plan, fallback will be worse
than throw exception directly
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50151 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a85beb177398ae13f8c1b13057e38c006e44d2d6, data reload: false

------ Round 1 ----------------------------------
q1	18232	4382	4327	4327
q2	2091	202	190	190
q3	10171	1942	1984	1942
q4	10130	1271	1356	1271
q5	8532	3888	3947	3888
q6	271	142	143	142
q7	2070	1650	1655	1650
q8	9585	2762	2732	2732
q9	13997	10248	10173	10173
q10	8634	3560	3511	3511
q11	430	277	272	272
q12	504	335	332	332
q13	18347	4008	4050	4008
q14	373	358	363	358
q15	548	501	504	501
q16	674	606	606	606
q17	1134	907	922	907
q18	7204	7007	6911	6911
q19	1733	1584	1539	1539
q20	563	339	344	339
q21	4422	4179	4118	4118
q22	546	434	436	434
Total cold run time: 120191 ms
Total hot run time: 50151 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4325	4308	4293	4293
q2	355	262	264	262
q3	4155	4131	4127	4127
q4	2763	2765	2763	2763
q5	7135	7177	7126	7126
q6	262	139	138	138
q7	3240	2921	2963	2921
q8	4416	4477	4493	4477
q9	16946	16806	16957	16806
q10	4274	4256	4292	4256
q11	761	703	706	703
q12	1058	901	860	860
q13	6801	3734	3749	3734
q14	473	454	445	445
q15	546	504	517	504
q16	769	716	701	701
q17	3818	3821	3874	3821
q18	8876	8777	8733	8733
q19	1779	1747	1705	1705
q20	2389	2181	2129	2129
q21	8606	8465	8469	8465
q22	1039	964	975	964
Total cold run time: 84786 ms
Total hot run time: 79933 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 209779 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit a85beb177398ae13f8c1b13057e38c006e44d2d6, data reload: false

query1	975	417	413	413
query2	6759	2308	2208	2208
query3	6947	234	220	220
query4	19886	18227	18065	18065
query5	19944	6740	6786	6740
query6	321	255	272	255
query7	4186	334	342	334
query8	502	439	477	439
query9	3293	2831	2777	2777
query10	479	393	378	378
query11	11307	10840	10791	10791
query12	161	118	113	113
query13	5680	719	729	719
query14	18259	14099	13866	13866
query15	386	273	254	254
query16	6497	327	305	305
query17	1661	1468	952	952
query18	2342	507	490	490
query19	239	182	183	182
query20	121	113	116	113
query21	243	151	148	148
query22	5230	5081	5004	5004
query23	32596	32324	31969	31969
query24	7091	6554	6563	6554
query25	562	493	495	493
query26	566	195	192	192
query27	1840	344	336	336
query28	6028	2427	2384	2384
query29	2897	2883	2886	2883
query30	285	208	207	207
query31	1010	847	839	839
query32	103	92	93	92
query33	485	348	328	328
query34	917	544	557	544
query35	1174	1010	1010	1010
query36	1336	1274	1237	1237
query37	124	90	92	90
query38	3067	3004	2927	2927
query39	1506	1428	1444	1428
query40	261	148	152	148
query41	171	155	154	154
query42	110	114	112	112
query43	660	679	720	679
query44	1175	784	787	784
query45	282	273	270	270
query46	1277	1004	1053	1004
query47	1866	1914	2054	1914
query48	1058	742	759	742
query49	828	582	587	582
query50	953	704	704	704
query51	4752	4781	4710	4710
query52	118	107	119	107
query53	510	379	385	379
query54	2736	2523	2571	2523
query55	104	102	117	102
query56	339	299	297	297
query57	1277	1180	1122	1122
query58	331	307	308	307
query59	3469	3281	3257	3257
query60	291	289	277	277
query61	163	159	157	157
query62	907	561	574	561
query63	557	405	414	405
query64	2763	1695	1621	1621
query65	3710	3650	3637	3637
query66	1203	818	818	818
query67	15719	16818	15556	15556
query68	9197	699	721	699
query69	641	414	412	412
query70	1569	1391	1486	1391
query71	479	358	395	358
query72	6632	3585	3547	3547
query73	785	369	369	369
query74	6388	5904	5911	5904
query75	5323	3809	3921	3809
query76	5649	1097	1262	1097
query77	1130	429	424	424
query78	12731	11768	11623	11623
query79	9134	692	690	690
query80	1242	555	560	555
query81	541	283	280	280
query82	1288	125	127	125
query83	265	219	220	219
query84	293	99	100	99
query85	973	409	415	409
query86	369	334	319	319
query87	3226	3106	3051	3051
query88	4978	2545	2545	2545
query89	480	345	326	326
query90	2006	257	257	257
query91	197	159	167	159
query92	92	85	83	83
query93	6117	629	594	594
query94	740	256	255	255
query95	1177	1108	1118	1108
query96	668	350	356	350
query97	6478	6394	6425	6394
query98	218	207	201	201
query99	3161	924	919	919
Total cold run time: 318762 ms
Total hot run time: 209779 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 32.2 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a85beb177398ae13f8c1b13057e38c006e44d2d6, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.03	0.02
query3	0.25	0.05	0.05
query4	1.78	0.06	0.06
query5	0.54	0.52	0.54
query6	1.24	0.62	0.61
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.52	0.49	0.49
query10	0.54	0.55	0.54
query11	0.13	0.10	0.10
query12	0.14	0.10	0.10
query13	0.63	0.61	0.61
query14	0.79	0.79	0.78
query15	0.80	0.77	0.78
query16	0.38	0.38	0.37
query17	1.05	1.04	1.02
query18	0.21	0.27	0.24
query19	1.91	1.87	1.86
query20	0.02	0.01	0.02
query21	15.46	0.57	0.57
query22	2.09	2.10	2.03
query23	17.15	0.99	1.01
query24	6.11	1.45	1.53
query25	0.42	0.07	0.05
query26	0.75	0.15	0.16
query27	0.05	0.06	0.04
query28	6.13	0.74	0.76
query29	12.80	2.29	2.31
query30	0.75	0.69	0.70
query31	2.84	0.40	0.38
query32	3.37	0.51	0.50
query33	3.10	3.08	3.10
query34	15.25	4.78	4.80
query35	4.86	4.83	4.85
query36	1.08	1.04	1.03
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.03	0.03
query40	0.18	0.16	0.17
query41	0.08	0.03	0.03
query42	0.04	0.04	0.03
query43	0.04	0.04	0.04
Total cold run time: 103.8 s
Total hot run time: 32.2 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit a85beb177398ae13f8c1b13057e38c006e44d2d6 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@morrySnow morrySnow merged commit 9c455bf into apache:branch-2.0 Aug 22, 2024
@morrySnow morrySnow deleted the 2.0_39499 branch August 22, 2024 03:21
mongo360 pushed a commit to mongo360/doris that referenced this pull request Dec 11, 2024
…che#39499) (apache#39722)

pick from master apache#39499

since legacy planner will cost more time to plan, fallback will be worse
than throw exception directly
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants