Skip to content

Conversation

@starocean999
Copy link
Contributor

Proposed changes

pick from master #33990

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@starocean999
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49718 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d5958f2af7651f5e7e5b2f85cae6be22195ef802, data reload: false

------ Round 1 ----------------------------------
q1	17615	4399	4359	4359
q2	2061	148	139	139
q3	10486	1903	1897	1897
q4	10297	1279	1313	1279
q5	8382	3918	3942	3918
q6	224	120	121	120
q7	2026	1603	1573	1573
q8	9288	2692	2678	2678
q9	10814	10419	10477	10419
q10	8618	3506	3468	3468
q11	409	241	246	241
q12	462	293	308	293
q13	18330	3954	4013	3954
q14	343	314	316	314
q15	520	462	456	456
q16	703	590	579	579
q17	1119	969	958	958
q18	7192	6744	6976	6744
q19	1696	1534	1589	1534
q20	532	309	313	309
q21	4479	4102	4125	4102
q22	489	407	384	384
Total cold run time: 116085 ms
Total hot run time: 49718 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4376	4276	4301	4276
q2	314	226	219	219
q3	4154	4101	4078	4078
q4	2716	2702	2728	2702
q5	7226	7087	7144	7087
q6	230	118	115	115
q7	3194	2850	2832	2832
q8	4328	4424	4434	4424
q9	17088	16891	16870	16870
q10	4208	4220	4237	4220
q11	769	684	684	684
q12	1008	865	864	864
q13	6967	3723	3735	3723
q14	444	425	415	415
q15	504	448	506	448
q16	749	711	703	703
q17	3832	3816	3887	3816
q18	8699	8787	8800	8787
q19	1723	1685	1629	1629
q20	2387	2169	2103	2103
q21	8407	8358	8469	8358
q22	1036	962	909	909
Total cold run time: 84359 ms
Total hot run time: 79262 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203476 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d5958f2af7651f5e7e5b2f85cae6be22195ef802, data reload: false

query1	918	383	378	378
query2	6542	2817	2815	2815
query3	6918	198	200	198
query4	20875	17922	18097	17922
query5	19705	6528	6446	6446
query6	289	220	236	220
query7	4157	289	293	289
query8	254	262	228	228
query9	3114	2698	2615	2615
query10	406	293	305	293
query11	11476	10830	10777	10777
query12	120	71	69	69
query13	5581	673	674	673
query14	17484	13446	13144	13144
query15	357	234	234	234
query16	6456	268	251	251
query17	1587	1465	847	847
query18	2282	404	405	404
query19	209	153	144	144
query20	76	79	74	74
query21	186	102	91	91
query22	5219	5124	5090	5090
query23	32370	31739	31943	31739
query24	6950	6506	6435	6435
query25	527	417	408	408
query26	528	162	153	153
query27	1941	291	289	289
query28	6148	2391	2335	2335
query29	2788	2786	2769	2769
query30	237	158	163	158
query31	914	722	714	714
query32	66	58	57	57
query33	398	243	237	237
query34	863	457	473	457
query35	1124	906	927	906
query36	1390	1154	1169	1154
query37	88	60	55	55
query38	3073	2869	2964	2869
query39	1348	1320	1310	1310
query40	203	96	94	94
query41	43	35	34	34
query42	85	84	83	83
query43	707	695	724	695
query44	1137	740	736	736
query45	245	221	227	221
query46	1226	967	988	967
query47	1778	1819	1692	1692
query48	1010	708	702	702
query49	615	360	366	360
query50	854	582	608	582
query51	4756	4622	4713	4622
query52	94	74	72	72
query53	450	313	312	312
query54	2666	2452	2447	2447
query55	86	79	78	78
query56	197	202	205	202
query57	1244	1093	1136	1093
query58	215	188	199	188
query59	3989	4069	3811	3811
query60	197	177	185	177
query61	87	81	88	81
query62	820	468	437	437
query63	471	328	332	328
query64	2521	1606	1477	1477
query65	3630	3543	3544	3543
query66	761	378	377	377
query67	15855	17200	15733	15733
query68	6805	634	664	634
query69	557	349	338	338
query70	1529	1396	1518	1396
query71	401	293	308	293
query72	6402	3423	3410	3410
query73	718	323	307	307
query74	6404	6257	6272	6257
query75	4820	3724	3734	3724
query76	4359	1145	1223	1145
query77	519	251	249	249
query78	12839	11728	11896	11728
query79	8389	636	651	636
query80	2448	394	382	382
query81	529	237	228	228
query82	1539	95	100	95
query83	196	130	129	129
query84	257	68	67	67
query85	1354	295	289	289
query86	340	277	291	277
query87	3267	3031	2959	2959
query88	5271	2323	2325	2323
query89	346	304	281	281
query90	1764	208	210	208
query91	168	146	131	131
query92	54	52	50	50
query93	4326	550	588	550
query94	834	209	206	206
query95	1124	1052	1074	1052
query96	646	328	329	328
query97	6425	6356	6413	6356
query98	183	171	174	171
query99	2854	886	946	886
Total cold run time: 308573 ms
Total hot run time: 203476 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.56 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d5958f2af7651f5e7e5b2f85cae6be22195ef802, data reload: false

query1	0.03	0.02	0.02
query2	0.07	0.02	0.02
query3	0.24	0.05	0.04
query4	1.79	0.06	0.06
query5	0.54	0.52	0.52
query6	1.26	0.69	0.62
query7	0.02	0.02	0.01
query8	0.04	0.02	0.02
query9	0.54	0.49	0.50
query10	0.55	0.54	0.53
query11	0.12	0.08	0.08
query12	0.11	0.09	0.08
query13	0.62	0.61	0.62
query14	0.76	0.79	0.78
query15	0.77	0.77	0.76
query16	0.36	0.36	0.37
query17	1.01	1.01	0.97
query18	0.23	0.25	0.26
query19	1.94	1.83	1.79
query20	0.01	0.01	0.01
query21	15.53	0.56	0.55
query22	1.91	2.32	1.47
query23	17.20	1.02	0.94
query24	3.99	1.08	2.54
query25	0.35	0.10	0.05
query26	0.62	0.15	0.15
query27	0.04	0.04	0.05
query28	8.10	0.80	0.71
query29	12.72	2.30	2.35
query30	0.60	0.52	0.51
query31	2.80	0.38	0.37
query32	3.39	0.51	0.48
query33	3.06	3.08	3.04
query34	15.27	4.80	4.77
query35	4.86	4.81	4.84
query36	1.07	1.04	1.01
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.17	0.14	0.14
query41	0.07	0.01	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.02
Total cold run time: 102.92 s
Total hot run time: 30.56 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d5958f2af7651f5e7e5b2f85cae6be22195ef802 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@carlvinhust2012
Copy link
Contributor

can you add some test case?

@starocean999
Copy link
Contributor Author

can you add some test case?

I found the NPE in some unexpected scenario. Like user defined var is not replaced by literal in right place would cause the translation fail. After the bug is fixed, there is not NPE now. So this is just a fail-safe for unexpected fail in future.

@starocean999 starocean999 merged commit a793ca8 into apache:branch-2.0 Apr 26, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants