Skip to content

Conversation

@ByteYue
Copy link
Contributor

@ByteYue ByteYue commented Apr 15, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@ByteYue
Copy link
Contributor Author

ByteYue commented Apr 15, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 39013 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit be520f07a3e738c3429e53ffd9a45cf030d441bb, data reload: false

------ Round 1 ----------------------------------
q1	17667	4528	4383	4383
q2	2611	196	209	196
q3	10863	1201	1212	1201
q4	10390	870	774	774
q5	7617	2753	2681	2681
q6	222	136	135	135
q7	1028	615	624	615
q8	9487	2104	2069	2069
q9	7500	6740	6591	6591
q10	8527	3547	3535	3535
q11	454	226	229	226
q12	384	221	216	216
q13	18788	2960	2953	2953
q14	270	225	234	225
q15	538	483	489	483
q16	561	390	400	390
q17	965	683	714	683
q18	7429	6787	6831	6787
q19	5434	1520	1512	1512
q20	659	314	311	311
q21	3611	2746	2808	2746
q22	393	308	301	301
Total cold run time: 115398 ms
Total hot run time: 39013 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4392	4238	4232	4232
q2	376	274	267	267
q3	3034	2706	2797	2706
q4	1886	1605	1514	1514
q5	5307	5365	5310	5310
q6	209	126	122	122
q7	2260	1913	1844	1844
q8	3177	3356	3362	3356
q9	8615	8574	8690	8574
q10	3905	3669	3647	3647
q11	577	468	483	468
q12	760	578	586	578
q13	17430	2921	2988	2921
q14	308	271	280	271
q15	511	481	471	471
q16	476	453	430	430
q17	1799	1484	1492	1484
q18	7612	7508	7506	7506
q19	1662	1513	1499	1499
q20	1981	1755	1737	1737
q21	4927	4828	4764	4764
q22	531	478	475	475
Total cold run time: 71735 ms
Total hot run time: 54176 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit be520f07a3e738c3429e53ffd9a45cf030d441bb, data reload: false

query1	910	379	363	363
query2	6480	2441	2311	2311
query3	6658	203	214	203
query4	25163	21376	21319	21319
query5	4172	411	396	396
query6	267	179	167	167
query7	4592	285	283	283
query8	225	173	172	172
query9	8432	2320	2299	2299
query10	574	245	254	245
query11	14696	14226	14224	14224
query12	140	96	84	84
query13	1633	365	361	361
query14	10097	7498	7538	7498
query15	260	183	182	182
query16	8123	254	252	252
query17	1943	574	551	551
query18	2091	287	269	269
query19	203	143	149	143
query20	92	82	86	82
query21	196	121	126	121
query22	4990	4833	4892	4833
query23	33722	33237	33229	33229
query24	11499	2973	2905	2905
query25	654	365	374	365
query26	1712	152	143	143
query27	3020	313	308	308
query28	7455	1999	1974	1974
query29	982	599	612	599
query30	313	174	173	173
query31	972	719	729	719
query32	94	54	55	54
query33	744	274	247	247
query34	1050	492	469	469
query35	831	715	684	684
query36	1009	888	884	884
query37	132	66	69	66
query38	3330	3184	3189	3184
query39	1598	1549	1524	1524
query40	277	122	131	122
query41	45	42	44	42
query42	100	95	102	95
query43	532	539	550	539
query44	1286	718	732	718
query45	285	258	258	258
query46	1084	721	718	718
query47	1920	1871	1834	1834
query48	371	292	287	287
query49	1156	374	361	361
query50	749	367	371	367
query51	6696	6649	6658	6649
query52	105	86	84	84
query53	344	276	280	276
query54	294	254	226	226
query55	73	73	71	71
query56	244	222	222	222
query57	1215	1125	1113	1113
query58	227	191	190	190
query59	3279	3070	3178	3070
query60	263	235	228	228
query61	89	86	87	86
query62	651	437	456	437
query63	298	278	279	278
query64	6275	3862	3788	3788
query65	3151	3081	3028	3028
query66	1398	338	341	338
query67	15306	14918	14923	14918
query68	5334	532	540	532
query69	526	310	302	302
query70	1244	1164	1143	1143
query71	458	270	260	260
query72	6763	2598	2456	2456
query73	721	315	314	314
query74	6912	6446	6346	6346
query75	3951	2626	2606	2606
query76	3943	1006	966	966
query77	639	251	266	251
query78	11125	10220	10107	10107
query79	3330	511	500	500
query80	2074	429	436	429
query81	526	246	245	245
query82	1543	107	97	97
query83	333	164	167	164
query84	259	86	80	80
query85	1645	330	254	254
query86	474	321	299	299
query87	3524	3291	3282	3282
query88	4423	2313	2314	2313
query89	469	376	364	364
query90	2011	180	177	177
query91	125	94	97	94
query92	59	45	46	45
query93	4513	508	496	496
query94	1227	181	175	175
query95	388	292	283	283
query96	591	274	259	259
query97	3151	2916	2968	2916
query98	253	215	216	215
query99	1269	864	850	850
Total cold run time: 292570 ms
Total hot run time: 183048 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.06 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit be520f07a3e738c3429e53ffd9a45cf030d441bb, data reload: false

query1	0.04	0.03	0.03
query2	0.08	0.04	0.04
query3	0.23	0.06	0.06
query4	1.66	0.08	0.07
query5	0.53	0.50	0.51
query6	1.48	0.72	0.72
query7	0.02	0.01	0.02
query8	0.05	0.04	0.04
query9	0.55	0.50	0.51
query10	0.55	0.57	0.55
query11	0.16	0.11	0.11
query12	0.14	0.12	0.12
query13	0.61	0.59	0.58
query14	0.77	0.77	0.76
query15	0.82	0.81	0.82
query16	0.37	0.36	0.36
query17	1.03	0.95	1.01
query18	0.25	0.22	0.21
query19	1.75	1.65	1.68
query20	0.02	0.01	0.01
query21	15.47	0.65	0.64
query22	4.28	7.74	1.77
query23	18.30	1.39	1.18
query24	1.78	0.26	0.21
query25	0.15	0.08	0.08
query26	0.25	0.16	0.16
query27	0.07	0.08	0.08
query28	13.36	1.00	0.97
query29	12.59	3.27	3.27
query30	0.26	0.06	0.06
query31	2.86	0.39	0.37
query32	3.29	0.47	0.46
query33	2.80	2.86	2.83
query34	17.12	4.35	4.40
query35	4.47	4.49	4.51
query36	0.65	0.50	0.46
query37	0.18	0.15	0.16
query38	0.14	0.14	0.14
query39	0.05	0.04	0.04
query40	0.17	0.14	0.14
query41	0.10	0.05	0.06
query42	0.06	0.06	0.05
query43	0.04	0.04	0.03
Total cold run time: 109.55 s
Total hot run time: 30.06 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit be520f07a3e738c3429e53ffd9a45cf030d441bb with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.4 seconds inserted 10000000 Rows, about 746K ops/s

platoneko
platoneko previously approved these changes Apr 16, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 16, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@ByteYue
Copy link
Contributor Author

ByteYue commented Apr 16, 2024

run buildall

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 16, 2024
@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38882 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit fa9b0d09d25e6b70a3158dddea881a031160563c, data reload: false

------ Round 1 ----------------------------------
q1	17899	5088	4443	4443
q2	2535	199	193	193
q3	11034	1206	1222	1206
q4	11091	843	757	757
q5	7862	2783	2698	2698
q6	223	137	142	137
q7	1016	604	592	592
q8	9207	2052	2048	2048
q9	7438	6609	6512	6512
q10	8442	3537	3501	3501
q11	437	232	229	229
q12	407	221	219	219
q13	18519	2943	2938	2938
q14	265	231	233	231
q15	517	481	476	476
q16	520	406	381	381
q17	972	740	774	740
q18	7423	6788	6682	6682
q19	7087	1528	1508	1508
q20	650	324	312	312
q21	3497	2771	2806	2771
q22	374	308	321	308
Total cold run time: 117415 ms
Total hot run time: 38882 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4360	4238	4316	4238
q2	384	265	268	265
q3	2964	2748	2739	2739
q4	1890	1558	1599	1558
q5	5346	5365	5301	5301
q6	209	124	127	124
q7	2257	1896	1864	1864
q8	3210	3378	3344	3344
q9	8611	8541	8544	8541
q10	3889	3743	3690	3690
q11	580	474	485	474
q12	758	596	571	571
q13	17090	2952	2964	2952
q14	299	266	274	266
q15	522	473	465	465
q16	467	409	415	409
q17	1766	1491	1472	1472
q18	7653	7476	7481	7476
q19	1640	1532	1478	1478
q20	1924	1749	1758	1749
q21	4836	4764	4650	4650
q22	562	457	455	455
Total cold run time: 71217 ms
Total hot run time: 54081 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 184055 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit fa9b0d09d25e6b70a3158dddea881a031160563c, data reload: false

query1	893	377	365	365
query2	6479	2559	2295	2295
query3	6649	205	209	205
query4	23533	21308	21338	21308
query5	4185	423	417	417
query6	263	177	178	177
query7	4582	304	290	290
query8	230	181	173	173
query9	8497	2371	2353	2353
query10	591	255	256	255
query11	14691	14277	14266	14266
query12	141	95	89	89
query13	1651	362	364	362
query14	10212	7773	7331	7331
query15	255	189	186	186
query16	8172	274	259	259
query17	1967	596	564	564
query18	2107	277	275	275
query19	342	159	156	156
query20	91	85	84	84
query21	194	132	129	129
query22	4953	4824	4775	4775
query23	33766	33062	33091	33062
query24	11678	3006	3053	3006
query25	641	364	372	364
query26	1753	159	156	156
query27	2968	313	313	313
query28	7599	2013	2001	2001
query29	1020	602	617	602
query30	282	181	177	177
query31	950	730	737	730
query32	99	55	56	55
query33	759	274	247	247
query34	1053	483	486	483
query35	845	693	716	693
query36	1025	933	902	902
query37	276	74	78	74
query38	3391	3220	3265	3220
query39	1590	1535	1522	1522
query40	270	127	126	126
query41	47	44	43	43
query42	101	97	95	95
query43	562	517	544	517
query44	1196	713	719	713
query45	282	266	259	259
query46	1076	727	748	727
query47	1921	1847	1834	1834
query48	367	300	290	290
query49	1147	376	369	369
query50	756	386	401	386
query51	6711	6614	6472	6472
query52	101	92	88	88
query53	350	282	286	282
query54	317	252	223	223
query55	74	71	71	71
query56	254	230	228	228
query57	1209	1133	1157	1133
query58	223	202	195	195
query59	3315	2995	2976	2976
query60	248	239	234	234
query61	91	89	90	89
query62	672	448	436	436
query63	311	280	279	279
query64	6381	3967	3853	3853
query65	3086	3032	3044	3032
query66	1369	357	335	335
query67	15301	14945	14772	14772
query68	5251	536	554	536
query69	521	299	300	299
query70	1259	1193	1155	1155
query71	1399	1266	1270	1266
query72	6475	2624	2472	2472
query73	726	331	324	324
query74	6888	6529	6381	6381
query75	3599	2665	2583	2583
query76	3890	931	992	931
query77	591	264	265	264
query78	10918	10119	10097	10097
query79	3068	539	529	529
query80	1964	428	427	427
query81	533	241	245	241
query82	741	97	99	97
query83	327	167	166	166
query84	264	86	84	84
query85	1861	356	267	267
query86	465	296	294	294
query87	3443	3295	3317	3295
query88	4656	2428	2416	2416
query89	478	376	367	367
query90	1993	189	193	189
query91	127	97	99	97
query92	67	47	46	46
query93	4564	511	503	503
query94	1233	183	181	181
query95	395	308	301	301
query96	599	273	272	272
query97	3148	2932	2941	2932
query98	245	222	220	220
query99	1257	845	865	845
Total cold run time: 291056 ms
Total hot run time: 184055 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.15 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit fa9b0d09d25e6b70a3158dddea881a031160563c, data reload: false

query1	0.03	0.03	0.03
query2	0.08	0.04	0.04
query3	0.22	0.05	0.05
query4	1.68	0.09	0.08
query5	0.48	0.48	0.50
query6	1.48	0.72	0.72
query7	0.02	0.02	0.01
query8	0.04	0.04	0.04
query9	0.54	0.49	0.49
query10	0.53	0.56	0.53
query11	0.15	0.11	0.12
query12	0.15	0.11	0.12
query13	0.59	0.59	0.60
query14	0.77	0.76	0.78
query15	0.83	0.81	0.82
query16	0.37	0.35	0.36
query17	1.02	0.96	0.98
query18	0.20	0.23	0.25
query19	1.74	1.66	1.69
query20	0.02	0.01	0.02
query21	15.42	0.66	0.66
query22	4.37	6.35	2.68
query23	18.32	1.36	1.23
query24	1.79	0.24	0.24
query25	0.14	0.08	0.08
query26	0.27	0.17	0.17
query27	0.07	0.07	0.07
query28	13.27	1.01	0.98
query29	12.58	3.31	3.32
query30	0.26	0.07	0.05
query31	2.85	0.39	0.39
query32	3.29	0.47	0.47
query33	2.81	2.83	2.80
query34	16.94	4.48	4.45
query35	4.54	4.48	4.53
query36	0.64	0.47	0.45
query37	0.19	0.16	0.15
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.18	0.14	0.14
query41	0.09	0.05	0.05
query42	0.05	0.05	0.04
query43	0.05	0.04	0.03
Total cold run time: 109.26 s
Total hot run time: 31.15 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit fa9b0d09d25e6b70a3158dddea881a031160563c with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 17, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@dataroaring dataroaring merged commit 14439d2 into apache:master Apr 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. meta-change reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants