Skip to content

Conversation

@mrhhsg
Copy link
Member

@mrhhsg mrhhsg commented Mar 26, 2024

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@mrhhsg
Copy link
Member Author

mrhhsg commented Mar 26, 2024

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@xiaokang
Copy link
Contributor

#32623

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (8046/21285)
Line Coverage: 29.47% (65716/223012)
Region Coverage: 28.93% (33814/116899)
Branch Coverage: 24.78% (17359/70064)
Coverage Report: http://coverage.selectdb-in.cc/coverage/6c3822fc0ec51d01675d0ce991711a60a0a69077_6c3822fc0ec51d01675d0ce991711a60a0a69077/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49982 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6c3822fc0ec51d01675d0ce991711a60a0a69077, data reload: false

------ Round 1 ----------------------------------
q1	17614	4362	4349	4349
q2	2125	151	143	143
q3	10653	1873	1931	1873
q4	10378	1237	1455	1237
q5	8379	3927	3965	3927
q6	235	125	124	124
q7	2019	1600	1599	1599
q8	9281	2697	2704	2697
q9	10749	10559	10532	10532
q10	8880	3552	3589	3552
q11	423	245	244	244
q12	465	310	306	306
q13	18853	3916	4014	3916
q14	349	315	329	315
q15	510	464	455	455
q16	690	590	580	580
q17	1129	957	957	957
q18	7258	6896	6940	6896
q19	1662	1569	1527	1527
q20	538	290	307	290
q21	4413	4063	4073	4063
q22	498	408	400	400
Total cold run time: 117101 ms
Total hot run time: 49982 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4313	4302	4302
q2	318	228	223	223
q3	4113	4152	4170	4152
q4	2755	2739	2751	2739
q5	7264	7196	7171	7171
q6	232	119	117	117
q7	3231	2813	2819	2813
q8	4378	4429	4462	4429
q9	17061	16818	16820	16818
q10	4205	4276	4217	4217
q11	812	675	707	675
q12	1006	833	836	833
q13	7065	3752	3708	3708
q14	454	433	420	420
q15	511	444	442	442
q16	767	695	716	695
q17	3866	3899	3845	3845
q18	8759	8872	8728	8728
q19	1703	1699	1661	1661
q20	2381	2160	2121	2121
q21	8522	8449	8506	8449
q22	1041	1017	954	954
Total cold run time: 84778 ms
Total hot run time: 79512 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 202430 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6c3822fc0ec51d01675d0ce991711a60a0a69077, data reload: false

query1	949	394	379	379
query2	6542	2280	2072	2072
query3	6919	201	203	201
query4	20062	18138	18064	18064
query5	19739	6609	6604	6604
query6	309	217	223	217
query7	4349	309	305	305
query8	244	254	239	239
query9	3072	2674	2587	2587
query10	421	283	295	283
query11	11396	10774	10715	10715
query12	129	82	73	73
query13	5590	661	652	652
query14	18262	13891	13723	13723
query15	368	256	234	234
query16	6437	266	262	262
query17	1511	1497	868	868
query18	2294	406	405	405
query19	202	141	147	141
query20	86	76	73	73
query21	188	102	95	95
query22	5199	4964	5077	4964
query23	32515	32175	32119	32119
query24	6786	6526	6602	6526
query25	523	435	424	424
query26	523	164	159	159
query27	1884	297	309	297
query28	6125	2239	2198	2198
query29	2898	2810	2637	2637
query30	258	165	163	163
query31	906	747	733	733
query32	71	53	64	53
query33	377	252	239	239
query34	870	488	475	475
query35	1104	899	912	899
query36	1304	1092	1201	1092
query37	93	62	59	59
query38	3070	2944	2916	2916
query39	1381	1347	1337	1337
query40	207	98	102	98
query41	40	32	31	31
query42	95	84	83	83
query43	725	629	543	543
query44	1147	732	723	723
query45	242	226	225	225
query46	1252	983	1026	983
query47	1986	1753	1732	1732
query48	1005	697	674	674
query49	613	355	361	355
query50	860	594	623	594
query51	4772	4626	4657	4626
query52	87	92	84	84
query53	450	320	319	319
query54	2673	2451	2482	2451
query55	90	77	88	77
query56	217	209	186	186
query57	1281	1102	1024	1024
query58	200	209	198	198
query59	3563	3170	3222	3170
query60	200	210	186	186
query61	84	84	87	84
query62	826	471	498	471
query63	467	338	333	333
query64	2460	1508	1294	1294
query65	3717	3602	3587	3587
query66	760	369	354	354
query67	15809	15559	15839	15559
query68	8247	679	677	677
query69	581	359	354	354
query70	1909	1379	1561	1379
query71	421	299	317	299
query72	6669	3455	3418	3418
query73	742	332	326	326
query74	6304	5860	5844	5844
query75	4577	3735	3784	3735
query76	4555	1178	1214	1178
query77	537	248	259	248
query78	12667	11830	11772	11772
query79	7935	658	652	652
query80	1688	408	409	408
query81	509	248	237	237
query82	1556	107	100	100
query83	178	135	132	132
query84	259	68	68	68
query85	1250	285	279	279
query86	361	289	336	289
query87	3179	2977	3035	2977
query88	5084	2347	2347	2347
query89	347	303	302	302
query90	1800	212	209	209
query91	153	116	116	116
query92	57	50	49	49
query93	3992	577	594	577
query94	879	197	204	197
query95	1116	1058	1074	1058
query96	651	316	337	316
query97	6528	6339	6419	6339
query98	177	179	171	171
query99	2988	1020	851	851
Total cold run time: 308680 ms
Total hot run time: 202430 ms

@mrhhsg
Copy link
Member Author

mrhhsg commented Mar 26, 2024

run buildall

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 6c3822fc0ec51d01675d0ce991711a60a0a69077 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.4 seconds inserted 10000000 Rows, about 446K ops/s

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 50309 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c083702939b54147d1418eadad36705008400a84, data reload: false

------ Round 1 ----------------------------------
q1	17905	4465	4396	4396
q2	2062	159	151	151
q3	10322	1941	1958	1941
q4	10426	1253	1346	1253
q5	8652	3982	4022	3982
q6	234	122	124	122
q7	2084	1631	1611	1611
q8	9270	2733	2767	2733
q9	10898	10774	10382	10382
q10	8693	3537	3567	3537
q11	397	240	252	240
q12	467	305	302	302
q13	18354	3987	4069	3987
q14	356	331	332	331
q15	508	455	472	455
q16	691	591	582	582
q17	1148	975	1000	975
q18	7240	6956	7086	6956
q19	1703	1555	1557	1555
q20	535	319	306	306
q21	4527	4120	4125	4120
q22	514	392	399	392
Total cold run time: 116986 ms
Total hot run time: 50309 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4353	4327	4365	4327
q2	326	223	227	223
q3	4163	4146	4167	4146
q4	2776	2766	2777	2766
q5	7309	7254	7217	7217
q6	237	118	117	117
q7	3268	2882	2813	2813
q8	4400	4486	4498	4486
q9	17163	17091	17005	17005
q10	4272	4255	4316	4255
q11	743	672	704	672
q12	1039	861	850	850
q13	6925	3725	3765	3725
q14	444	421	420	420
q15	502	459	457	457
q16	775	713	695	695
q17	3876	3796	3932	3796
q18	8787	8813	8894	8813
q19	1722	1714	1699	1699
q20	2369	2166	2131	2131
q21	8659	8573	8442	8442
q22	1009	958	932	932
Total cold run time: 85117 ms
Total hot run time: 79987 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.80% (8046/21285)
Line Coverage: 29.47% (65716/223012)
Region Coverage: 28.93% (33816/116900)
Branch Coverage: 24.78% (17364/70064)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c083702939b54147d1418eadad36705008400a84_c083702939b54147d1418eadad36705008400a84/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 201717 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c083702939b54147d1418eadad36705008400a84, data reload: false

query1	926	399	383	383
query2	6516	2228	1967	1967
query3	6918	199	196	196
query4	20575	17906	17929	17906
query5	19707	6544	6543	6543
query6	284	219	232	219
query7	4148	303	310	303
query8	279	274	250	250
query9	3172	2740	2664	2664
query10	418	283	327	283
query11	11317	10595	10622	10595
query12	124	74	69	69
query13	5584	646	643	643
query14	17980	13154	13331	13154
query15	360	246	241	241
query16	6429	274	261	261
query17	1719	1452	897	897
query18	2290	410	405	405
query19	206	139	147	139
query20	80	79	75	75
query21	191	94	94	94
query22	5282	4820	4918	4820
query23	32404	32010	32199	32010
query24	6926	6498	6556	6498
query25	505	432	438	432
query26	530	161	162	161
query27	1894	308	299	299
query28	6057	2274	2257	2257
query29	2999	2632	2891	2632
query30	239	165	164	164
query31	905	741	771	741
query32	65	59	60	59
query33	408	252	254	252
query34	852	465	492	465
query35	1127	936	910	910
query36	1243	1129	1233	1129
query37	90	60	63	60
query38	3081	2922	2959	2922
query39	1387	1330	1339	1330
query40	209	95	96	95
query41	34	33	34	33
query42	93	89	93	89
query43	651	574	550	550
query44	1156	714	710	710
query45	247	227	226	226
query46	1235	977	982	977
query47	1956	1815	1989	1815
query48	994	697	691	691
query49	617	363	367	363
query50	872	637	613	613
query51	4766	4704	4686	4686
query52	84	88	80	80
query53	452	324	313	313
query54	2651	2479	2458	2458
query55	99	88	88	88
query56	218	227	214	214
query57	1107	1200	1190	1190
query58	204	190	200	190
query59	3555	3360	3145	3145
query60	203	190	193	190
query61	85	81	87	81
query62	871	494	466	466
query63	478	340	339	339
query64	2535	1484	1456	1456
query65	3691	3570	3591	3570
query66	821	387	368	368
query67	18635	16631	15653	15653
query68	8409	658	671	658
query69	556	368	382	368
query70	1666	1462	1355	1355
query71	394	298	318	298
query72	6521	3439	3463	3439
query73	726	320	323	320
query74	6310	5844	5888	5844
query75	4632	3765	3788	3765
query76	4643	1150	1148	1148
query77	581	248	253	248
query78	12457	11793	11519	11519
query79	8411	647	646	646
query80	2299	395	400	395
query81	534	227	229	227
query82	1538	97	97	97
query83	179	139	132	132
query84	261	71	70	70
query85	1347	301	288	288
query86	351	311	289	289
query87	3203	3003	3029	3003
query88	5178	2307	2311	2307
query89	390	283	297	283
query90	1777	209	208	208
query91	152	130	118	118
query92	59	51	53	51
query93	4999	587	591	587
query94	853	211	197	197
query95	1097	1087	1080	1080
query96	646	322	325	322
query97	6508	6371	6422	6371
query98	195	174	176	174
query99	2821	833	904	833
Total cold run time: 313429 ms
Total hot run time: 201717 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c083702939b54147d1418eadad36705008400a84 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
Insert into select:       22.4 seconds inserted 10000000 Rows, about 446K ops/s

@mrhhsg
Copy link
Member Author

mrhhsg commented Mar 26, 2024

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49704 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c3ac970e828992ee8b6463e9947214e46aa51d70, data reload: false

------ Round 1 ----------------------------------
q1	17682	4374	4358	4358
q2	2070	144	138	138
q3	10478	1914	1860	1860
q4	10384	1239	1294	1239
q5	8404	3910	3942	3910
q6	229	121	122	121
q7	2046	1563	1596	1563
q8	9489	2693	2700	2693
q9	12906	10561	10471	10471
q10	8617	3522	3473	3473
q11	421	238	224	224
q12	460	294	306	294
q13	18345	3958	4005	3958
q14	342	330	318	318
q15	507	450	455	450
q16	691	602	580	580
q17	1114	973	963	963
q18	7373	6821	6808	6808
q19	1690	1560	1541	1541
q20	519	304	290	290
q21	4441	4056	4110	4056
q22	503	396	403	396
Total cold run time: 118711 ms
Total hot run time: 49704 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4394	4362	4333	4333
q2	329	231	218	218
q3	4196	4192	4153	4153
q4	2767	2764	2764	2764
q5	7253	7171	7243	7171
q6	234	122	118	118
q7	3269	2843	2842	2842
q8	4380	4446	4501	4446
q9	17234	16965	17525	16965
q10	4394	4468	4443	4443
q11	796	718	693	693
q12	1035	859	850	850
q13	9503	3771	3758	3758
q14	439	439	420	420
q15	497	468	462	462
q16	774	719	701	701
q17	3774	3776	3894	3776
q18	8694	8826	8808	8808
q19	1701	1680	1669	1669
q20	2346	2137	2061	2061
q21	8614	8505	8465	8465
q22	1087	938	925	925
Total cold run time: 87710 ms
Total hot run time: 80041 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.81% (8048/21285)
Line Coverage: 29.47% (65728/223010)
Region Coverage: 28.94% (33828/116900)
Branch Coverage: 24.79% (17368/70064)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c3ac970e828992ee8b6463e9947214e46aa51d70_c3ac970e828992ee8b6463e9947214e46aa51d70/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 201236 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c3ac970e828992ee8b6463e9947214e46aa51d70, data reload: false

query1	935	387	375	375
query2	6518	2176	2247	2176
query3	6921	196	196	196
query4	20836	17757	17906	17757
query5	19723	6509	6488	6488
query6	283	215	223	215
query7	4156	300	296	296
query8	259	251	236	236
query9	3179	2755	2662	2662
query10	404	300	289	289
query11	11283	10754	10668	10668
query12	112	67	73	67
query13	5584	672	634	634
query14	17842	13687	13397	13397
query15	385	242	235	235
query16	6436	266	258	258
query17	1593	1434	853	853
query18	2315	420	405	405
query19	208	146	147	146
query20	79	79	71	71
query21	180	92	92	92
query22	5059	4985	5037	4985
query23	32489	31867	31855	31855
query24	6898	6445	6564	6445
query25	531	424	399	399
query26	498	159	157	157
query27	1787	296	300	296
query28	6081	2263	2236	2236
query29	2801	2801	2688	2688
query30	237	160	160	160
query31	906	733	729	729
query32	64	57	61	57
query33	392	274	247	247
query34	879	482	486	482
query35	1105	935	864	864
query36	1323	1175	1352	1175
query37	88	61	58	58
query38	3036	2911	2955	2911
query39	1372	1337	1336	1336
query40	203	91	93	91
query41	36	32	32	32
query42	85	87	87	87
query43	633	546	554	546
query44	1143	718	727	718
query45	243	224	228	224
query46	1225	972	980	972
query47	1911	1751	1964	1751
query48	990	675	689	675
query49	615	386	376	376
query50	853	627	619	619
query51	4759	4681	4689	4681
query52	96	75	78	75
query53	445	325	319	319
query54	2682	2404	2430	2404
query55	81	68	76	68
query56	213	212	217	212
query57	1213	1100	1038	1038
query58	209	208	191	191
query59	3340	3113	3206	3113
query60	216	192	190	190
query61	84	81	85	81
query62	821	473	476	473
query63	472	338	343	338
query64	2433	1514	1361	1361
query65	3626	3543	3536	3536
query66	737	371	358	358
query67	15717	15763	16904	15763
query68	8374	670	645	645
query69	585	338	334	334
query70	1788	1393	1470	1393
query71	394	304	296	296
query72	6430	3392	3413	3392
query73	737	324	316	316
query74	6246	5893	5828	5828
query75	4957	3724	3551	3551
query76	4892	1145	1211	1145
query77	725	245	241	241
query78	12525	11739	11697	11697
query79	7882	632	647	632
query80	1381	389	389	389
query81	492	230	234	230
query82	1519	104	96	96
query83	162	132	130	130
query84	255	67	66	66
query85	1069	292	276	276
query86	341	289	331	289
query87	3228	2991	3008	2991
query88	5100	2311	2322	2311
query89	377	265	290	265
query90	1852	208	214	208
query91	148	121	114	114
query92	55	52	49	49
query93	5046	556	591	556
query94	776	205	209	205
query95	1124	1068	1063	1063
query96	637	327	327	327
query97	6531	6314	6374	6314
query98	189	168	174	168
query99	2985	884	831	831
Total cold run time: 308660 ms
Total hot run time: 201236 ms

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c3ac970e828992ee8b6463e9947214e46aa51d70 with default session variables
Stream load json:         20 seconds loaded 2358488459 Bytes, about 112 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.3 seconds inserted 10000000 Rows, about 448K ops/s

@yiguolei yiguolei merged commit 73ed833 into apache:branch-2.0 Mar 27, 2024
@mrhhsg mrhhsg deleted the pick_32623 branch March 27, 2024 01:49
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants