Skip to content

Conversation

@airborne12
Copy link
Member

Proposed changes

Issue Number: close #xxx

Pick from #31075

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@airborne12
Copy link
Member Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.82% (8033/21240)
Line Coverage: 29.48% (65476/222115)
Region Coverage: 28.95% (33687/116373)
Branch Coverage: 24.80% (17290/69714)
Coverage Report: http://coverage.selectdb-in.cc/coverage/f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc_f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 49929 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc, data reload: false

------ Round 1 ----------------------------------
q1	17873	4353	4306	4306
q2	2050	153	137	137
q3	10319	1895	1956	1895
q4	10098	1253	1314	1253
q5	8399	4004	4001	4001
q6	230	124	125	124
q7	2075	1624	1614	1614
q8	9308	2702	2698	2698
q9	10887	10491	10442	10442
q10	8642	3538	3506	3506
q11	429	237	248	237
q12	464	302	308	302
q13	18356	3997	3992	3992
q14	352	320	329	320
q15	517	446	468	446
q16	697	597	585	585
q17	1123	965	979	965
q18	7259	6846	6980	6846
q19	1679	1589	1513	1513
q20	516	310	323	310
q21	4454	4135	4054	4054
q22	500	383	401	383
Total cold run time: 116227 ms
Total hot run time: 49929 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4320	4308	4316	4308
q2	318	221	225	221
q3	4189	4152	4142	4142
q4	2754	2746	2744	2744
q5	7281	7221	7194	7194
q6	235	117	116	116
q7	3239	2878	2824	2824
q8	4344	4472	4454	4454
q9	17128	17083	16880	16880
q10	4259	4311	4266	4266
q11	757	675	697	675
q12	1018	862	850	850
q13	6805	3756	3746	3746
q14	462	428	404	404
q15	498	460	455	455
q16	761	717	711	711
q17	3798	3815	3906	3815
q18	8739	8773	8674	8674
q19	1722	1705	1657	1657
q20	2402	2156	2132	2132
q21	8577	8582	8655	8582
q22	1021	965	948	948
Total cold run time: 84627 ms
Total hot run time: 79798 ms

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 18, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@doris-robot
Copy link

TPC-DS: Total hot run time: 239295 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc, data reload: false

query1	924	386	378	378
query2	6515	2142	2259	2142
query3	6911	207	201	201
query4	20143	17909	17956	17909
query5	19721	6523	6467	6467
query6	283	207	235	207
query7	4150	294	299	294
query8	244	218	235	218
query9	3187	2759	2657	2657
query10	412	315	308	308
query11	11410	10791	10668	10668
query12	127	74	76	74
query13	5587	642	625	625
query14	17695	13893	13275	13275
query15	383	244	237	237
query16	6439	266	262	262
query17	1724	1465	860	860
query18	2330	413	396	396
query19	206	138	140	138
query20	77	80	75	75
query21	184	99	96	96
query22	5315	5085	5136	5085
query23	32844	31916	32066	31916
query24	6993	6490	6465	6465
query25	531	433	414	414
query26	532	170	157	157
query27	1876	291	295	291
query28	6099	2241	2212	2212
query29	2988	2822	2697	2697
query30	240	166	160	160
query31	889	733	723	723
query32	60	61	60	60
query33	392	258	248	248
query34	869	484	472	472
query35	1128	912	871	871
query36	1706	1670	1689	1670
query37	91	61	58	58
query38	3090	2893	2929	2893
query39	1375	1318	1333	1318
query40	205	101	96	96
query41	33	31	35	31
query42	90	83	81	81
query43	645	590	706	590
query44	1154	712	719	712
query45	250	230	224	224
query46	1235	999	994	994
query47	1893	1728	1606	1606
query48	990	693	673	673
query49	617	367	375	367
query50	869	642	615	615
query51	5620	5448	5399	5399
query52	85	88	77	77
query53	441	319	313	313
query54	2672	2474	2461	2461
query55	95	77	88	77
query56	221	216	209	209
query57	1177	1173	1135	1135
query58	200	209	189	189
query59	3478	3301	3275	3275
query60	199	213	194	194
query61	96	83	80	80
query62	862	480	506	480
query63	474	331	332	331
query64	2416	1489	1428	1428
query65	3666	3552	3581	3552
query66	770	376	353	353
query67	16922	16528	15831	15831
query68	8373	670	686	670
query69	580	346	339	339
query70	1897	1818	1660	1660
query71	406	294	300	294
query72	6450	3372	3398	3372
query73	735	330	327	327
query74	6361	5901	5808	5808
query75	4661	3798	3762	3762
query76	4653	1162	1188	1162
query77	537	244	242	242
query78	32055	51333	47248	47248
query79	14671	639	648	639
query80	4414	373	380	373
query81	566	230	233	230
query82	908	96	94	94
query83	331	138	131	131
query84	256	68	66	66
query85	2125	282	273	273
query86	456	365	376	365
query87	3234	3021	3033	3021
query88	6839	2354	2334	2334
query89	388	311	289	289
query90	2509	218	216	216
query91	156	115	124	115
query92	57	55	54	54
query93	4889	601	564	564
query94	1657	204	207	204
query95	1119	1068	1051	1051
query96	658	331	331	331
query97	6491	6393	6381	6381
query98	195	161	162	161
query99	3699	979	949	949
Total cold run time: 345371 ms
Total hot run time: 239295 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.67 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.83	0.06	0.07
query5	0.53	0.53	0.53
query6	1.26	0.64	0.62
query7	0.02	0.02	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.54	0.52	0.54
query11	0.11	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.62
query14	0.78	0.79	0.79
query15	0.77	0.77	0.77
query16	0.35	0.36	0.38
query17	1.01	1.00	1.03
query18	0.22	0.27	0.25
query19	1.93	1.80	1.79
query20	0.01	0.01	0.01
query21	15.48	0.61	0.55
query22	2.15	2.35	1.27
query23	17.64	1.09	0.92
query24	6.66	1.12	1.38
query25	1.44	0.11	0.11
query26	0.29	0.14	0.13
query27	0.11	0.12	0.11
query28	7.58	0.74	0.73
query29	12.74	2.39	2.30
query30	0.60	0.56	0.52
query31	2.82	0.37	0.39
query32	3.45	0.50	0.49
query33	3.08	3.06	3.11
query34	15.27	4.79	4.80
query35	4.84	4.87	4.84
query36	1.05	1.03	1.03
query37	0.06	0.04	0.04
query38	0.04	0.02	0.02
query39	0.01	0.02	0.02
query40	0.16	0.14	0.14
query41	0.06	0.01	0.02
query42	0.01	0.02	0.01
query43	0.03	0.01	0.02
Total cold run time: 106.53 s
Total hot run time: 30.67 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit f750c43a63e79fbaf5cbff2ae6d34d7ff736a9dc with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.5 seconds inserted 10000000 Rows, about 465K ops/s

@airborne12
Copy link
Member Author

run buildall

@github-actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 49697 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4641cb30e81062eee9d5fe58618e612da44685f7, data reload: false

------ Round 1 ----------------------------------
q1	17823	4369	4391	4369
q2	2020	151	144	144
q3	10455	1873	1888	1873
q4	10347	1212	1319	1212
q5	8893	3931	3932	3931
q6	223	121	123	121
q7	1984	1578	1607	1578
q8	9284	2694	2716	2694
q9	10730	10592	10303	10303
q10	8614	3521	3531	3521
q11	415	249	243	243
q12	460	292	302	292
q13	18339	4002	3971	3971
q14	343	328	327	327
q15	496	459	467	459
q16	696	587	586	586
q17	1123	964	946	946
q18	7350	6893	6976	6893
q19	1660	1544	1476	1476
q20	537	297	312	297
q21	4465	4109	4065	4065
q22	496	413	396	396
Total cold run time: 116753 ms
Total hot run time: 49697 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4277	4280	4267	4267
q2	314	218	220	218
q3	4172	4140	4122	4122
q4	2771	2727	2743	2727
q5	7252	7139	7227	7139
q6	232	115	118	115
q7	3198	2843	2820	2820
q8	4412	4406	4476	4406
q9	17070	16938	16950	16938
q10	4245	4237	4278	4237
q11	776	662	692	662
q12	1029	866	860	860
q13	6815	3743	3742	3742
q14	450	420	417	417
q15	497	455	449	449
q16	776	696	712	696
q17	3903	3896	3863	3863
q18	8771	8647	8700	8647
q19	1690	1680	1631	1631
q20	2355	2165	2090	2090
q21	8644	8494	8530	8494
q22	1042	977	936	936
Total cold run time: 84691 ms
Total hot run time: 79476 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.83% (8035/21240)
Line Coverage: 29.49% (65494/222118)
Region Coverage: 28.96% (33698/116376)
Branch Coverage: 24.81% (17295/69716)
Coverage Report: http://coverage.selectdb-in.cc/coverage/4641cb30e81062eee9d5fe58618e612da44685f7_4641cb30e81062eee9d5fe58618e612da44685f7/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 241309 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4641cb30e81062eee9d5fe58618e612da44685f7, data reload: false

query1	901	388	385	385
query2	6516	2230	2000	2000
query3	6920	206	201	201
query4	20822	18081	18098	18081
query5	19719	6477	6469	6469
query6	292	207	223	207
query7	4154	294	293	293
query8	260	238	231	231
query9	3110	2685	2620	2620
query10	428	277	301	277
query11	11340	10831	10795	10795
query12	121	80	75	75
query13	5576	656	622	622
query14	17876	13821	13291	13291
query15	355	234	246	234
query16	6434	275	265	265
query17	1698	1470	866	866
query18	2317	409	397	397
query19	208	144	142	142
query20	70	77	80	77
query21	179	93	93	93
query22	5317	5138	5149	5138
query23	32398	31984	31971	31971
query24	6943	6442	6483	6442
query25	498	445	412	412
query26	605	159	160	159
query27	2031	285	290	285
query28	6126	2203	2177	2177
query29	2963	2625	2877	2625
query30	240	163	159	159
query31	894	727	728	727
query32	65	60	56	56
query33	374	250	248	248
query34	856	479	478	478
query35	1092	921	904	904
query36	1546	1466	1381	1381
query37	90	58	63	58
query38	3068	2897	2868	2868
query39	1371	1317	1331	1317
query40	203	93	96	93
query41	33	30	35	30
query42	83	84	79	79
query43	668	607	592	592
query44	1085	709	728	709
query45	235	232	224	224
query46	1221	965	970	965
query47	1800	1645	1718	1645
query48	971	674	664	664
query49	611	372	363	363
query50	873	646	610	610
query51	5509	5493	5464	5464
query52	94	77	83	77
query53	435	320	318	318
query54	2662	2469	2482	2469
query55	102	79	91	79
query56	225	191	205	191
query57	1229	1072	1231	1072
query58	214	193	208	193
query59	3414	3299	3279	3279
query60	206	208	189	189
query61	82	83	81	81
query62	786	510	493	493
query63	475	335	348	335
query64	2322	1500	1365	1365
query65	3584	3541	3538	3538
query66	773	375	356	356
query67	15348	14775	15723	14775
query68	8668	639	661	639
query69	565	321	350	321
query70	1864	1606	1502	1502
query71	414	298	296	296
query72	6644	3388	3389	3388
query73	722	317	324	317
query74	6334	5920	5860	5860
query75	5132	3645	3693	3645
query76	5253	1133	1004	1004
query77	887	255	250	250
query78	32472	51159	53102	51159
query79	15296	644	618	618
query80	4757	373	379	373
query81	566	225	230	225
query82	1305	100	94	94
query83	332	135	135	135
query84	265	69	69	69
query85	1875	280	274	274
query86	442	364	364	364
query87	3247	2987	2986	2986
query88	7151	2307	2304	2304
query89	424	285	286	285
query90	2562	206	214	206
query91	153	124	119	119
query92	62	52	49	49
query93	6132	563	569	563
query94	1652	203	204	203
query95	1103	1063	1072	1063
query96	645	325	318	318
query97	6537	6422	6289	6289
query98	186	179	172	172
query99	3708	955	945	945
Total cold run time: 348371 ms
Total hot run time: 241309 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.34 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4641cb30e81062eee9d5fe58618e612da44685f7, data reload: false

query1	0.03	0.03	0.02
query2	0.06	0.02	0.02
query3	0.25	0.04	0.04
query4	1.84	0.08	0.08
query5	0.53	0.52	0.51
query6	1.25	0.61	0.60
query7	0.01	0.01	0.01
query8	0.03	0.02	0.02
query9	0.52	0.48	0.48
query10	0.53	0.53	0.53
query11	0.12	0.08	0.09
query12	0.12	0.09	0.10
query13	0.63	0.61	0.62
query14	0.78	0.79	0.78
query15	0.78	0.75	0.76
query16	0.38	0.36	0.37
query17	1.02	1.02	1.00
query18	0.22	0.26	0.26
query19	1.91	1.88	1.84
query20	0.01	0.01	0.01
query21	15.48	0.54	0.55
query22	1.99	2.27	1.62
query23	17.40	1.08	0.96
query24	6.56	0.60	1.57
query25	1.60	0.11	0.12
query26	0.30	0.14	0.13
query27	0.11	0.11	0.11
query28	6.35	0.71	0.70
query29	12.66	2.27	2.19
query30	0.66	0.54	0.49
query31	2.81	0.37	0.38
query32	3.42	0.49	0.49
query33	3.05	3.06	3.05
query34	15.52	4.80	4.80
query35	4.87	4.84	4.86
query36	1.05	1.01	1.02
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.01	0.02	0.01
query40	0.15	0.14	0.15
query41	0.07	0.01	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 105.22 s
Total hot run time: 30.34 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 4641cb30e81062eee9d5fe58618e612da44685f7 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.6 seconds inserted 10000000 Rows, about 462K ops/s

Copy link
Contributor

@xiaokang xiaokang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@xiaokang xiaokang merged commit 1485f37 into apache:branch-2.0 Feb 19, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. reviewed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants