Skip to content

Conversation

@cambyzju
Copy link
Contributor

@cambyzju cambyzju commented Feb 2, 2024

Proposed changes

cherry-pick #30738

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@github-actions
Copy link
Contributor

github-actions bot commented Feb 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@cambyzju
Copy link
Contributor Author

cambyzju commented Feb 2, 2024

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50061 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 05397a20b5080365b0ada8682e2cacb6176da8e5, data reload: false

------ Round 1 ----------------------------------
q1	17645	4486	4337	4337
q2	2049	155	151	151
q3	10330	1885	1912	1885
q4	10093	1254	1316	1254
q5	8787	3960	3990	3960
q6	228	123	122	122
q7	2007	1641	1589	1589
q8	9278	2703	2708	2703
q9	11094	10674	10511	10511
q10	8655	3513	3524	3513
q11	412	248	243	243
q12	471	295	300	295
q13	18310	3938	3986	3938
q14	350	323	322	322
q15	495	445	454	445
q16	688	593	590	590
q17	1128	969	969	969
q18	7273	7044	6925	6925
q19	1683	1582	1559	1559
q20	527	296	320	296
q21	4458	4108	4073	4073
q22	498	385	381	381
Total cold run time: 116459 ms
Total hot run time: 50061 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4291	4299	4307	4299
q2	317	221	223	221
q3	4149	4146	4101	4101
q4	2752	2731	2727	2727
q5	7306	7210	7164	7164
q6	229	118	120	118
q7	3235	2846	2834	2834
q8	4316	4462	4469	4462
q9	17586	17329	17319	17319
q10	4204	4266	4246	4246
q11	746	675	717	675
q12	1018	873	850	850
q13	7635	3690	3761	3690
q14	442	413	429	413
q15	500	458	458	458
q16	745	710	700	700
q17	3926	3862	3842	3842
q18	8714	8794	8842	8794
q19	1698	1698	1660	1660
q20	2378	2127	2109	2109
q21	8509	8494	8592	8494
q22	1030	914	932	914
Total cold run time: 85726 ms
Total hot run time: 80090 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.85% (8036/21232)
Line Coverage: 29.55% (65482/221633)
Region Coverage: 28.96% (33633/116121)
Branch Coverage: 24.82% (17257/69520)
Coverage Report: http://coverage.selectdb-in.cc/coverage/05397a20b5080365b0ada8682e2cacb6176da8e5_05397a20b5080365b0ada8682e2cacb6176da8e5/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 233477 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 05397a20b5080365b0ada8682e2cacb6176da8e5, data reload: false

query1	914	390	376	376
query2	6509	2120	2101	2101
query3	6913	205	212	205
query4	20776	17893	17817	17817
query5	19723	6282	6183	6183
query6	284	220	219	219
query7	4152	299	293	293
query8	271	263	250	250
query9	3089	2680	2595	2595
query10	423	289	273	273
query11	11302	10606	10576	10576
query12	116	75	73	73
query13	5640	637	615	615
query14	17709	12870	13256	12870
query15	362	231	235	231
query16	6449	264	267	264
query17	3187	863	862	862
query18	2269	401	406	401
query19	211	146	150	146
query20	72	77	76	76
query21	194	92	105	92
query22	5259	5063	4969	4969
query23	32298	31887	31759	31759
query24	6925	6428	6441	6428
query25	533	418	428	418
query26	521	161	161	161
query27	1850	297	300	297
query28	6078	2223	2188	2188
query29	2924	2798	2699	2699
query30	242	159	157	157
query31	901	709	735	709
query32	65	57	59	57
query33	408	249	244	244
query34	849	462	485	462
query35	1143	892	975	892
query36	1828	1423	1498	1423
query37	89	57	60	57
query38	3031	2968	2940	2940
query39	1364	1312	1316	1312
query40	201	92	95	92
query41	33	32	32	32
query42	84	86	78	78
query43	770	688	650	650
query44	1111	702	707	702
query45	239	230	225	225
query46	1228	958	976	958
query47	1757	1833	1702	1702
query48	975	671	649	649
query49	622	363	361	361
query50	844	597	621	597
query51	5550	5495	5479	5479
query52	77	89	78	78
query53	438	317	318	317
query54	2408	2220	2239	2220
query55	82	77	80	77
query56	212	196	192	192
query57	1168	1114	1030	1030
query58	202	193	204	193
query59	3540	3294	3492	3294
query60	209	186	190	186
query61	83	87	92	87
query62	802	477	488	477
query63	468	327	327	327
query64	2022	1311	1289	1289
query65	3669	3621	3609	3609
query66	819	368	364	364
query67	16480	16722	15402	15402
query68	8440	652	658	652
query69	543	335	359	335
query70	2024	2008	1729	1729
query71	380	309	311	309
query72	4600	3305	3350	3305
query73	732	330	323	323
query74	6327	5892	5798	5798
query75	4723	3524	3532	3524
query76	5244	1180	1228	1180
query77	811	258	252	252
query78	32694	51326	44024	44024
query79	15116	641	645	641
query80	4867	384	408	384
query81	559	234	238	234
query82	800	99	96	96
query83	381	138	129	129
query84	258	71	68	68
query85	2610	279	282	279
query86	489	393	386	386
query87	3208	2991	3012	2991
query88	6481	2341	2349	2341
query89	392	297	288	288
query90	2504	207	196	196
query91	163	115	124	115
query92	68	49	52	49
query93	4887	610	583	583
query94	1853	210	208	208
query95	1100	1062	1053	1053
query96	632	329	319	319
query97	6576	6302	6536	6302
query98	181	167	168	167
query99	3937	906	860	860
Total cold run time: 346546 ms
Total hot run time: 233477 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.37 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 05397a20b5080365b0ada8682e2cacb6176da8e5, data reload: false

query1	0.02	0.02	0.02
query2	0.06	0.02	0.02
query3	0.24	0.05	0.04
query4	1.83	0.06	0.07
query5	0.53	0.53	0.52
query6	1.27	0.61	0.60
query7	0.02	0.00	0.00
query8	0.03	0.02	0.03
query9	0.53	0.48	0.49
query10	0.54	0.53	0.54
query11	0.12	0.08	0.09
query12	0.11	0.09	0.09
query13	0.62	0.62	0.61
query14	0.77	0.79	0.80
query15	0.77	0.77	0.76
query16	0.36	0.38	0.37
query17	0.99	0.96	1.02
query18	0.23	0.27	0.25
query19	1.93	1.83	1.76
query20	0.01	0.01	0.00
query21	15.48	0.57	0.57
query22	1.94	2.07	2.36
query23	17.27	0.93	1.03
query24	4.68	1.33	1.61
query25	1.57	0.11	0.11
query26	0.26	0.14	0.12
query27	0.11	0.11	0.10
query28	7.44	0.72	0.70
query29	12.68	2.35	2.12
query30	0.54	0.54	0.58
query31	2.82	0.37	0.39
query32	3.42	0.50	0.49
query33	3.06	3.08	3.05
query34	15.27	4.84	4.84
query35	4.82	4.87	4.82
query36	1.06	1.01	1.01
query37	0.06	0.04	0.05
query38	0.04	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.14	0.14
query41	0.07	0.01	0.01
query42	0.02	0.01	0.01
query43	0.03	0.02	0.01
Total cold run time: 103.8 s
Total hot run time: 31.37 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 05397a20b5080365b0ada8682e2cacb6176da8e5 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@yiguolei
Copy link
Contributor

yiguolei commented Feb 3, 2024

@cambyzju I think we should first push the PR to master branch and then pick to other branch. Then we will not miss any bugs that is fixed in lower version.

@yiguolei yiguolei merged commit 02c162a into apache:branch-2.0 Feb 3, 2024
@cambyzju
Copy link
Contributor Author

cambyzju commented Feb 4, 2024

@cambyzju I think we should first push the PR to master branch and then pick to other branch. Then we will not miss any bugs that is fixed in lower version.

@yiguolei master already change to dynamic_cast, and remove down_cast for performance reason. (#23068)

And the pr marked dev/2.0.3-merged, but it seems not merged in branch-2.0.

mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants