Skip to content

Conversation

@HappenLee
Copy link
Contributor

Proposed changes

Issue Number: cherry pick #38985

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions


#pragma once

#include <glog/logging.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: 'glog/logging.h' file not found [clang-diagnostic-error]

#include <glog/logging.h>
         ^

std::partial_sort(res.begin(), sort_end, res.end(),
[this](size_t a, size_t b) { return data[a] < data[b]; });
} else {
if (reverse)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
if (reverse)
if (reverse) {

be/src/vec/columns/column_decimal.h:309:

-             else
+             } else

if (reverse)
pdqsort(res.begin(), res.end(),
[this](size_t a, size_t b) { return data[a] > data[b]; });
else
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
else
else {

be/src/vec/columns/column_decimal.h:311:

-                         [this](size_t a, size_t b) { return data[a] < data[b]; });
+                         [this](size_t a, size_t b) { return data[a] < data[b]; });
+ }

limit = 0;
}
// std::partial_sort need limit << s can get performance benefit
if (limit > (s / 8.0)) limit = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
if (limit > (s / 8.0)) limit = 0;
if (limit > (s / 8.0)) { limit = 0;
}


if (limit >= s) limit = 0;
// std::partial_sort need limit << s can get performance benefit
if (limit > (s / 8.0)) limit = 0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: statement should be inside braces [readability-braces-around-statements]

Suggested change
if (limit > (s / 8.0)) limit = 0;
if (limit > (s / 8.0)) { limit = 0;
}

@HappenLee
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 50159 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2d3bb330c12b63829bf7269c95b6c1a9009c14d8, data reload: false

------ Round 1 ----------------------------------
q1	17616	4474	4397	4397
q2	2140	154	149	149
q3	10271	1943	1937	1937
q4	10358	1288	1344	1288
q5	8545	3956	4000	3956
q6	238	125	126	125
q7	2043	1596	1647	1596
q8	9293	2776	2749	2749
q9	10607	10241	10300	10241
q10	8637	3517	3508	3508
q11	424	257	243	243
q12	473	306	300	300
q13	18343	3985	4057	3985
q14	352	337	333	333
q15	514	461	464	461
q16	696	580	577	577
q17	1139	978	1006	978
q18	7438	6898	6858	6858
q19	1814	1749	1638	1638
q20	554	315	319	315
q21	4495	4196	4084	4084
q22	534	441	451	441
Total cold run time: 116524 ms
Total hot run time: 50159 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4357	4330	4338	4330
q2	331	227	223	223
q3	4172	4152	4134	4134
q4	2767	2749	2770	2749
q5	7279	7133	7110	7110
q6	245	123	123	123
q7	3275	2827	2808	2808
q8	4388	4513	4547	4513
q9	17035	16888	16678	16678
q10	4232	4297	4272	4272
q11	823	694	658	658
q12	1060	857	846	846
q13	6857	3724	3728	3724
q14	461	421	427	421
q15	501	453	455	453
q16	750	698	681	681
q17	3818	3823	3905	3823
q18	8767	8979	8884	8884
q19	1747	1716	1658	1658
q20	2382	2119	2094	2094
q21	8529	8463	8492	8463
q22	1048	961	970	961
Total cold run time: 84824 ms
Total hot run time: 79606 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 203556 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2d3bb330c12b63829bf7269c95b6c1a9009c14d8, data reload: false

query1	928	402	408	402
query2	6543	2823	2613	2613
query3	6920	220	210	210
query4	21066	18102	17982	17982
query5	19769	6574	6514	6514
query6	295	230	225	225
query7	4171	307	324	307
query8	428	446	429	429
query9	3087	2662	2602	2602
query10	435	302	290	290
query11	11388	10651	10816	10651
query12	129	73	75	73
query13	5605	740	705	705
query14	18298	13259	13140	13140
query15	365	250	243	243
query16	6454	290	267	267
query17	1694	1445	869	869
query18	2322	413	423	413
query19	208	149	153	149
query20	76	78	78	78
query21	191	98	99	98
query22	5070	4996	5092	4996
query23	32655	31937	31952	31937
query24	6996	6647	6517	6517
query25	548	436	443	436
query26	527	168	159	159
query27	1871	298	294	294
query28	6172	2410	2359	2359
query29	2875	2725	2845	2725
query30	243	169	171	169
query31	908	728	756	728
query32	76	63	62	62
query33	401	262	267	262
query34	859	472	478	472
query35	1123	907	926	907
query36	1386	1281	1185	1185
query37	96	60	63	60
query38	3117	2877	2954	2877
query39	1398	1322	1346	1322
query40	206	98	92	92
query41	48	45	44	44
query42	85	81	90	81
query43	676	685	704	685
query44	1134	740	755	740
query45	246	238	235	235
query46	1225	968	990	968
query47	1804	1740	1661	1661
query48	1017	711	727	711
query49	618	375	368	368
query50	883	626	639	626
query51	4717	4655	4727	4655
query52	99	85	96	85
query53	452	321	317	317
query54	2660	2420	2457	2420
query55	91	85	85	85
query56	218	213	213	213
query57	1236	1142	1154	1142
query58	230	201	196	196
query59	4214	4008	3815	3815
query60	214	206	226	206
query61	99	99	101	99
query62	813	474	526	474
query63	478	348	336	336
query64	2475	1558	1511	1511
query65	3621	3573	3570	3570
query66	896	386	378	378
query67	15692	15657	15239	15239
query68	8872	664	648	648
query69	579	356	374	356
query70	2052	1495	1412	1412
query71	422	313	318	313
query72	6624	3435	3501	3435
query73	746	337	335	335
query74	6362	5761	5894	5761
query75	5443	3740	3709	3709
query76	5428	1194	1218	1194
query77	896	263	256	256
query78	12623	11525	11829	11525
query79	7604	648	640	640
query80	1031	411	409	409
query81	485	244	232	232
query82	1439	103	98	98
query83	178	139	134	134
query84	265	75	70	70
query85	892	334	329	329
query86	351	303	293	293
query87	3199	3000	3020	3000
query88	5175	2459	2469	2459
query89	375	301	292	292
query90	1949	210	215	210
query91	175	141	143	141
query92	59	51	53	51
query93	4753	563	544	544
query94	704	211	211	211
query95	1112	1042	1074	1042
query96	663	345	348	345
query97	6448	6340	6415	6340
query98	196	169	170	169
query99	2890	924	953	924
Total cold run time: 312830 ms
Total hot run time: 203556 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.31 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2d3bb330c12b63829bf7269c95b6c1a9009c14d8, data reload: false

query1	0.02	0.02	0.02
query2	0.07	0.02	0.02
query3	0.25	0.05	0.06
query4	1.77	0.06	0.07
query5	0.53	0.53	0.51
query6	1.31	0.61	0.62
query7	0.01	0.01	0.01
query8	0.03	0.03	0.02
query9	0.54	0.48	0.48
query10	0.54	0.52	0.53
query11	0.12	0.09	0.09
query12	0.11	0.09	0.10
query13	0.63	0.62	0.62
query14	0.79	0.80	0.79
query15	0.79	0.76	0.76
query16	0.37	0.38	0.37
query17	1.02	1.04	1.00
query18	0.22	0.24	0.24
query19	1.94	1.86	1.83
query20	0.02	0.00	0.00
query21	15.45	0.54	0.55
query22	1.97	1.91	1.75
query23	17.37	1.01	0.95
query24	5.56	1.39	1.37
query25	0.40	0.12	0.04
query26	0.60	0.15	0.15
query27	0.04	0.05	0.03
query28	7.23	0.75	0.71
query29	12.60	2.38	2.35
query30	0.58	0.53	0.55
query31	2.81	0.39	0.38
query32	3.36	0.51	0.50
query33	3.10	3.10	3.06
query34	15.25	4.80	4.78
query35	4.88	4.85	4.82
query36	1.06	1.01	1.02
query37	0.06	0.05	0.04
query38	0.04	0.03	0.02
query39	0.02	0.01	0.02
query40	0.16	0.15	0.14
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.02
Total cold run time: 103.73 s
Total hot run time: 31.31 s

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.74% (8104/21471)
Line Coverage: 29.39% (66392/225938)
Region Coverage: 28.89% (34265/118603)
Branch Coverage: 24.77% (17611/71084)
Coverage Report: http://coverage.selectdb-in.cc/coverage/2d3bb330c12b63829bf7269c95b6c1a9009c14d8_2d3bb330c12b63829bf7269c95b6c1a9009c14d8/report/index.html

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 2d3bb330c12b63829bf7269c95b6c1a9009c14d8 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.3 seconds inserted 10000000 Rows, about 469K ops/s

@HappenLee HappenLee merged commit 0fe67a8 into apache:branch-2.0 Aug 10, 2024
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Aug 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants