Skip to content

Conversation

@zzzxl1993
Copy link
Contributor

…dex is missing

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at dev@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return Status::OK();
};

Status InvertedIndexReader::check_file_exist(const std::string& index_file_key) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'check_file_exist' can be made static [readability-convert-member-functions-to-static]

Suggested change
Status InvertedIndexReader::check_file_exist(const std::string& index_file_key) {
static Status InvertedIndexReader::check_file_exist(const std::string& index_file_key) {

@xiaokang xiaokang added usercase Important user case type label dev/2.0.x labels Apr 15, 2024
@zzzxl1993
Copy link
Contributor Author

run buildall

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

class InvertedIndexColumnWriterEmptyImpl final : public InvertedIndexColumnWriter {
public:
Status init() override { return Status::OK(); }
Status add_values(const std::string name, const void* values,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: method 'add_values' can be made static [readability-convert-member-functions-to-static]

Suggested change
Status add_values(const std::string name, const void* values,
static Status add_values(const std::string name, const void* values,
Additional context

be/src/util/debug_points.h:39: expanded from macro 'DBUG_EXECUTE_IF'

            code;                                                             \
            ^

Copy link
Member

@airborne12 airborne12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

PR approved by anyone and no changes requested.

qidaye
qidaye previously approved these changes Apr 16, 2024
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 16, 2024
@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@doris-robot
Copy link

TPC-H: Total hot run time: 38609 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c9a16f05afa797649b9857795d075bef55c7e4f3, data reload: false

------ Round 1 ----------------------------------
q1	17624	4401	4277	4277
q2	2186	190	195	190
q3	10477	1175	1215	1175
q4	10512	790	764	764
q5	7865	2753	2673	2673
q6	223	134	134	134
q7	1034	611	580	580
q8	9228	2099	2042	2042
q9	7115	6633	6564	6564
q10	8516	3569	3505	3505
q11	447	229	218	218
q12	416	219	222	219
q13	17801	2919	2950	2919
q14	271	225	225	225
q15	537	487	474	474
q16	528	379	381	379
q17	979	623	748	623
q18	7456	6845	6897	6845
q19	6195	1512	1542	1512
q20	649	318	300	300
q21	3494	2689	2889	2689
q22	368	307	302	302
Total cold run time: 113921 ms
Total hot run time: 38609 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4304	4293	4189	4189
q2	381	281	269	269
q3	3017	2763	2771	2763
q4	1878	1607	1590	1590
q5	5384	5342	5330	5330
q6	207	123	123	123
q7	2245	1881	1865	1865
q8	3205	3346	3327	3327
q9	8635	8601	8625	8601
q10	3893	3729	3713	3713
q11	596	488	488	488
q12	753	597	579	579
q13	16859	2939	2935	2935
q14	296	268	292	268
q15	512	467	469	467
q16	470	418	428	418
q17	1782	1476	1457	1457
q18	7720	7674	7377	7377
q19	1650	1548	1556	1548
q20	1982	1778	1771	1771
q21	5038	4776	4729	4729
q22	544	475	463	463
Total cold run time: 71351 ms
Total hot run time: 54270 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.49% (8907/25094)
Line Coverage: 27.22% (73148/268755)
Region Coverage: 26.34% (37818/143555)
Branch Coverage: 23.12% (19269/83332)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c9a16f05afa797649b9857795d075bef55c7e4f3_c9a16f05afa797649b9857795d075bef55c7e4f3/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 183123 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c9a16f05afa797649b9857795d075bef55c7e4f3, data reload: false

query1	910	375	372	372
query2	6505	2398	2195	2195
query3	6663	203	211	203
query4	24707	21320	21458	21320
query5	4134	404	410	404
query6	268	175	174	174
query7	4583	292	278	278
query8	224	175	198	175
query9	8495	2370	2378	2370
query10	608	260	255	255
query11	14665	14305	14152	14152
query12	140	92	88	88
query13	1643	369	375	369
query14	9961	7503	7914	7503
query15	259	183	180	180
query16	8169	264	254	254
query17	1942	591	542	542
query18	2103	270	284	270
query19	332	149	149	149
query20	96	85	86	85
query21	193	126	129	126
query22	5080	4876	4899	4876
query23	33689	33339	33003	33003
query24	12004	2913	2878	2878
query25	625	368	386	368
query26	1728	151	148	148
query27	3066	303	312	303
query28	7618	2004	2009	2004
query29	977	606	595	595
query30	302	174	179	174
query31	958	724	740	724
query32	89	53	54	53
query33	755	250	260	250
query34	1039	474	471	471
query35	852	694	711	694
query36	1083	894	893	893
query37	259	68	68	68
query38	3383	3196	3199	3196
query39	1562	1516	1532	1516
query40	268	124	121	121
query41	43	43	44	43
query42	100	93	97	93
query43	571	510	493	493
query44	1181	714	711	711
query45	285	267	255	255
query46	1070	716	721	716
query47	1933	1849	1858	1849
query48	364	296	296	296
query49	1163	377	362	362
query50	758	377	380	377
query51	6715	6652	6613	6613
query52	101	94	85	85
query53	346	274	273	273
query54	302	229	221	221
query55	76	73	69	69
query56	245	249	224	224
query57	1204	1137	1126	1126
query58	216	192	195	192
query59	3230	3071	3179	3071
query60	258	239	239	239
query61	92	89	86	86
query62	640	449	444	444
query63	304	273	276	273
query64	6296	3845	3754	3754
query65	3131	3037	3010	3010
query66	1361	347	336	336
query67	15577	14927	15258	14927
query68	5131	536	537	536
query69	470	301	293	293
query70	1231	1198	1198	1198
query71	409	274	262	262
query72	6460	2595	2465	2465
query73	728	322	317	317
query74	6880	6476	6666	6476
query75	3380	2614	2605	2605
query76	3396	953	892	892
query77	391	261	266	261
query78	10990	10307	10116	10116
query79	3246	515	518	515
query80	1797	428	418	418
query81	522	241	242	241
query82	775	95	93	93
query83	286	176	173	173
query84	259	90	89	89
query85	2085	270	305	270
query86	491	306	325	306
query87	3515	3327	3284	3284
query88	4525	2433	2484	2433
query89	474	376	377	376
query90	2019	179	177	177
query91	126	100	98	98
query92	63	47	48	47
query93	5023	510	494	494
query94	1246	180	179	179
query95	377	291	293	291
query96	593	270	268	268
query97	3130	2981	2930	2930
query98	237	228	217	217
query99	1228	866	866	866
Total cold run time: 290982 ms
Total hot run time: 183123 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 29.98 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c9a16f05afa797649b9857795d075bef55c7e4f3, data reload: false

query1	0.04	0.03	0.03
query2	0.07	0.04	0.04
query3	0.23	0.05	0.06
query4	1.66	0.07	0.08
query5	0.50	0.51	0.52
query6	1.48	0.73	0.72
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.54	0.49	0.49
query10	0.54	0.55	0.55
query11	0.16	0.12	0.11
query12	0.15	0.12	0.12
query13	0.61	0.58	0.58
query14	0.75	0.77	0.81
query15	0.83	0.80	0.81
query16	0.37	0.35	0.35
query17	0.94	0.96	1.00
query18	0.21	0.26	0.25
query19	1.73	1.66	1.68
query20	0.01	0.02	0.01
query21	15.41	0.66	0.65
query22	4.34	7.73	1.50
query23	18.30	1.32	1.24
query24	1.73	0.22	0.29
query25	0.15	0.09	0.08
query26	0.26	0.17	0.16
query27	0.08	0.08	0.08
query28	13.27	1.00	0.98
query29	12.57	3.27	3.24
query30	0.26	0.07	0.06
query31	2.87	0.38	0.38
query32	3.27	0.47	0.45
query33	2.81	2.85	2.83
query34	17.21	4.50	4.45
query35	4.47	4.49	4.50
query36	0.65	0.46	0.46
query37	0.18	0.16	0.15
query38	0.16	0.15	0.15
query39	0.04	0.03	0.03
query40	0.18	0.14	0.15
query41	0.10	0.06	0.05
query42	0.06	0.04	0.04
query43	0.04	0.03	0.03
Total cold run time: 109.29 s
Total hot run time: 29.98 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit c9a16f05afa797649b9857795d075bef55c7e4f3 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

@zzzxl1993
Copy link
Contributor Author

run p0

@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Apr 17, 2024
@zzzxl1993
Copy link
Contributor Author

run p0

@zzzxl1993
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 38758 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 498ef78112b8b5646ebf13c7375cde6f8e96428c, data reload: false

------ Round 1 ----------------------------------
q1	17741	4536	4297	4297
q2	2018	190	181	181
q3	10699	1131	1194	1131
q4	10597	752	848	752
q5	7739	2784	2708	2708
q6	222	137	141	137
q7	1027	653	622	622
q8	9567	2110	2072	2072
q9	7679	6754	6688	6688
q10	8987	3633	3546	3546
q11	450	229	229	229
q12	452	245	224	224
q13	17758	2910	2939	2910
q14	271	218	230	218
q15	523	479	472	472
q16	509	391	390	390
q17	958	760	667	667
q18	7410	6805	6699	6699
q19	5390	1528	1472	1472
q20	637	308	299	299
q21	3413	2740	2848	2740
q22	370	304	304	304
Total cold run time: 114417 ms
Total hot run time: 38758 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4334	4205	4246	4205
q2	370	265	269	265
q3	3049	2696	2794	2696
q4	1871	1645	1576	1576
q5	5325	5350	5310	5310
q6	210	126	129	126
q7	2251	1889	1894	1889
q8	3225	3381	3362	3362
q9	8629	8564	8607	8564
q10	3877	3715	3718	3715
q11	605	473	490	473
q12	745	591	579	579
q13	16307	2930	2941	2930
q14	311	267	287	267
q15	509	476	464	464
q16	462	424	435	424
q17	1775	1488	1463	1463
q18	7806	7460	7546	7460
q19	1627	1557	1517	1517
q20	1936	1764	1762	1762
q21	4904	4778	4887	4778
q22	540	464	450	450
Total cold run time: 70668 ms
Total hot run time: 54275 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.48% (8905/25098)
Line Coverage: 27.21% (73130/268789)
Region Coverage: 26.34% (37814/143583)
Branch Coverage: 23.12% (19263/83330)
Coverage Report: http://coverage.selectdb-in.cc/coverage/498ef78112b8b5646ebf13c7375cde6f8e96428c_498ef78112b8b5646ebf13c7375cde6f8e96428c/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 185029 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 498ef78112b8b5646ebf13c7375cde6f8e96428c, data reload: false

query1	905	373	362	362
query2	6492	2432	2535	2432
query3	6657	200	213	200
query4	24541	21357	21254	21254
query5	4139	403	408	403
query6	282	172	175	172
query7	4596	299	294	294
query8	225	180	169	169
query9	8513	2370	2355	2355
query10	577	243	251	243
query11	14800	14130	14155	14130
query12	135	89	97	89
query13	1646	376	372	372
query14	9687	8086	7960	7960
query15	282	187	180	180
query16	8167	267	260	260
query17	1906	591	551	551
query18	2209	276	270	270
query19	329	150	155	150
query20	94	86	88	86
query21	213	135	128	128
query22	5154	4981	4945	4945
query23	34019	33266	33127	33127
query24	12082	2989	2979	2979
query25	647	366	368	366
query26	1741	153	150	150
query27	2934	302	309	302
query28	7603	2025	2001	2001
query29	1013	597	592	592
query30	315	172	170	170
query31	990	760	741	741
query32	94	52	54	52
query33	744	242	243	242
query34	1011	473	478	473
query35	889	686	707	686
query36	1039	922	907	907
query37	273	69	70	69
query38	3319	3200	3237	3200
query39	1569	1514	1525	1514
query40	272	130	128	128
query41	47	43	45	43
query42	101	95	99	95
query43	549	505	521	505
query44	1263	717	722	717
query45	281	274	294	274
query46	1079	708	741	708
query47	1930	1820	1862	1820
query48	360	286	298	286
query49	1155	381	366	366
query50	755	381	387	381
query51	6762	6558	6648	6558
query52	110	88	89	88
query53	341	276	281	276
query54	324	237	228	228
query55	74	73	73	73
query56	244	225	224	224
query57	1233	1134	1137	1134
query58	222	196	200	196
query59	3436	3149	3099	3099
query60	250	228	233	228
query61	90	115	87	87
query62	682	437	441	437
query63	302	280	280	280
query64	6295	3956	3892	3892
query65	3052	3038	3050	3038
query66	1365	331	334	331
query67	15189	14949	14880	14880
query68	5301	541	533	533
query69	493	304	302	302
query70	1211	1197	1223	1197
query71	1408	1273	1272	1272
query72	6421	2597	2459	2459
query73	721	323	322	322
query74	6672	6367	6350	6350
query75	3447	2652	2590	2590
query76	3357	1028	946	946
query77	429	263	258	258
query78	10869	10293	10104	10104
query79	3435	527	532	527
query80	1250	428	432	428
query81	509	250	241	241
query82	995	98	93	93
query83	196	172	170	170
query84	267	88	83	83
query85	1430	268	303	268
query86	466	313	307	307
query87	3464	3285	3284	3284
query88	4668	2407	2419	2407
query89	483	381	384	381
query90	2064	179	183	179
query91	139	103	97	97
query92	58	49	48	48
query93	4763	504	496	496
query94	1162	178	179	178
query95	392	315	296	296
query96	580	260	267	260
query97	3120	2935	2923	2923
query98	240	218	216	216
query99	1259	866	882	866
Total cold run time: 290863 ms
Total hot run time: 185029 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.5 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 498ef78112b8b5646ebf13c7375cde6f8e96428c, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.04	0.04
query3	0.23	0.05	0.05
query4	1.67	0.08	0.09
query5	0.52	0.51	0.51
query6	1.46	0.72	0.72
query7	0.02	0.01	0.01
query8	0.05	0.04	0.05
query9	0.54	0.50	0.51
query10	0.54	0.56	0.55
query11	0.15	0.12	0.12
query12	0.14	0.12	0.13
query13	0.60	0.58	0.58
query14	0.77	0.76	0.76
query15	0.82	0.79	0.79
query16	0.36	0.37	0.37
query17	1.01	1.00	1.02
query18	0.22	0.21	0.25
query19	1.81	1.65	1.75
query20	0.01	0.02	0.01
query21	15.43	0.65	0.65
query22	4.44	7.56	2.00
query23	18.32	1.39	1.26
query24	1.30	0.43	0.21
query25	0.15	0.09	0.08
query26	0.25	0.17	0.16
query27	0.08	0.08	0.08
query28	13.37	0.99	0.98
query29	12.56	3.27	3.26
query30	0.25	0.07	0.06
query31	2.87	0.39	0.36
query32	3.29	0.45	0.46
query33	2.84	2.87	2.88
query34	17.08	4.44	4.44
query35	4.50	4.49	4.48
query36	0.66	0.45	0.46
query37	0.19	0.15	0.14
query38	0.15	0.15	0.14
query39	0.05	0.04	0.03
query40	0.17	0.13	0.14
query41	0.10	0.05	0.05
query42	0.05	0.05	0.04
query43	0.04	0.04	0.04
Total cold run time: 109.18 s
Total hot run time: 30.5 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 498ef78112b8b5646ebf13c7375cde6f8e96428c with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      33 seconds loaded 861443392 Bytes, about 24 MB/s
Insert into select:       13.6 seconds inserted 10000000 Rows, about 735K ops/s

@zzzxl1993
Copy link
Contributor Author

run p0

@github-actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 17, 2024
@qidaye qidaye merged commit 8522d63 into apache:master Apr 17, 2024
yiguolei pushed a commit that referenced this pull request Apr 17, 2024
yiguolei pushed a commit that referenced this pull request Apr 17, 2024
xiaokang pushed a commit that referenced this pull request Apr 18, 2024
dataroaring pushed a commit that referenced this pull request Apr 24, 2024
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.0.9-merged reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants