[opt](Inverted index)Avoid repeated calculations of suffix paths #48137

qidaye · 2025-02-20T07:53:06Z

What problem does this PR solve?

Introduced a new local variable escaped_suffix to store the result of escape_for_path_name(suffix_path).
This change improves performance by avoiding repeated function calls.

Release note

None

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2025-02-20T07:53:12Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

qidaye · 2025-02-20T07:55:24Z

run buildall

eldenmoon

LGTM

github-actions · 2025-02-20T07:58:01Z

PR approved by at least one committer and no changes requested.

github-actions · 2025-02-20T07:58:04Z

PR approved by anyone and no changes requested.

doris-robot · 2025-02-20T08:32:33Z

TPC-H: Total hot run time: 31192 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a2f9e4b46bb259501b3c48f0fa2eb57d53f312a4, data reload: false

------ Round 1 ----------------------------------
q1	17590	5292	5060	5060
q2	2051	296	173	173
q3	10402	1237	746	746
q4	10214	1030	543	543
q5	7537	2459	2257	2257
q6	188	165	132	132
q7	910	751	589	589
q8	9295	1310	1156	1156
q9	4806	4599	4495	4495
q10	6823	2309	1859	1859
q11	469	279	257	257
q12	341	353	221	221
q13	17754	3706	3061	3061
q14	229	250	203	203
q15	509	457	469	457
q16	627	622	574	574
q17	571	846	332	332
q18	6927	6217	6151	6151
q19	1222	965	525	525
q20	312	323	194	194
q21	2835	2214	1892	1892
q22	363	339	315	315
Total cold run time: 101975 ms
Total hot run time: 31192 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5084	5122	5092	5092
q2	238	325	228	228
q3	2191	2694	2334	2334
q4	1469	1859	1348	1348
q5	4267	4159	4184	4159
q6	215	167	127	127
q7	1876	1841	1647	1647
q8	2651	2723	2612	2612
q9	7446	7443	7392	7392
q10	3123	3229	2863	2863
q11	605	514	501	501
q12	701	795	648	648
q13	3414	3982	3341	3341
q14	289	307	285	285
q15	504	467	465	465
q16	655	681	629	629
q17	1137	1601	1361	1361
q18	7526	7403	7325	7325
q19	773	897	1009	897
q20	2113	2092	1936	1936
q21	5614	5248	4965	4965
q22	625	598	554	554
Total cold run time: 52516 ms
Total hot run time: 50709 ms

qidaye · 2025-02-20T08:35:56Z

run buildall

doris-robot · 2025-02-20T09:06:39Z

TPC-H: Total hot run time: 31702 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1271796a5a80fbdab9a9c5a5913fcc2036e6cf6, data reload: false

------ Round 1 ----------------------------------
q1	18593	5310	5149	5149
q2	2048	294	167	167
q3	11616	1347	752	752
q4	10199	1038	554	554
q5	7542	2426	2387	2387
q6	198	168	138	138
q7	908	772	611	611
q8	9371	1357	1120	1120
q9	5787	4784	4740	4740
q10	6888	2334	1933	1933
q11	465	274	251	251
q12	348	366	224	224
q13	18165	3779	3101	3101
q14	245	237	217	217
q15	510	467	459	459
q16	694	612	577	577
q17	758	870	339	339
q18	6968	6098	6029	6029
q19	2284	937	531	531
q20	304	313	195	195
q21	2735	2213	1923	1923
q22	364	335	305	305
Total cold run time: 106990 ms
Total hot run time: 31702 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5276	5137	5146	5137
q2	242	347	223	223
q3	2143	2712	2303	2303
q4	1486	1825	1368	1368
q5	4196	4113	4124	4113
q6	208	167	127	127
q7	1965	1918	1706	1706
q8	2600	2618	2520	2520
q9	7257	7353	7328	7328
q10	3102	3274	2855	2855
q11	571	508	474	474
q12	730	796	637	637
q13	3666	4083	3418	3418
q14	284	319	284	284
q15	528	481	462	462
q16	655	710	653	653
q17	1149	1649	1339	1339
q18	7701	7380	7278	7278
q19	830	848	1096	848
q20	1951	2014	1861	1861
q21	5502	4923	4693	4693
q22	629	563	563	563
Total cold run time: 52671 ms
Total hot run time: 50190 ms

doris-robot · 2025-02-20T09:18:48Z

TPC-DS: Total hot run time: 192133 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1271796a5a80fbdab9a9c5a5913fcc2036e6cf6, data reload: false

query1	1326	967	949	949
query2	6114	1908	1873	1873
query3	11083	4543	4450	4450
query4	56495	25914	23949	23949
query5	5150	516	498	498
query6	374	199	186	186
query7	5108	500	296	296
query8	325	248	248	248
query9	6432	2537	2565	2537
query10	414	314	264	264
query11	15535	15099	15041	15041
query12	152	108	106	106
query13	1138	547	400	400
query14	10664	6469	7008	6469
query15	215	197	174	174
query16	7161	638	503	503
query17	1080	747	607	607
query18	1502	447	324	324
query19	207	196	179	179
query20	131	131	128	128
query21	219	130	118	118
query22	4352	4557	4470	4470
query23	33927	33377	33598	33377
query24	5634	2558	2454	2454
query25	484	473	404	404
query26	696	269	162	162
query27	1757	504	353	353
query28	2762	2462	2415	2415
query29	558	572	429	429
query30	212	197	156	156
query31	916	895	868	868
query32	71	63	64	63
query33	455	358	311	311
query34	784	869	522	522
query35	801	836	767	767
query36	999	1018	892	892
query37	118	110	69	69
query38	4326	4401	4382	4382
query39	1516	1455	1443	1443
query40	211	117	104	104
query41	51	53	47	47
query42	122	110	110	110
query43	520	527	509	509
query44	1343	836	820	820
query45	179	172	167	167
query46	878	1070	682	682
query47	1864	1856	1798	1798
query48	390	438	321	321
query49	712	506	413	413
query50	715	736	433	433
query51	4275	4324	4286	4286
query52	111	112	105	105
query53	239	261	188	188
query54	489	509	428	428
query55	87	80	78	78
query56	270	301	232	232
query57	1216	1194	1119	1119
query58	249	250	251	250
query59	3047	3022	2967	2967
query60	290	300	267	267
query61	115	112	132	112
query62	686	696	653	653
query63	226	187	185	185
query64	1458	985	723	723
query65	3205	3173	3152	3152
query66	697	382	291	291
query67	16001	15533	15502	15502
query68	5328	786	514	514
query69	513	293	276	276
query70	1230	1149	1121	1121
query71	453	294	272	272
query72	6331	3603	3684	3603
query73	1019	733	364	364
query74	9209	9057	9015	9015
query75	3441	3140	2685	2685
query76	3714	1173	735	735
query77	544	355	282	282
query78	10186	10098	9305	9305
query79	1859	849	630	630
query80	685	557	444	444
query81	524	276	245	245
query82	418	127	96	96
query83	177	246	161	161
query84	287	95	72	72
query85	746	348	309	309
query86	330	304	267	267
query87	4531	4638	4463	4463
query88	3007	2221	2253	2221
query89	404	314	286	286
query90	1788	199	190	190
query91	133	146	108	108
query92	64	57	55	55
query93	2177	1015	587	587
query94	613	397	265	265
query95	349	278	267	267
query96	510	546	264	264
query97	2741	2849	2722	2722
query98	216	202	193	193
query99	1290	1405	1338	1338
Total cold run time: 295656 ms
Total hot run time: 192133 ms

doris-robot · 2025-02-20T09:24:05Z

ClickBench: Total hot run time: 30.46 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c1271796a5a80fbdab9a9c5a5913fcc2036e6cf6, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.41	0.40	0.40
query6	1.16	0.67	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.04
query9	0.62	0.54	0.51
query10	0.58	0.57	0.58
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.64	0.61	0.61
query14	2.68	2.73	2.71
query15	0.93	0.86	0.86
query16	0.37	0.37	0.38
query17	1.01	1.03	1.03
query18	0.21	0.20	0.19
query19	1.89	1.81	2.02
query20	0.01	0.02	0.01
query21	15.36	0.87	0.56
query22	0.76	1.17	0.72
query23	14.89	1.43	0.63
query24	11.76	0.90	0.58
query25	0.38	0.25	0.17
query26	0.94	0.17	0.14
query27	0.05	0.04	0.04
query28	6.02	0.78	0.43
query29	12.55	4.00	3.28
query30	0.24	0.09	0.06
query31	2.84	0.58	0.38
query32	3.24	0.56	0.48
query33	2.98	2.95	3.04
query34	15.78	5.17	4.48
query35	4.56	4.54	4.58
query36	0.67	0.49	0.48
query37	0.09	0.06	0.06
query38	0.05	0.04	0.04
query39	0.02	0.02	0.03
query40	0.17	0.14	0.13
query41	0.07	0.02	0.03
query42	0.03	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.31 s
Total hot run time: 30.46 s

doris-robot · 2025-02-20T09:26:45Z

TeamCity be ut coverage result:
Function Coverage: 43.83% (11639/26556)
Line Coverage: 33.76% (97594/289092)
Region Coverage: 32.85% (49953/152064)
Branch Coverage: 28.57% (25109/87900)
Coverage Report: http://coverage.selectdb-in.cc/coverage/c1271796a5a80fbdab9a9c5a5913fcc2036e6cf6_c1271796a5a80fbdab9a9c5a5913fcc2036e6cf6/report/index.html

xiaokang

LGTM

github-actions · 2025-02-20T09:57:38Z

PR approved by at least one committer and no changes requested.

eldenmoon

LGTM

) Introduced a new local variable `escaped_suffix` to store the result of `escape_for_path_name(suffix_path)`. This change improves performance by avoiding repeated function calls.

…che#48137)

…x paths #48137 (#48153) Cherry-picked from #48137 Co-authored-by: qiye <luen@selectdb.com>

…che#48137) Introduced a new local variable `escaped_suffix` to store the result of `escape_for_path_name(suffix_path)`. This change improves performance by avoiding repeated function calls.

) (#48155) bp #48137

…che#48137) Introduced a new local variable `escaped_suffix` to store the result of `escape_for_path_name(suffix_path)`. This change improves performance by avoiding repeated function calls.

qidaye changed the title ~~[opt]Avoid repeated calculations of suffix paths~~ [opt](Inverted index)Avoid repeated calculations of suffix paths Feb 20, 2025

eldenmoon previously approved these changes Feb 20, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 20, 2025

github-actions bot added the reviewed label Feb 20, 2025

eldenmoon added dev/2.1.x dev/3.0.x labels Feb 20, 2025

[opt]Avoid repeated calculations of suffix paths

c127179

qidaye dismissed eldenmoon’s stale review via c127179 February 20, 2025 08:16

qidaye force-pushed the opt_escape_suffix_path branch from a2f9e4b to c127179 Compare February 20, 2025 08:16

github-actions bot removed the approved Indicates a PR has been approved by one committer. label Feb 20, 2025

xiaokang approved these changes Feb 20, 2025

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 20, 2025

eldenmoon approved these changes Feb 20, 2025

View reviewed changes

qidaye merged commit 3b18d72 into apache:master Feb 20, 2025
24 of 27 checks passed

qidaye deleted the opt_escape_suffix_path branch February 20, 2025 11:59

github-actions bot mentioned this pull request Feb 20, 2025

branch-3.0: [opt](Inverted index)Avoid repeated calculations of suffix paths #48137 #48153

Merged

github-actions bot added the dev/2.1.x-conflict label Feb 20, 2025

qidaye pushed a commit to qidaye/incubator-doris that referenced this pull request Feb 20, 2025

[opt](Inverted index)Avoid repeated calculations of suffix paths (apa…

1260fca

…che#48137)

qidaye mentioned this pull request Feb 20, 2025

[opt](Inverted index)Avoid repeated calculations of suffix paths (#48137) #48155

Merged

dataroaring pushed a commit that referenced this pull request Feb 21, 2025

branch-3.0: [opt](Inverted index)Avoid repeated calculations of suffi…

a11296a

…x paths #48137 (#48153) Cherry-picked from #48137 Co-authored-by: qiye <luen@selectdb.com>

dataroaring added dev/3.0.5-merged and removed dev/3.0.x labels Feb 21, 2025

qidaye removed dev/2.1.x dev/2.1.x-conflict labels Feb 21, 2025

xiaokang added usercase Important user case type label dev/2.1.x labels Mar 4, 2025

github-actions bot added the dev/2.1.x-conflict label Mar 4, 2025

xiaokang pushed a commit that referenced this pull request Mar 5, 2025

[opt](Inverted index)Avoid repeated calculations of suffix paths (#48137

84c638e

) (#48155) bp #48137

xiaokang added dev/2.1.9-merged and removed dev/2.1.x dev/2.1.x-conflict labels Mar 5, 2025

[opt](Inverted index)Avoid repeated calculations of suffix paths #48137

[opt](Inverted index)Avoid repeated calculations of suffix paths #48137

Uh oh!

Conversation

qidaye commented Feb 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Feb 20, 2025

Uh oh!

qidaye commented Feb 20, 2025

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 20, 2025

Uh oh!

github-actions bot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

qidaye commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

doris-robot commented Feb 20, 2025

Uh oh!

xiaokang left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Feb 20, 2025

Uh oh!

eldenmoon left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

qidaye commented Feb 20, 2025 •

edited

Loading