Skip to content

Conversation

@morrySnow
Copy link
Contributor

What problem does this PR solve?

Related PR: #40441

Problem Summary:

The delimiters in split-related string functions that have special meanings in regular expressions should be escaped.

.$|()[{^?*+\

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 6, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morrySnow morrySnow added usercase Important user case type label p0_w dev/2.1.x dev/3.0.x labels Mar 6, 2025
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32553 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4b924838e71d65f4ef4bc9c75e6e853285d80022, data reload: false

------ Round 1 ----------------------------------
q1	17584	5221	5061	5061
q2	2044	301	165	165
q3	10408	1248	739	739
q4	10203	1015	531	531
q5	7534	2362	2288	2288
q6	192	165	132	132
q7	884	726	609	609
q8	9294	1252	1051	1051
q9	4835	4817	4724	4724
q10	6854	2300	1908	1908
q11	484	278	259	259
q12	346	353	216	216
q13	17768	3663	3113	3113
q14	228	232	211	211
q15	538	497	487	487
q16	622	638	589	589
q17	558	850	347	347
q18	6799	6533	6457	6457
q19	1569	966	541	541
q20	329	321	202	202
q21	2883	2133	1950	1950
q22	1057	1031	973	973
Total cold run time: 103013 ms
Total hot run time: 32553 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5165	5100	5119	5100
q2	244	328	242	242
q3	2147	2647	2322	2322
q4	1444	1891	1358	1358
q5	4204	4090	4160	4090
q6	211	163	129	129
q7	1844	1893	1759	1759
q8	2636	2594	2517	2517
q9	7258	7163	7118	7118
q10	3217	3261	2791	2791
q11	579	501	504	501
q12	714	789	617	617
q13	3423	3931	3279	3279
q14	276	308	286	286
q15	521	475	465	465
q16	644	684	641	641
q17	1117	1562	1333	1333
q18	7846	7674	7571	7571
q19	786	796	811	796
q20	1954	2017	1872	1872
q21	5375	4978	4694	4694
q22	1133	1082	1017	1017
Total cold run time: 52738 ms
Total hot run time: 50498 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 191136 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4b924838e71d65f4ef4bc9c75e6e853285d80022, data reload: false

query1	1373	1015	992	992
query2	6140	2041	1969	1969
query3	11043	4494	4563	4494
query4	53047	25837	23049	23049
query5	5212	570	482	482
query6	345	200	192	192
query7	4885	513	293	293
query8	320	246	241	241
query9	5660	2571	2565	2565
query10	430	305	259	259
query11	15194	14974	14796	14796
query12	171	116	108	108
query13	1046	521	408	408
query14	10631	6710	6442	6442
query15	242	200	169	169
query16	7124	639	467	467
query17	1062	723	555	555
query18	1560	404	305	305
query19	212	186	161	161
query20	123	121	133	121
query21	201	134	114	114
query22	4426	4727	4426	4426
query23	33979	33492	33164	33164
query24	5712	2454	2430	2430
query25	484	458	395	395
query26	716	288	161	161
query27	1694	505	339	339
query28	2783	2461	2429	2429
query29	575	575	439	439
query30	276	230	194	194
query31	876	869	807	807
query32	74	58	59	58
query33	485	347	294	294
query34	767	865	497	497
query35	797	860	761	761
query36	952	1011	907	907
query37	120	99	72	72
query38	4183	4169	4152	4152
query39	1464	1421	1412	1412
query40	212	117	102	102
query41	52	51	52	51
query42	130	103	108	103
query43	506	534	505	505
query44	1294	804	792	792
query45	179	171	163	163
query46	859	1031	642	642
query47	1885	1888	1787	1787
query48	381	408	310	310
query49	706	536	451	451
query50	713	787	436	436
query51	4306	4377	4257	4257
query52	110	105	97	97
query53	232	260	182	182
query54	473	496	429	429
query55	83	87	82	82
query56	269	264	262	262
query57	1181	1192	1118	1118
query58	251	240	236	236
query59	3012	3098	2694	2694
query60	302	279	257	257
query61	156	119	118	118
query62	732	756	675	675
query63	235	192	190	190
query64	1509	1011	678	678
query65	4539	4397	4433	4397
query66	759	400	322	322
query67	15873	15528	15375	15375
query68	5517	820	506	506
query69	555	308	275	275
query70	1208	1124	1111	1111
query71	426	304	258	258
query72	5745	3571	3649	3571
query73	1345	727	345	345
query74	9229	9121	8659	8659
query75	3209	3129	2684	2684
query76	3781	1177	729	729
query77	550	354	277	277
query78	9987	10096	9289	9289
query79	2598	826	606	606
query80	667	510	466	466
query81	507	260	215	215
query82	528	126	93	93
query83	171	177	168	168
query84	285	88	71	71
query85	765	346	299	299
query86	388	317	291	291
query87	4340	4682	4377	4377
query88	3741	2266	2238	2238
query89	503	314	287	287
query90	1716	208	208	208
query91	139	141	106	106
query92	73	58	59	58
query93	2645	1026	569	569
query94	687	408	295	295
query95	343	271	261	261
query96	473	573	282	282
query97	3385	3400	3253	3253
query98	219	200	199	199
query99	1350	1391	1260	1260
Total cold run time: 294398 ms
Total hot run time: 191136 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.21 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4b924838e71d65f4ef4bc9c75e6e853285d80022, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.06	0.06
query4	1.62	0.10	0.11
query5	0.55	0.54	0.54
query6	1.19	0.71	0.71
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.52	0.51
query10	0.56	0.60	0.58
query11	0.16	0.10	0.11
query12	0.14	0.11	0.11
query13	0.62	0.62	0.60
query14	2.84	2.66	2.70
query15	0.92	0.85	0.84
query16	0.37	0.37	0.37
query17	1.00	1.02	1.01
query18	0.21	0.19	0.20
query19	1.90	1.77	1.96
query20	0.02	0.01	0.01
query21	15.36	0.88	0.53
query22	0.75	1.22	0.70
query23	14.84	1.34	0.63
query24	6.76	1.76	1.11
query25	0.52	0.14	0.25
query26	0.70	0.16	0.13
query27	0.06	0.05	0.05
query28	10.03	0.85	0.43
query29	12.57	4.13	3.45
query30	0.26	0.09	0.06
query31	2.82	0.58	0.39
query32	3.22	0.54	0.46
query33	2.97	3.05	3.08
query34	15.78	5.09	4.52
query35	4.51	4.55	4.51
query36	0.66	0.49	0.47
query37	0.09	0.06	0.06
query38	0.06	0.04	0.03
query39	0.03	0.02	0.02
query40	0.17	0.13	0.13
query41	0.09	0.03	0.02
query42	0.03	0.02	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.38 s
Total hot run time: 31.21 s

@morrySnow morrySnow force-pushed the fix_string_folding branch from 4b92483 to 3d6f119 Compare March 6, 2025 16:04
@morrySnow
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 32613 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 3d6f1191b02a912e7e044cd510a491b736a8653d, data reload: false

------ Round 1 ----------------------------------
q1	17610	5117	5172	5117
q2	2051	303	163	163
q3	10426	1276	736	736
q4	10249	1014	530	530
q5	7799	2369	2305	2305
q6	189	166	139	139
q7	882	738	592	592
q8	9502	1273	1036	1036
q9	4863	4837	4877	4837
q10	6817	2297	1910	1910
q11	483	278	259	259
q12	347	360	217	217
q13	17774	3707	3082	3082
q14	245	228	214	214
q15	541	497	480	480
q16	643	604	614	604
q17	576	836	340	340
q18	6810	6517	6360	6360
q19	1215	938	536	536
q20	317	326	195	195
q21	2804	2185	1979	1979
q22	1051	1020	982	982
Total cold run time: 103194 ms
Total hot run time: 32613 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5164	5115	5117	5115
q2	238	331	233	233
q3	2178	2674	2305	2305
q4	1412	1902	1355	1355
q5	4229	4105	4111	4105
q6	211	162	123	123
q7	1891	1883	1752	1752
q8	2579	2625	2603	2603
q9	7194	7132	7193	7132
q10	2987	3273	2768	2768
q11	582	511	495	495
q12	717	796	594	594
q13	3416	4069	3246	3246
q14	276	281	266	266
q15	517	480	463	463
q16	647	678	660	660
q17	1158	1625	1348	1348
q18	7849	7570	7406	7406
q19	795	798	857	798
q20	1978	1996	1919	1919
q21	5399	4997	4688	4688
q22	1116	1092	1030	1030
Total cold run time: 52533 ms
Total hot run time: 50404 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 192450 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 3d6f1191b02a912e7e044cd510a491b736a8653d, data reload: false

query1	1365	1000	982	982
query2	6218	2014	1964	1964
query3	11040	4433	4430	4430
query4	53681	25629	23357	23357
query5	5067	528	501	501
query6	352	202	180	180
query7	4955	502	291	291
query8	300	225	224	224
query9	5806	2546	2555	2546
query10	443	314	261	261
query11	15168	15241	14941	14941
query12	162	108	102	102
query13	1079	494	373	373
query14	10154	6487	6864	6487
query15	193	205	175	175
query16	6986	674	476	476
query17	1075	715	555	555
query18	1555	403	313	313
query19	193	197	164	164
query20	122	121	125	121
query21	211	125	109	109
query22	4491	4648	4314	4314
query23	33920	33470	33270	33270
query24	5672	2395	2425	2395
query25	496	465	396	396
query26	677	282	162	162
query27	1731	483	328	328
query28	2718	2484	2475	2475
query29	576	547	435	435
query30	285	217	197	197
query31	889	887	792	792
query32	76	66	62	62
query33	469	372	312	312
query34	765	874	514	514
query35	767	853	789	789
query36	968	1020	917	917
query37	119	110	75	75
query38	4151	4190	4228	4190
query39	1482	1441	1443	1441
query40	211	115	104	104
query41	51	55	53	53
query42	131	108	107	107
query43	526	516	508	508
query44	1300	806	810	806
query45	178	179	207	179
query46	840	1030	681	681
query47	1860	1862	1803	1803
query48	384	424	310	310
query49	692	527	437	437
query50	707	746	430	430
query51	4355	4335	4194	4194
query52	107	108	98	98
query53	230	261	196	196
query54	491	502	427	427
query55	89	86	89	86
query56	274	279	263	263
query57	1154	1167	1106	1106
query58	256	259	240	240
query59	2811	3129	2931	2931
query60	306	269	271	269
query61	133	118	116	116
query62	742	762	690	690
query63	231	200	183	183
query64	1429	1025	674	674
query65	4606	4464	4380	4380
query66	715	391	308	308
query67	15843	15332	15397	15332
query68	5496	833	502	502
query69	508	301	270	270
query70	1221	1117	1115	1115
query71	419	302	281	281
query72	6108	3624	3744	3624
query73	1260	738	341	341
query74	9101	9047	8988	8988
query75	3287	3183	2688	2688
query76	3748	1175	748	748
query77	538	352	276	276
query78	9989	10113	9284	9284
query79	2170	841	584	584
query80	704	525	458	458
query81	500	262	215	215
query82	212	127	97	97
query83	174	170	158	158
query84	285	90	69	69
query85	783	352	306	306
query86	346	303	290	290
query87	4458	4549	4513	4513
query88	3538	2246	2235	2235
query89	399	330	298	298
query90	1804	214	204	204
query91	141	145	112	112
query92	72	62	55	55
query93	1862	1068	569	569
query94	674	395	294	294
query95	420	263	266	263
query96	476	585	276	276
query97	3328	3400	3309	3309
query98	222	218	197	197
query99	1354	1393	1304	1304
Total cold run time: 292626 ms
Total hot run time: 192450 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.87 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 3d6f1191b02a912e7e044cd510a491b736a8653d, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.03
query3	0.23	0.06	0.07
query4	1.62	0.10	0.12
query5	0.56	0.56	0.54
query6	1.18	0.70	0.71
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.58	0.52	0.51
query10	0.58	0.60	0.58
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.82	2.70	2.66
query15	0.93	0.86	0.84
query16	0.38	0.38	0.40
query17	1.02	1.07	1.04
query18	0.22	0.19	0.20
query19	1.89	1.86	1.92
query20	0.01	0.01	0.01
query21	15.36	0.90	0.55
query22	0.76	1.19	0.63
query23	14.97	1.41	0.62
query24	7.47	0.75	1.19
query25	0.49	0.11	0.28
query26	0.66	0.16	0.14
query27	0.05	0.05	0.05
query28	9.81	0.87	0.43
query29	12.57	4.19	3.46
query30	0.25	0.10	0.08
query31	2.82	0.59	0.39
query32	3.23	0.54	0.46
query33	2.98	3.04	3.03
query34	16.04	5.11	4.48
query35	4.51	4.55	4.48
query36	0.67	0.50	0.50
query37	0.09	0.06	0.07
query38	0.05	0.04	0.03
query39	0.04	0.02	0.03
query40	0.16	0.13	0.13
query41	0.09	0.03	0.02
query42	0.03	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 106.23 s
Total hot run time: 30.87 s

Copy link
Contributor

@zhiqiang-hhhh zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions
Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by anyone and no changes requested.

Copy link
Contributor

@LiBinfeng-01 LiBinfeng-01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 7, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Mar 7, 2025

PR approved by at least one committer and no changes requested.

@morrySnow morrySnow merged commit 5e64736 into apache:master Mar 7, 2025
27 of 28 checks passed
@morrySnow morrySnow deleted the fix_string_folding branch March 7, 2025 08:36
github-actions bot pushed a commit that referenced this pull request Mar 7, 2025
…#48783)

### What problem does this PR solve?

Related PR: #40441

Problem Summary:

The delimiters in split-related string functions that have special
meanings in regular expressions should be escaped.

.$|()[{^?*+\
dataroaring pushed a commit that referenced this pull request Mar 10, 2025
…m by mistake #48783 (#48822)

Cherry-picked from #48783

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
morrySnow added a commit that referenced this pull request Mar 10, 2025
…m by mistake #48783

pick from master #48783

Related PR: #40441

Problem Summary:

The delimiters in split-related string functions that have special
meanings in regular expressions should be escaped.

.$|()[{^?*+\
yiguolei pushed a commit that referenced this pull request Mar 11, 2025
@gavinchou gavinchou mentioned this pull request Apr 23, 2025
deardeng pushed a commit to deardeng/incubator-doris that referenced this pull request Apr 30, 2025
…m by mistake apache#48783 (apache#48876) (apache#355)

pick from master apache#48783

Co-authored-by: morrySnow <zhangwenxin@selectdb.com>
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
…apache#48783)

### What problem does this PR solve?

Related PR: apache#40441

Problem Summary:

The delimiters in split-related string functions that have special
meanings in regular expressions should be escaped.

.$|()[{^?*+\
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.9-merged dev/3.0.5-merged p0_w reviewed usercase Important user case type label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants