Skip to content

[fix](nereids) fix fold constant return wrong scale of datetime type #50142

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
924060929 merged 9 commits into from
Apr 21, 2025
Merged

[fix](nereids) fix fold constant return wrong scale of datetime type #50142

924060929 merged 9 commits into from
Apr 21, 2025

Conversation

@924060929
Copy link
Contributor

@924060929 924060929 commented Apr 17, 2025

What problem does this PR solve?

fix fold constant return wrong scale of datetime type

for example, this sql should return data type datatime(6), but the fold constant rule will direct return the first argument, so return the wrong data type datetime(0)

select nvl(cast('2025-04-17 01:02:03' as datetime(0)), cast('2025-04-17 01:02:03.123456' as datetime(6)))

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

hello-stephen commented Apr 17, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@924060929
Copy link
Contributor Author

924060929 commented Apr 17, 2025

run buildall

@924060929
Copy link
Contributor Author

924060929 commented Apr 17, 2025

run buildall

@924060929
Copy link
Contributor Author

924060929 commented Apr 17, 2025

run buildall

@924060929
Copy link
Contributor Author

924060929 commented Apr 17, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 17, 2025

TPC-H: Total hot run time: 33979 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e5624acea876a32787cd79a52b1a775949d09b64, data reload: false

------ Round 1 ----------------------------------
q1	26022	5072	5106	5072
q2	2062	278	178	178
q3	10389	1215	720	720
q4	10224	984	524	524
q5	7543	2392	2302	2302
q6	185	160	131	131
q7	903	778	636	636
q8	9302	1271	998	998
q9	6955	5084	5120	5084
q10	6849	2306	1890	1890
q11	477	286	281	281
q12	345	354	222	222
q13	17781	3680	3170	3170
q14	218	231	205	205
q15	548	493	481	481
q16	453	444	403	403
q17	585	849	372	372
q18	7487	7372	7078	7078
q19	2027	937	555	555
q20	334	326	220	220
q21	4113	3369	2472	2472
q22	1052	985	999	985
Total cold run time: 115854 ms
Total hot run time: 33979 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5215	5044	5080	5044
q2	234	333	234	234
q3	2189	2688	2257	2257
q4	1401	1807	1432	1432
q5	4415	4405	4337	4337
q6	214	168	126	126
q7	1971	1903	1772	1772
q8	2614	2547	2548	2547
q9	7212	7323	7185	7185
q10	2976	3198	2747	2747
q11	577	512	495	495
q12	683	761	595	595
q13	3522	3853	3297	3297
q14	314	298	272	272
q15	528	488	480	480
q16	466	509	465	465
q17	1154	1524	1446	1446
q18	7769	7671	7481	7481
q19	808	803	991	803
q20	1925	1982	1833	1833
q21	5201	4830	4770	4770
q22	1129	1043	1017	1017
Total cold run time: 52517 ms
Total hot run time: 50635 ms

@doris-robot
Copy link

doris-robot commented Apr 17, 2025

TPC-DS: Total hot run time: 193012 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e5624acea876a32787cd79a52b1a775949d09b64, data reload: false

query1	1395	1089	1040	1040
query2	6354	1776	1779	1776
query3	11183	4606	4463	4463
query4	53633	25389	23513	23513
query5	4947	507	470	470
query6	347	206	196	196
query7	4877	489	273	273
query8	315	244	223	223
query9	5193	2562	2555	2555
query10	427	311	254	254
query11	15136	15028	14834	14834
query12	153	113	105	105
query13	1025	514	407	407
query14	10102	6421	6337	6337
query15	216	197	187	187
query16	7082	698	507	507
query17	1074	730	560	560
query18	1555	409	298	298
query19	203	186	173	173
query20	132	121	128	121
query21	207	123	106	106
query22	4568	4671	4416	4416
query23	34164	33567	33557	33557
query24	6707	2406	2454	2406
query25	494	467	399	399
query26	676	270	149	149
query27	2289	512	333	333
query28	2987	2147	2132	2132
query29	556	555	439	439
query30	272	224	186	186
query31	863	843	796	796
query32	73	60	64	60
query33	452	351	308	308
query34	779	869	519	519
query35	810	850	777	777
query36	938	1013	887	887
query37	130	103	85	85
query38	4172	4260	4256	4256
query39	1532	1457	1465	1457
query40	228	123	118	118
query41	59	57	59	57
query42	127	114	114	114
query43	510	495	468	468
query44	1339	824	811	811
query45	185	192	169	169
query46	851	1041	676	676
query47	1876	1922	1835	1835
query48	382	418	339	339
query49	678	546	412	412
query50	648	715	410	410
query51	4194	4244	4200	4200
query52	113	110	96	96
query53	235	255	182	182
query54	566	564	522	522
query55	83	85	83	83
query56	299	291	281	281
query57	1178	1193	1164	1164
query58	260	264	265	264
query59	2767	2803	2725	2725
query60	331	335	302	302
query61	133	165	132	132
query62	739	747	673	673
query63	225	191	185	185
query64	1749	1054	677	677
query65	4476	4261	4232	4232
query66	782	397	301	301
query67	16060	15653	15466	15466
query68	7566	872	504	504
query69	537	297	257	257
query70	1157	1061	1052	1052
query71	489	317	279	279
query72	5962	4784	5030	4784
query73	1504	683	345	345
query74	9032	9071	8951	8951
query75	3833	3216	2684	2684
query76	4205	1173	738	738
query77	611	374	276	276
query78	10149	10233	9270	9270
query79	2341	812	557	557
query80	636	498	437	437
query81	492	264	216	216
query82	372	128	96	96
query83	281	244	218	218
query84	293	99	89	89
query85	786	353	306	306
query86	379	314	289	289
query87	4444	4372	4263	4263
query88	3523	2238	2213	2213
query89	399	314	280	280
query90	1771	206	212	206
query91	144	149	108	108
query92	79	59	56	56
query93	1867	946	580	580
query94	645	410	309	309
query95	372	290	287	287
query96	494	559	276	276
query97	3096	3209	3146	3146
query98	233	207	203	203
query99	1344	1370	1283	1283
Total cold run time: 298127 ms
Total hot run time: 193012 ms

@doris-robot
Copy link

doris-robot commented Apr 17, 2025

ClickBench: Total hot run time: 29.61 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e5624acea876a32787cd79a52b1a775949d09b64, data reload: false

query1	0.04	0.04	0.02
query2	0.12	0.10	0.11
query3	0.25	0.20	0.19
query4	1.60	0.19	0.19
query5	0.62	0.57	0.58
query6	1.18	0.72	0.72
query7	0.02	0.02	0.02
query8	0.04	0.04	0.03
query9	0.58	0.53	0.51
query10	0.57	0.57	0.58
query11	0.15	0.11	0.11
query12	0.14	0.12	0.11
query13	0.61	0.60	0.60
query14	1.20	1.17	1.20
query15	0.88	0.85	0.86
query16	0.40	0.38	0.37
query17	1.03	1.04	1.04
query18	0.22	0.20	0.20
query19	1.93	1.80	1.77
query20	0.01	0.01	0.01
query21	15.40	0.93	0.56
query22	0.74	1.26	0.76
query23	14.75	1.39	0.66
query24	6.78	2.33	0.62
query25	0.54	0.18	0.06
query26	0.65	0.16	0.13
query27	0.06	0.05	0.05
query28	9.95	0.87	0.43
query29	12.56	4.06	3.34
query30	0.24	0.09	0.06
query31	2.84	0.58	0.39
query32	3.22	0.54	0.47
query33	3.06	3.04	3.07
query34	15.61	5.11	4.52
query35	4.56	4.54	4.55
query36	0.66	0.50	0.49
query37	0.08	0.06	0.07
query38	0.05	0.04	0.04
query39	0.03	0.02	0.03
query40	0.17	0.14	0.14
query41	0.08	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.69 s
Total hot run time: 29.61 s

@924060929
Copy link
Contributor Author

924060929 commented Apr 18, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-H: Total hot run time: 34047 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 20724c0016e331ddf88a40cfd6d5e8d476c1eba4, data reload: false

------ Round 1 ----------------------------------
q1	26217	5095	5057	5057
q2	2073	293	186	186
q3	10378	1269	708	708
q4	10216	1009	529	529
q5	7517	2390	2374	2374
q6	182	163	131	131
q7	915	745	637	637
q8	9304	1285	1088	1088
q9	6723	5135	5154	5135
q10	6839	2321	1897	1897
q11	478	284	264	264
q12	351	352	225	225
q13	17805	3673	3064	3064
q14	234	230	205	205
q15	533	491	494	491
q16	444	446	402	402
q17	595	878	372	372
q18	7584	7132	7220	7132
q19	1714	967	527	527
q20	326	330	216	216
q21	3953	2623	2435	2435
q22	1078	1041	972	972
Total cold run time: 115459 ms
Total hot run time: 34047 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5189	5091	5091	5091
q2	242	343	226	226
q3	2141	2635	2275	2275
q4	1405	1816	1365	1365
q5	4496	4459	4400	4400
q6	219	168	128	128
q7	2056	1896	1778	1778
q8	2593	2484	2491	2484
q9	7340	7227	6935	6935
q10	3004	3229	2744	2744
q11	587	508	484	484
q12	691	767	622	622
q13	3547	3867	3269	3269
q14	292	286	286	286
q15	537	493	516	493
q16	481	526	457	457
q17	1152	1568	1414	1414
q18	7877	7589	7491	7491
q19	807	801	917	801
q20	1972	2052	1855	1855
q21	5255	4922	4912	4912
q22	1095	1066	1051	1051
Total cold run time: 52978 ms
Total hot run time: 50561 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

TPC-DS: Total hot run time: 192994 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 20724c0016e331ddf88a40cfd6d5e8d476c1eba4, data reload: false

query1	1391	1104	1108	1104
query2	6249	1828	1777	1777
query3	11041	4474	4733	4474
query4	54709	25216	23530	23530
query5	4960	506	468	468
query6	321	197	196	196
query7	4902	515	287	287
query8	298	272	243	243
query9	5464	2556	2541	2541
query10	404	345	265	265
query11	14981	15188	14714	14714
query12	155	112	105	105
query13	1032	504	389	389
query14	10152	6256	6250	6250
query15	198	199	181	181
query16	7182	624	490	490
query17	1075	724	572	572
query18	1692	411	299	299
query19	187	187	161	161
query20	130	129	130	129
query21	210	128	106	106
query22	4397	4543	4259	4259
query23	34241	33594	33470	33470
query24	6512	2446	2451	2446
query25	477	497	420	420
query26	701	272	157	157
query27	2254	495	335	335
query28	3233	2142	2119	2119
query29	616	605	464	464
query30	274	224	193	193
query31	864	862	789	789
query32	78	66	66	66
query33	466	380	373	373
query34	761	868	535	535
query35	825	851	765	765
query36	962	1009	934	934
query37	118	108	76	76
query38	4251	4323	4184	4184
query39	1481	1467	1448	1448
query40	218	123	104	104
query41	53	64	64	64
query42	129	119	105	105
query43	515	515	465	465
query44	1315	844	821	821
query45	182	177	168	168
query46	894	1032	665	665
query47	1847	1877	1772	1772
query48	398	426	302	302
query49	683	503	454	454
query50	672	748	424	424
query51	4230	4260	4259	4259
query52	108	110	99	99
query53	237	261	189	189
query54	587	598	531	531
query55	88	90	81	81
query56	323	308	292	292
query57	1190	1190	1144	1144
query58	281	274	261	261
query59	2628	2740	2616	2616
query60	336	333	318	318
query61	129	135	157	135
query62	769	759	665	665
query63	227	201	193	193
query64	1920	1143	820	820
query65	4425	4253	4273	4253
query66	749	397	301	301
query67	15787	15514	15194	15194
query68	8273	895	516	516
query69	577	307	259	259
query70	1190	1084	1134	1084
query71	498	322	299	299
query72	5532	4866	5068	4866
query73	1491	706	347	347
query74	8938	9124	9027	9027
query75	3943	3202	2642	2642
query76	4259	1197	742	742
query77	613	366	289	289
query78	9959	10024	9297	9297
query79	6383	806	545	545
query80	686	502	433	433
query81	487	261	218	218
query82	733	125	96	96
query83	347	251	247	247
query84	295	105	83	83
query85	807	355	382	355
query86	357	315	303	303
query87	4501	4415	4375	4375
query88	3427	2242	2233	2233
query89	453	313	290	290
query90	1949	209	216	209
query91	132	142	109	109
query92	73	59	55	55
query93	3257	933	577	577
query94	689	417	315	315
query95	369	287	291	287
query96	472	566	270	270
query97	3197	3238	3137	3137
query98	218	205	198	198
query99	1423	1418	1282	1282
Total cold run time: 305692 ms
Total hot run time: 192994 ms

@doris-robot
Copy link

doris-robot commented Apr 18, 2025

ClickBench: Total hot run time: 29.57 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 20724c0016e331ddf88a40cfd6d5e8d476c1eba4, data reload: false

query1	0.04	0.03	0.03
query2	0.13	0.11	0.11
query3	0.26	0.19	0.20
query4	1.60	0.19	0.20
query5	0.58	0.59	0.59
query6	1.19	0.71	0.70
query7	0.03	0.02	0.02
query8	0.04	0.03	0.04
query9	0.58	0.52	0.51
query10	0.58	0.56	0.56
query11	0.16	0.11	0.11
query12	0.16	0.11	0.11
query13	0.61	0.60	0.60
query14	1.16	1.20	1.17
query15	0.89	0.86	0.86
query16	0.39	0.38	0.38
query17	1.01	1.04	1.02
query18	0.22	0.19	0.19
query19	1.90	1.77	1.84
query20	0.02	0.01	0.02
query21	15.41	0.92	0.56
query22	0.75	1.19	0.67
query23	14.94	1.40	0.63
query24	7.55	1.35	0.80
query25	0.50	0.15	0.11
query26	0.60	0.17	0.15
query27	0.05	0.05	0.04
query28	9.72	0.83	0.42
query29	12.54	3.96	3.34
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.23	0.54	0.46
query33	3.07	3.07	3.05
query34	15.84	5.07	4.47
query35	4.51	4.55	4.47
query36	0.67	0.50	0.47
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.04	0.02	0.03
query40	0.16	0.15	0.13
query41	0.09	0.04	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.04
Total cold run time: 104.51 s
Total hot run time: 29.57 s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Apr 21, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Apr 21, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Apr 21, 2025

PR approved by anyone and no changes requested.

@924060929 924060929 merged commit c679c83 into apache:master Apr 21, 2025
25 of 26 checks passed
@924060929 924060929 deleted the fix_fold_nvl_datetime_type branch April 21, 2025 07:47
924060929 added a commit to 924060929/incubator-doris that referenced this pull request May 8, 2025
@924060929
[fix](nereids) fix fold constant return wrong scale of datetime type (a…
dbe9250 
…pache#50142)

fix fold constant return wrong scale of datetime type

for example, this sql should return data type `datatime(6)`, but the
fold constant rule will direct return the first argument, so return the
wrong data type `datetime(0)`
```sql
select nvl(cast('2025-04-17 01:02:03' as datetime(0)), cast('2025-04-17 01:02:03.123456' as datetime(6)))
```

(cherry picked from commit c679c83)
@924060929 924060929 mentioned this pull request May 8, 2025
Merged
924060929 added a commit to 924060929/incubator-doris that referenced this pull request May 8, 2025
@924060929
[fix](nereids) fix fold constant return wrong scale of datetime type (a…
4b24095 
…pache#50142)

fix fold constant return wrong scale of datetime type

for example, this sql should return data type `datatime(6)`, but the
fold constant rule will direct return the first argument, so return the
wrong data type `datetime(0)`
```sql
select nvl(cast('2025-04-17 01:02:03' as datetime(0)), cast('2025-04-17 01:02:03.123456' as datetime(6)))
```

(cherry picked from commit c679c83)
@924060929 924060929 mentioned this pull request May 8, 2025
Merged
yiguolei pushed a commit that referenced this pull request May 9, 2025
@924060929
[fix](nereids) fix fold constant return wrong scale of datetime type (#…
37f3c8f 
…50142) (#50716)

cherry pick from #50142
@yiguolei yiguolei mentioned this pull request May 13, 2025
Closed
dataroaring pushed a commit that referenced this pull request May 13, 2025
@924060929
[fix](nereids) fix fold constant return wrong scale of datetime type (#…
bec367b 
…50142) (#50717)

cherry pick from #50142
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@924060929 @koarz
[fix](nereids) fix fold constant return wrong scale of datetime type (a…
ad105a2 
…pache#50142)

fix fold constant return wrong scale of datetime type

for example, this sql should return data type `datatime(6)`, but the
fold constant rule will direct return the first argument, so return the
wrong data type `datetime(0)`
```sql
select nvl(cast('2025-04-17 01:02:03' as datetime(0)), cast('2025-04-17 01:02:03.123456' as datetime(6)))
```
@gavinchou gavinchou mentioned this pull request Jun 11, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@starocean999 starocean999 starocean999 approved these changes

@morrySnow morrySnow morrySnow approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.6-merged p0_b reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@924060929 @hello-stephen @doris-robot @starocean999 @morrySnow @yiguolei @dataroaring