Skip to content

[improve](function) support lead/lag function input column as third params #49381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
HappenLee merged 3 commits into from
Mar 25, 2025
Merged

[improve](function) support lead/lag function input column as third params #49381

HappenLee merged 3 commits into from
Mar 25, 2025

Conversation

@zhangstar333
Copy link
Contributor

@zhangstar333 zhangstar333 commented Mar 24, 2025

What problem does this PR solve?

Problem Summary:

support lead/lag function could input column
eg: lead(col,1,col)/lag(col,1,col)

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 24, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Mar 24, 2025

run buildall

@zhangstar333
Copy link
Contributor Author

zhangstar333 commented Mar 24, 2025

run buildall

@doris-robot
Copy link

doris-robot commented Mar 24, 2025

TPC-H: Total hot run time: 34287 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit bfbf7ebcc9dfcdfe08b361e96428e16867def5db, data reload: false

------ Round 1 ----------------------------------
q1	24626	5052	5063	5052
q2	2040	315	184	184
q3	10375	1259	725	725
q4	10226	993	548	548
q5	7541	2369	2370	2369
q6	182	161	133	133
q7	925	744	621	621
q8	9329	1308	1095	1095
q9	6808	5114	5087	5087
q10	6793	2308	1916	1916
q11	476	268	254	254
q12	357	356	217	217
q13	17750	3669	3059	3059
q14	218	223	226	223
q15	535	473	473	473
q16	629	611	577	577
q17	581	867	355	355
q18	7824	7212	7240	7212
q19	1173	945	548	548
q20	319	327	192	192
q21	4033	2733	2481	2481
q22	1051	1016	966	966
Total cold run time: 113791 ms
Total hot run time: 34287 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5155	5067	5073	5067
q2	233	326	227	227
q3	2144	2672	2304	2304
q4	1374	1815	1389	1389
q5	4426	4573	4444	4444
q6	221	172	126	126
q7	1970	1922	1710	1710
q8	2586	2530	2543	2530
q9	7238	7063	7203	7063
q10	2998	3144	2745	2745
q11	586	491	488	488
q12	665	766	634	634
q13	3383	3880	3202	3202
q14	286	299	296	296
q15	517	478	486	478
q16	659	667	666	666
q17	1131	1577	1326	1326
q18	7883	7645	7486	7486
q19	814	836	818	818
q20	2024	1974	1841	1841
q21	5343	4647	4681	4647
q22	1063	1026	968	968
Total cold run time: 52699 ms
Total hot run time: 50455 ms

@doris-robot
Copy link

doris-robot commented Mar 24, 2025

TPC-DS: Total hot run time: 186687 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit bfbf7ebcc9dfcdfe08b361e96428e16867def5db, data reload: false

query1	1002	490	460	460
query2	6530	1947	1876	1876
query3	6813	217	217	217
query4	26572	23321	23104	23104
query5	4361	641	468	468
query6	296	200	185	185
query7	4601	483	272	272
query8	296	240	248	240
query9	8607	2607	2555	2555
query10	479	306	256	256
query11	15691	15139	14853	14853
query12	173	113	108	108
query13	1656	513	387	387
query14	11459	6895	6419	6419
query15	230	196	167	167
query16	7759	639	485	485
query17	1621	731	554	554
query18	2031	406	306	306
query19	248	183	157	157
query20	121	120	118	118
query21	213	120	106	106
query22	4367	4322	4006	4006
query23	34137	33054	33075	33054
query24	7271	2348	2350	2348
query25	507	454	390	390
query26	1215	267	140	140
query27	2053	492	321	321
query28	3946	2450	2403	2403
query29	686	558	451	451
query30	278	223	189	189
query31	946	876	751	751
query32	76	65	66	65
query33	564	360	310	310
query34	775	854	499	499
query35	788	815	738	738
query36	942	990	895	895
query37	118	100	78	78
query38	4112	4155	4166	4155
query39	1463	1381	1389	1381
query40	209	137	104	104
query41	53	52	54	52
query42	118	105	98	98
query43	481	497	458	458
query44	1286	792	790	790
query45	174	170	168	168
query46	844	1017	624	624
query47	1743	1791	1720	1720
query48	371	419	296	296
query49	772	523	421	421
query50	692	735	393	393
query51	4172	4242	4209	4209
query52	112	98	94	94
query53	221	250	177	177
query54	510	486	416	416
query55	80	83	83	83
query56	279	272	247	247
query57	1148	1174	1075	1075
query58	243	242	242	242
query59	2632	2716	2449	2449
query60	292	287	256	256
query61	130	133	131	131
query62	798	717	690	690
query63	222	183	179	179
query64	4232	1050	672	672
query65	4431	4342	4351	4342
query66	1064	414	362	362
query67	15773	15338	15427	15338
query68	8304	873	502	502
query69	468	306	265	265
query70	1230	1099	1108	1099
query71	456	291	274	274
query72	5831	5077	5212	5077
query73	723	627	403	403
query74	8985	8887	8958	8887
query75	3923	3220	2720	2720
query76	3765	1199	750	750
query77	795	374	280	280
query78	10075	10405	9302	9302
query79	3032	810	567	567
query80	588	510	444	444
query81	492	255	220	220
query82	705	127	96	96
query83	185	171	154	154
query84	245	96	76	76
query85	794	353	314	314
query86	386	321	276	276
query87	4500	4424	4353	4353
query88	3665	2254	2243	2243
query89	396	304	275	275
query90	1876	205	206	205
query91	142	139	109	109
query92	75	61	55	55
query93	1754	1045	583	583
query94	653	413	294	294
query95	353	276	257	257
query96	482	567	274	274
query97	3294	3391	3286	3286
query98	234	218	204	204
query99	1331	1422	1290	1290
Total cold run time: 277822 ms
Total hot run time: 186687 ms

@doris-robot
Copy link

doris-robot commented Mar 24, 2025

ClickBench: Total hot run time: 30.84 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit bfbf7ebcc9dfcdfe08b361e96428e16867def5db, data reload: false

query1	0.04	0.04	0.03
query2	0.12	0.11	0.10
query3	0.26	0.19	0.20
query4	1.59	0.19	0.19
query5	0.59	0.59	0.60
query6	1.18	0.71	0.72
query7	0.02	0.02	0.02
query8	0.05	0.03	0.03
query9	0.58	0.54	0.51
query10	0.59	0.59	0.57
query11	0.16	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.61
query14	2.81	2.71	2.73
query15	0.93	0.86	0.84
query16	0.38	0.37	0.38
query17	1.04	1.00	1.04
query18	0.22	0.20	0.20
query19	1.92	1.98	1.80
query20	0.02	0.01	0.01
query21	15.35	0.91	0.54
query22	0.76	1.25	0.72
query23	14.80	1.36	0.65
query24	6.46	2.63	0.42
query25	0.37	0.26	0.14
query26	0.54	0.17	0.14
query27	0.06	0.05	0.05
query28	9.49	0.91	0.42
query29	12.55	3.99	3.31
query30	0.25	0.09	0.07
query31	2.82	0.58	0.38
query32	3.23	0.54	0.47
query33	3.15	3.00	3.03
query34	15.84	5.11	4.54
query35	4.53	4.50	4.51
query36	0.68	0.50	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.18	0.14	0.12
query41	0.07	0.03	0.03
query42	0.04	0.02	0.03
query43	0.04	0.03	0.03
Total cold run time: 104.66 s
Total hot run time: 30.84 s

@hello-stephen
Copy link
Contributor

hello-stephen commented Mar 24, 2025

BE UT Coverage Report

Increment line coverage 0.00% (0/24) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 49.97% (13372/26759)
Line Coverage 39.43% (115707/293414)
Region Coverage 38.14% (58729/154001)
Branch Coverage 33.28% (29648/89084)

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 25, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Mar 25, 2025

PR approved by at least one committer and no changes requested.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 25, 2025

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Mar 25, 2025
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HappenLee HappenLee merged commit 6d42a93 into apache:master Mar 25, 2025
23 of 26 checks passed
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request May 7, 2025
@zhangstar333
[improve](function) support lead/lag function input column as third p…
9b2b5ff 
…arams (apache#49381)

support lead/lag function could input column
eg: lead(col,1,col)/lag(col,1,col)
@zhangstar333 zhangstar333 mentioned this pull request May 7, 2025
Merged
16 tasks
yiguolei pushed a commit that referenced this pull request May 7, 2025
@zhangstar333
[cherry-pick](branch-21) support lead/lag function input column as th…
9fe87a5 
…ird params (#49381) (#50653)

### What problem does this PR solve?

cherry-pick from master (#49381)

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
koarz pushed a commit to koarz/doris that referenced this pull request Jun 4, 2025
@zhangstar333
[improve](function) support lead/lag function input column as third p…
1efa6c9 
…arams (apache#49381)

support lead/lag function could input column
eg: lead(col,1,col)/lag(col,1,col)
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Jun 20, 2025
@zhangstar333
[improve](function) support lead/lag function input column as third p…
39d11d1 
…arams (apache#49381)

support lead/lag function could input column
eg: lead(col,1,col)/lag(col,1,col)
@zhangstar333 zhangstar333 mentioned this pull request Jun 20, 2025
Merged
16 tasks
zhangstar333 added a commit to zhangstar333/incubator-doris that referenced this pull request Jun 20, 2025
@zhangstar333
[improve](function) support lead/lag function input column as third p…
01b5906 
…arams (apache#49381)

support lead/lag function could input column
eg: lead(col,1,col)/lag(col,1,col)
@zhangstar333 zhangstar333 mentioned this pull request Jun 20, 2025
Closed
16 tasks
morrySnow pushed a commit that referenced this pull request Jun 20, 2025
@zhangstar333 @Jibing-Li
[cherry-pick](branch-31) pick 4 prs #38259 #47676 #51293 #49381 (#52005)
d538b6f 
pick from master #38259 #47676 #51293 #49381 


Co-authored-by: James <lijibing@selectdb.com>
GoGoWen pushed a commit to GoGoWen/incubator-doris that referenced this pull request Sep 26, 2025
feature(function): support lead/lag function input column as third pa…
1b3ceb0 
…rams (apache#49381)

    support lead/lag function could input column
    eg: lead(col,1,col)/lag(col,1,col)
    Conflicts:
        be/src/vec/aggregate_functions/aggregate_function_window.h
        regression-test/data/correctness_p0/test_lag_lead_window.out
        regression-test/suites/correctness_p0/test_lag_lead_window.groovy
w41ter pushed a commit to w41ter/incubator-doris that referenced this pull request Dec 26, 2025
@zhangstar333
[improve](function) support lead/lag function input column as third p…
86bf598 
…arams (apache#49381) (apache#6301)

pick from master (apache#49381)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@BiteTheDDDDt BiteTheDDDDt BiteTheDDDDt approved these changes

@HappenLee HappenLee HappenLee approved these changes

Assignees

No one assigned

Labels

approved Indicates a PR has been approved by one committer. dev/2.1.10-merged dev/3.0.x dev/3.0.x-conflict dev/3.1.0-merged reviewed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

9 participants

@zhangstar333 @Thearas @doris-robot @hello-stephen @BiteTheDDDDt @HappenLee @morningman @yiguolei @morrySnow