[Enhancement](doris-future) Support "REGR_" aggregation functions (PART II) #41240

Yoruet · 2024-09-24T13:18:03Z

Proposed changes

Issue Number: close #38975

mysql> select * from test;
+------+------+------+
| id   | x    | y    |
+------+------+------+
|    1 |   18 |   13 |
|    3 |   12 |    2 |
|    5 |   10 |   20 |
|    2 |   14 |   27 |
|    4 |    5 |    6 |
+------+------+------+
5 rows in set (0.07 sec)

mysql> select regr_slope(y,x) , regr_intercept(y,x) from test;
+--------------------+----------------------+
| regr_slope(y, x)   | regr_intercept(y, x) |
+--------------------+----------------------+
| 0.6853448275862069 |    5.512931034482759 |
+--------------------+----------------------+
1 row in set (0.15 sec)

doris-robot · 2024-09-24T13:18:08Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.h

zhiqiang-hhhh · 2024-09-25T02:52:42Z

@Yoruet Null property of regr_slope and regr_intercept should be AlwaysNullable on FE, so need to do some modification on be.

zhiqiang-hhhh · 2024-09-25T07:50:09Z

@Yoruet You can take this pr #40945 as a reference.

...re/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/RegrIntercept.java

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.cpp

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.h

zhiqiang-hhhh · 2024-09-25T08:01:49Z

@Yoruet I add more comments with detail information.

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.cpp

...re/src/main/java/org/apache/doris/nereids/trees/expressions/functions/agg/RegrIntercept.java

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.h

…et/41240

zhiqiang-hhhh · 2024-09-27T13:34:57Z

Yoruet#1

@Yoruet I did some fix on regr_intercept on your branch, you can merge the pull request and do modification on other function. Test case should be taked.

Reference https://dbfiddle.uk/MmKrIU9r

zhiqiang-hhhh

Null value is not processed correctly.

Yoruet · 2024-09-27T13:47:22Z

Null value is not processed correctly.

Can you provide corresponding test samples to me

zhiqiang-hhhh · 2024-09-27T14:16:57Z

Null value is not processed correctly.

Can you provide corresponding test samples to me

@Yoruet You can try test case added in this pr Yoruet#1 on regr_slope

Yoruet · 2024-09-27T14:56:35Z

Null value is not processed correctly.

Can you provide corresponding test samples to me

@Yoruet You can try test case added in this pr Yoruet#1 on regr_slope

@zhiqiang-hhhh I compared the data in the test_regr_intercept.out file with that in https://dbfiddle.uk/MmKrIU9r, and it seems that there is nothing wrong. Can you provide more detailed information?

zhiqiang-hhhh · 2024-09-27T15:46:40Z

Null value is not processed correctly.

Can you provide corresponding test samples to me

@Yoruet You can try test case added in this pr Yoruet#1 on regr_slope

Null value is not processed correctly.

Can you provide corresponding test samples to me

@Yoruet You can try test case added in this pr Yoruet#1 on regr_slope

@zhiqiang-hhhh I compared the data in the test_regr_intercept.out file with that in https://dbfiddle.uk/MmKrIU9r, and it seems that there is nothing wrong. Can you provide more detailed information?

The reason test_regr_intercept is correct is that I re-write the implementation of regr_itercept in that PR (I created that pull request on your branch to give you an example on how to fix regr_itercept).

You can compare the code I submitted with your implementation of regr_itercept, and do fix on regr_slope in a same way.

github-actions · 2024-09-28T05:19:25Z

PR approved by anyone and no changes requested.

zhiqiang-hhhh · 2024-09-29T02:47:20Z

We need todo a refactor on all REGR_ like agg functions, to make code simple. These agg functions have same structure.
eg. #39187

Yoruet · 2024-09-29T03:36:35Z

We need todo a refactor on all REGR_ like agg functions, to make code simple. These agg functions have same structure. eg. #39187

Does it include only regr_slope and regr_intercept or does it include other regr_ functions such as regr_sxx

zhiqiang-hhhh · 2024-09-29T04:04:30Z

We need todo a refactor on all REGR_ like agg functions, to make code simple. These agg functions have same structure. eg. #39187

Does it include only regr_slope and regr_intercept or does it include other regr_ functions such as regr_sxx

This is just a comment to remind us to do a refactor in the future, does not means we need to do refactor in this pr.

HappenLee · 2024-09-29T07:05:55Z

@Yoruet please add doc in https://github.com/apache/doris-website thank you very mush

Yoruet · 2024-09-29T15:42:34Z

@Yoruet please add doc in https://github.com/apache/doris-website thank you very mush

I have already pr the doc.plz check it

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h

be/src/vec/aggregate_functions/aggregate_function_regr_union.h

HappenLee

LGTM

github-actions · 2024-10-03T01:41:52Z

PR approved by at least one committer and no changes requested.

HappenLee · 2024-10-03T03:47:12Z

run buildall

doris-robot · 2024-10-03T04:24:06Z

TeamCity be ut coverage result:
Function Coverage: 37.30% (9632/25825)
Line Coverage: 28.68% (79854/278453)
Region Coverage: 28.11% (41291/146877)
Branch Coverage: 24.74% (21039/85034)
Coverage Report: http://coverage.selectdb-in.cc/coverage/68d8d740c68642db67051a62d6b3b8c18319ac4e_68d8d740c68642db67051a62d6b3b8c18319ac4e/report/index.html

…RT II) (apache#41240) ## Proposed changes Issue Number: close apache#38975  ```sql mysql> select * from test; +------+------+------+ | id | x | y | +------+------+------+ | 1 | 18 | 13 | | 3 | 12 | 2 | | 5 | 10 | 20 | | 2 | 14 | 27 | | 4 | 5 | 6 | +------+------+------+ 5 rows in set (0.07 sec) mysql> select regr_slope(y,x) , regr_intercept(y,x) from test; +--------------------+----------------------+ | regr_slope(y, x) | regr_intercept(y, x) | +--------------------+----------------------+ | 0.6853448275862069 | 5.512931034482759 | +--------------------+----------------------+ 1 row in set (0.15 sec) ``` --------- Co-authored-by: zhiqiang-hhhh <seuhezhiqiang@163.com>

add regr_intercept and regr_slope aggregate functions

96d4a20

Yoruet changed the title ~~add regr_intercept and regr_slope aggregate functions~~ [Enhancement](doris-future) Support "REGR_" aggregation functions (PART II) Sep 24, 2024

Yoruet added 2 commits September 24, 2024 21:30

delete unused files

5882639

fix bug in regr_slope and regr_intercept

7d3d24e

zhiqiang-hhhh suggested changes Sep 25, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h Outdated Show resolved Hide resolved

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h Outdated Show resolved Hide resolved

zhiqiang-hhhh suggested changes Sep 25, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_intercept.h Outdated Show resolved Hide resolved

zhiqiang-hhhh reviewed Sep 25, 2024

View reviewed changes

Yoruet added 3 commits September 25, 2024 18:43

update RegrSlope and RegrIntercept to be nullable

5bc299e

change the implementation of RegrSlope and RegrIntercept

19829a7

update the be of regr_slope and regr_intercept

875fb28

Yoruet requested a review from zhiqiang-hhhh September 26, 2024 15:06

zhiqiang-hhhh reviewed Sep 27, 2024

View reviewed changes

Modification details

4207554

Yoruet requested a review from zhiqiang-hhhh September 27, 2024 09:51

Yoruet and others added 3 commits September 27, 2024 19:15

fix bug in be of regr_slope and regr_intercept

2a35625

X

64711ad

Merge branch 'master' of https://github.com/yoruet/doris into pr/Yoru…

86ed7a3

…et/41240

zhiqiang-hhhh suggested changes Sep 27, 2024

View reviewed changes

update doris-cluster.conf

567f5e3

Yoruet added 2 commits September 27, 2024 23:05

Merge branch 'zhiqiang-hhhh-regr_intercept' and change other files

2ca9f40

Code Formatter / Clang Formatter

40b3f80

Yoruet mentioned this pull request Sep 29, 2024

[docs](function) support aggregrate function regr_slope regr_intercept apache/doris-website#1167

Closed

6 tasks

HappenLee reviewed Sep 30, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h Show resolved Hide resolved

HappenLee reviewed Sep 30, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_slope.h Outdated Show resolved Hide resolved

union regr_slope and regr_intercept to regr_union

29f6e52

HappenLee reviewed Sep 30, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_union.h Outdated Show resolved Hide resolved

Yoruet added 2 commits September 30, 2024 17:54

using sub class in regr_slope and regr_intercept

43c58df

code format

9b85156

Yoruet requested a review from HappenLee September 30, 2024 12:54

HappenLee reviewed Oct 2, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_union.h Outdated Show resolved Hide resolved

HappenLee reviewed Oct 2, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_union.h Outdated Show resolved Hide resolved

HappenLee reviewed Oct 2, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_union.h Outdated Show resolved Hide resolved

HappenLee reviewed Oct 2, 2024

View reviewed changes

be/src/vec/aggregate_functions/aggregate_function_regr_union.h Show resolved Hide resolved

Yoruet added 2 commits October 2, 2024 16:59

Move get_slope method to parent class

72c33e2

code format

68d8d74

Yoruet requested a review from HappenLee October 2, 2024 11:34

HappenLee approved these changes Oct 3, 2024

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Oct 3, 2024

zhangstar333 merged commit 0e5fd2f into apache:master Oct 8, 2024

[Enhancement](doris-future) Support "REGR_" aggregation functions (PART II) #41240

[Enhancement](doris-future) Support "REGR_" aggregation functions (PART II) #41240

Uh oh!

Conversation

Yoruet commented Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed changes

Uh oh!

doris-robot commented Sep 24, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhiqiang-hhhh commented Sep 25, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zhiqiang-hhhh commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhiqiang-hhhh left a comment

Choose a reason for hiding this comment

Uh oh!

Yoruet commented Sep 27, 2024

Uh oh!

zhiqiang-hhhh commented Sep 27, 2024

Uh oh!

Yoruet commented Sep 27, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhiqiang-hhhh commented Sep 27, 2024

Uh oh!

github-actions bot commented Sep 28, 2024

Uh oh!

zhiqiang-hhhh commented Sep 29, 2024

Uh oh!

Yoruet commented Sep 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zhiqiang-hhhh commented Sep 29, 2024

Uh oh!

HappenLee commented Sep 29, 2024

Uh oh!

Yoruet commented Sep 29, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

HappenLee left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Oct 3, 2024

Uh oh!

HappenLee commented Oct 3, 2024

Uh oh!

doris-robot commented Oct 3, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Yoruet commented Sep 24, 2024 •

edited

Loading

zhiqiang-hhhh commented Sep 27, 2024 •

edited

Loading

Yoruet commented Sep 27, 2024 •

edited

Loading

Yoruet commented Sep 29, 2024 •

edited

Loading