[fix](nereids) Fix not in aggregate's output err after eliminate by uniform when group sets exist #56942
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
TPC-DS: Total hot run time: 190484 ms |
ClickBench: Total hot run time: 30.57 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
seawinde
force-pushed
the
fix_group_sets_fail_when_agg_eliminate_by_uniform
branch
from
80639d5 to
28c4a89
Compare
Contributor
Author
|
run buildall |
TPC-DS: Total hot run time: 189982 ms |
ClickBench: Total hot run time: 30.27 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
| LogicalRepeat<?> sourceRepeat = (LogicalRepeat<?>) exprIdReplacer.rewriteExpr( | ||
| aggregate.getSourceRepeat().get(), replaceMap); | ||
| aggregate = aggregate.withSourceRepeat(sourceRepeat); | ||
| } |
Contributor
There was a problem hiding this comment.
maybe this will lead to agg source repeat be rewritten twice.
Contributor
Author
There was a problem hiding this comment.
this is not same instance, so doesn't repeat be rewritten twice.
| } | ||
| List<Expression> newChildren = new ArrayList<>(); | ||
| List<Expression> oldChildren = originExpression.get().children(); | ||
| for (Expression child : oldChildren) { |
Contributor
There was a problem hiding this comment.
- directly traverse the originExpression instead of using for loop traversing originExpression children maybe better.
| newChildren.add(child.accept(new DefaultExpressionRewriter<Void>() { | ||
| @Override | ||
| public Expression visitSlotReference(SlotReference slot, Void context) { | ||
| ExprId newId = replaceMap.get(slot.getExprId()); |
Contributor
There was a problem hiding this comment.
these code is repeated, use a function to avoid repeated code may be better.
| }).toRule(ExpressionRuleType.EXPR_ID_REWRITE_REPLACE) | ||
| }).toRule(ExpressionRuleType.EXPR_ID_REWRITE_REPLACE), | ||
| matchesType(VirtualSlotReference.class).thenApply(ctx -> { | ||
| VirtualSlotReference virtualSlot = ctx.expr; |
Contributor
There was a problem hiding this comment.
1.there is an attribute "realExpressions" in VirtualSlotReference, should we replace "realExpressions" too?
Contributor
Author
There was a problem hiding this comment.
realExpressions is derived from original Expressions, we have replaced original Expressions rightly, realExpressions is right auto
…niform when group sets exist
seawinde
force-pushed
the
fix_group_sets_fail_when_agg_eliminate_by_uniform
branch
from
28c4a89 to
7e9d158
Compare
Contributor
Author
|
run buildall |
TPC-DS: Total hot run time: 190540 ms |
ClickBench: Total hot run time: 27.49 s |
Contributor
FE Regression Coverage ReportIncrement line coverage |
Contributor
Author
|
run buildall |
TPC-DS: Total hot run time: 190091 ms |
ClickBench: Total hot run time: 27.57 s |
Contributor
FE UT Coverage ReportIncrement line coverage |
feiniaofeiafei
approved these changes
Nov 5, 2025
Contributor
|
PR approved by anyone and no changes requested. |
morrySnow
approved these changes
Nov 5, 2025
github-actions
bot
added
the
approved
Contributor
|
PR approved by at least one committer and no changes requested. |
github-actions bot
pushed a commit
that referenced
this pull request
Nov 6, 2025
…niform when group sets exist (#56942) Fix not in aggregate's output err after eliminate by uniform when group sets exist if query as following, would cause `ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's output` the pr fix this ```sql SELECT CASE WHEN GROUPING(event_date) = 1 THEN '(TOTAL)' ELSE CAST(event_date AS VARCHAR) END AS event_date, user_id, MAX(conversion_level) AS conversion_level, CASE WHEN GROUPING(event_name_group) = 1 THEN '(TOTAL)' ELSE event_name_group END AS event_name_group FROM ( SELECT src.event_date, src.user_id, WINDOW_FUNNEL( 3600 * 24 * 1, 'default', src.event_time, src.event_name = 'shop_buy', src.event_name = 'shop_buy' ) AS conversion_level, src.event_name_group FROM ( SELECT CAST(etb.`@dt` AS DATE) AS event_date, etb.`@event_name` AS event_name, etb.`@event_time` AS event_time, etb.`@event_name` AS event_name_group, etb.`@user_id` AS user_id FROM `test_event` AS etb WHERE etb.`@dt` between '2025-09-03 02:00:00' AND '2025-09-10 01:59:59' AND etb.`@event_name` = 'shop_buy' AND etb.`@user_id` IS NOT NULL AND etb.`@user_id` > '0' ) AS src GROUP BY src.event_date, src.user_id, src.event_name_group ) AS fwt GROUP BY GROUPING SETS ( (user_id), (user_id, event_date), (user_id, event_name_group), (user_id, event_date, event_name_group) ); ``` ### What problem does this PR solve? Issue Number: close #xxx Related PR: #43391 Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
yiguolei
pushed a commit
that referenced
this pull request
Nov 8, 2025
…niform when group sets exist (#56942) Fix not in aggregate's output err after eliminate by uniform when group sets exist if query as following, would cause `ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's output` the pr fix this ```sql SELECT CASE WHEN GROUPING(event_date) = 1 THEN '(TOTAL)' ELSE CAST(event_date AS VARCHAR) END AS event_date, user_id, MAX(conversion_level) AS conversion_level, CASE WHEN GROUPING(event_name_group) = 1 THEN '(TOTAL)' ELSE event_name_group END AS event_name_group FROM ( SELECT src.event_date, src.user_id, WINDOW_FUNNEL( 3600 * 24 * 1, 'default', src.event_time, src.event_name = 'shop_buy', src.event_name = 'shop_buy' ) AS conversion_level, src.event_name_group FROM ( SELECT CAST(etb.`@dt` AS DATE) AS event_date, etb.`@event_name` AS event_name, etb.`@event_time` AS event_time, etb.`@event_name` AS event_name_group, etb.`@user_id` AS user_id FROM `test_event` AS etb WHERE etb.`@dt` between '2025-09-03 02:00:00' AND '2025-09-10 01:59:59' AND etb.`@event_name` = 'shop_buy' AND etb.`@user_id` IS NOT NULL AND etb.`@user_id` > '0' ) AS src GROUP BY src.event_date, src.user_id, src.event_name_group ) AS fwt GROUP BY GROUPING SETS ( (user_id), (user_id, event_date), (user_id, event_name_group), (user_id, event_date, event_name_group) ); ``` ### What problem does this PR solve? Issue Number: close #xxx Related PR: #43391 Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
seawinde
added a commit
to seawinde/doris
that referenced
this pull request
Nov 10, 2025
…niform when group sets exist (apache#56942) Fix not in aggregate's output err after eliminate by uniform when group sets exist if query as following, would cause `ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's output` the pr fix this ```sql SELECT CASE WHEN GROUPING(event_date) = 1 THEN '(TOTAL)' ELSE CAST(event_date AS VARCHAR) END AS event_date, user_id, MAX(conversion_level) AS conversion_level, CASE WHEN GROUPING(event_name_group) = 1 THEN '(TOTAL)' ELSE event_name_group END AS event_name_group FROM ( SELECT src.event_date, src.user_id, WINDOW_FUNNEL( 3600 * 24 * 1, 'default', src.event_time, src.event_name = 'shop_buy', src.event_name = 'shop_buy' ) AS conversion_level, src.event_name_group FROM ( SELECT CAST(etb.`@dt` AS DATE) AS event_date, etb.`@event_name` AS event_name, etb.`@event_time` AS event_time, etb.`@event_name` AS event_name_group, etb.`@user_id` AS user_id FROM `test_event` AS etb WHERE etb.`@dt` between '2025-09-03 02:00:00' AND '2025-09-10 01:59:59' AND etb.`@event_name` = 'shop_buy' AND etb.`@user_id` IS NOT NULL AND etb.`@user_id` > '0' ) AS src GROUP BY src.event_date, src.user_id, src.event_name_group ) AS fwt GROUP BY GROUPING SETS ( (user_id), (user_id, event_date), (user_id, event_name_group), (user_id, event_date, event_name_group) ); ``` ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#43391 Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
seawinde
added a commit
to seawinde/doris
that referenced
this pull request
Nov 11, 2025
…niform when group sets exist (apache#56942) Fix not in aggregate's output err after eliminate by uniform when group sets exist if query as following, would cause `ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's output` the pr fix this ```sql SELECT CASE WHEN GROUPING(event_date) = 1 THEN '(TOTAL)' ELSE CAST(event_date AS VARCHAR) END AS event_date, user_id, MAX(conversion_level) AS conversion_level, CASE WHEN GROUPING(event_name_group) = 1 THEN '(TOTAL)' ELSE event_name_group END AS event_name_group FROM ( SELECT src.event_date, src.user_id, WINDOW_FUNNEL( 3600 * 24 * 1, 'default', src.event_time, src.event_name = 'shop_buy', src.event_name = 'shop_buy' ) AS conversion_level, src.event_name_group FROM ( SELECT CAST(etb.`@dt` AS DATE) AS event_date, etb.`@event_name` AS event_name, etb.`@event_time` AS event_time, etb.`@event_name` AS event_name_group, etb.`@user_id` AS user_id FROM `test_event` AS etb WHERE etb.`@dt` between '2025-09-03 02:00:00' AND '2025-09-10 01:59:59' AND etb.`@event_name` = 'shop_buy' AND etb.`@user_id` IS NOT NULL AND etb.`@user_id` > '0' ) AS src GROUP BY src.event_date, src.user_id, src.event_name_group ) AS fwt GROUP BY GROUPING SETS ( (user_id), (user_id, event_date), (user_id, event_name_group), (user_id, event_date, event_name_group) ); ``` Issue Number: close #xxx Related PR: apache#43391 Problem Summary: None - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
wyxxxcat
pushed a commit
to wyxxxcat/doris
that referenced
this pull request
Nov 18, 2025
…niform when group sets exist (apache#56942) Fix not in aggregate's output err after eliminate by uniform when group sets exist if query as following, would cause `ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's output` the pr fix this ```sql SELECT CASE WHEN GROUPING(event_date) = 1 THEN '(TOTAL)' ELSE CAST(event_date AS VARCHAR) END AS event_date, user_id, MAX(conversion_level) AS conversion_level, CASE WHEN GROUPING(event_name_group) = 1 THEN '(TOTAL)' ELSE event_name_group END AS event_name_group FROM ( SELECT src.event_date, src.user_id, WINDOW_FUNNEL( 3600 * 24 * 1, 'default', src.event_time, src.event_name = 'shop_buy', src.event_name = 'shop_buy' ) AS conversion_level, src.event_name_group FROM ( SELECT CAST(etb.`@dt` AS DATE) AS event_date, etb.`@event_name` AS event_name, etb.`@event_time` AS event_time, etb.`@event_name` AS event_name_group, etb.`@user_id` AS user_id FROM `test_event` AS etb WHERE etb.`@dt` between '2025-09-03 02:00:00' AND '2025-09-10 01:59:59' AND etb.`@event_name` = 'shop_buy' AND etb.`@user_id` IS NOT NULL AND etb.`@user_id` > '0' ) AS src GROUP BY src.event_date, src.user_id, src.event_name_group ) AS fwt GROUP BY GROUPING SETS ( (user_id), (user_id, event_date), (user_id, event_name_group), (user_id, event_date, event_name_group) ); ``` ### What problem does this PR solve? Issue Number: close #xxx Related PR: apache#43391 Problem Summary: ### Release note None ### Check List (For Author) - Test <!-- At least one of them must be included. --> - [ ] Regression test - [ ] Unit Test - [ ] Manual test (add detailed scripts or steps below) - [ ] No need to test or manual test. Explain why: - [ ] This is a refactor/code format and no logic has been changed. - [ ] Previous test can cover this change. - [ ] No code files have been changed. - [ ] Other reason <!-- Add your reason? --> - Behavior changed: - [ ] No. - [ ] Yes. <!-- Explain the behavior change --> - Does this need documentation? - [ ] No. - [ ] Yes. <!-- Add document PR link here. eg: apache/doris-website#1214 --> ### Check List (For Reviewer who merge this PR) - [ ] Confirm the release note - [ ] Confirm test cases - [ ] Confirm document - [ ] Add branch pick label <!-- Add branch pick label that this PR should merge into -->
w41ter
pushed a commit
to w41ter/incubator-doris
that referenced
this pull request
Dec 26, 2025
…niform when group sets exist apache#56942 (apache#5783) pr: apache#56942 commitId: a5e4e1c
yiguolei
pushed a commit
to yiguolei/incubator-doris
that referenced
this pull request
Dec 30, 2025
…niform when group sets exist (apache#56942) (apache#5631) pr: apache#56942 commitId: apache@961cef6 ## Proposed changes Issue Number: close #xxx <!--Describe your changes.-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Fix not in aggregate's output err after eliminate by uniform when group sets exist
if query as following, would cause
ERROR 1105 (HY000): errCode = 2, detailMessage = GROUPING_PREFIX_event_name_group not in aggregate's outputthe pr fix this
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #43391
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)