Skip to content

Fix the error case when there are multi top level unions#12017

Merged
abhishekagarwal87 merged 7 commits intoapache:masterfrom
LakshSingla:sql-explain-multi-union
Dec 6, 2021
Merged

Fix the error case when there are multi top level unions#12017
abhishekagarwal87 merged 7 commits intoapache:masterfrom
LakshSingla:sql-explain-multi-union

Conversation

@LakshSingla
Copy link
Copy Markdown
Contributor

@LakshSingla LakshSingla commented Dec 2, 2021

Description

This is a follow up to the PR #11908. This fixes the bug in top level union all queries when there are more than 2 SQL subqueries are present. For example, if the following query was passed:

SELECT channel FROM wikipedia
UNION ALL SELECT channel FROM wikipedia WHERE channel = '#en.wikipedia'
UNION ALL SELECT channel FROM wikipedia WHERE channel = '#de.wikipedia'

it would fail to be explained in the newer format, and would revert to the original plan.
Reason for the bug: In the original PR, I assumed that DruidUnionRel can only be the top level DruidRel node after parsing, according to the comment here, and all the three queries would be children to that top level parent node.

When there are more than 2 queries connected by a top level UNION ALL, then the DruidRel structure looks like this:

DruidUnionRel
  DruidRel (query 1)
  DruidUnionRel
    DruidRel (query 2)
    DruidRel (query 3)

This PR introduces a recursive method which flattens this structure, and produces the native queries as expected.


Key changed/added classes in this PR
  • DruidPlanner

This PR has:

  • been self-reviewed.
  • added documentation for new or modified features or behaviors.
  • added Javadocs for most classes and all non-trivial methods. Linked related entities via Javadoc links.
  • added or updated version, license, or notice information in licenses.yaml
  • added comments explaining the "why" and the intent of the code wherever would not be obvious for an unfamiliar reader.
  • added unit tests or modified existing tests to cover new code paths, ensuring the threshold for code coverage is met.
  • added integration tests.
  • been tested in a test Druid cluster.

.collect(Collectors.toList());

}
catch (UnsupportedOperationException unsupportedOperationException) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you throwing unsupportedOperationException somewhere in the code path? I am wondering why is it being caught separately.

Copy link
Copy Markdown
Contributor Author

@LakshSingla LakshSingla Dec 2, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

DruidUnionRel throws it for unsupported operations: DruidUnionRel Those operations which can be done on queries which translate directly to native queries). Here toDruidQuery can throw the exception.

Comment thread sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidPlanner.java Outdated
Comment thread sql/src/main/java/org/apache/druid/sql/calcite/planner/DruidPlanner.java Outdated
Copy link
Copy Markdown
Contributor

@abhishekagarwal87 abhishekagarwal87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@abhishekagarwal87 abhishekagarwal87 merged commit 44b2fb7 into apache:master Dec 6, 2021
@abhishekagarwal87
Copy link
Copy Markdown
Contributor

Merged since CI failure is unrelated.

@abhishekagarwal87 abhishekagarwal87 added this to the 0.23.0 milestone May 11, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants