Skip to content

SQL: Support GROUP BY and ORDER BY for NULL types.#16252

Merged
gianm merged 17 commits intoapache:masterfrom
gianm:sql-null-types
Feb 4, 2025
Merged

SQL: Support GROUP BY and ORDER BY for NULL types.#16252
gianm merged 17 commits intoapache:masterfrom
gianm:sql-null-types

Conversation

@gianm
Copy link
Copy Markdown
Contributor

@gianm gianm commented Apr 9, 2024

Treats SQL NULL types as strings at the native layer. This is consistent with how we treat unknown-type nulls in other contexts.

Treats SQL NULL types as strings at the native layer.
@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Apr 9, 2024

@kgyrtkirk FYI I encountered an error here in decoupled planning with one of the new tests, and disabled it: https://github.com/apache/druid/pull/16252/files#diff-8d40b0c4f1c04fc1a87f672f9fb48a91568c7bd4a4eb5ceda1c9cb008a22653fR70-R76

Comment on lines +6135 to +6136
@Test
public void testOrderByNullType()
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of overriding+disabling the testcase; it can be marked as unsupported with:

Suggested change
@Test
public void testOrderByNullType()
@NotYetSupported(Modes.NOT_ENOUGH_RULES)
@Test
public void testOrderByNullType()

this has the benefit that it will be verified that it still fails and could also be located based on the type of failure it have

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, thanks, cool idea!

It is supported for the basic planner, so I kept the override in DecoupledPlanningCalciteQueryTest and changed the annotation from @Disabled to @NotYetSupported.

if (dataType.getSqlTypeName() == SqlTypeName.NULL) {
return StringComparators.NATURAL;
} else {
final ColumnType valueType = getColumnTypeForRelDataType(dataType);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add a new columnType called null just like the relDataType : SqlTypeName.null
In that case, we can push this logic inside getStringComparatorsForValueType() and remove this custom handling ?

thoughts ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ColumnType is meant to represent types that the native query engine can store and process. We don't really have a pure-NULL type there, so adding one may have larger consequences that I did not want to run into in this patch. I thought it would be better to keep the changes to the SQL layer.

@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 60 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If you think
that's incorrect or this pull request should instead be reviewed, please simply
write any comment. Even if closed, you can still revive the PR at any time or
discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jul 22, 2024
@gianm
Copy link
Copy Markdown
Contributor Author

gianm commented Jul 24, 2024

NOT_ENOUGH_RULES was removed in #16770; I added it back so it could be used in this PR.

@github-actions github-actions Bot removed the stale label Jul 25, 2024
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 60 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If you think
that's incorrect or this pull request should instead be reviewed, please simply
write any comment. Even if closed, you can still revive the PR at any time or
discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

@github-actions github-actions Bot added the stale label Sep 23, 2024
@github-actions github-actions Bot removed the stale label Oct 3, 2024
Copy link
Copy Markdown
Member

@kgyrtkirk kgyrtkirk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess the +1 was forgotten for this one...

@kgyrtkirk
Copy link
Copy Markdown
Member

forcing PR recheck with close/open cycle

@kgyrtkirk kgyrtkirk closed this Nov 28, 2024
@kgyrtkirk kgyrtkirk reopened this Nov 28, 2024
@github-actions
Copy link
Copy Markdown

This pull request has been marked as stale due to 60 days of inactivity.
It will be closed in 4 weeks if no further activity occurs. If you think
that's incorrect or this pull request should instead be reviewed, please simply
write any comment. Even if closed, you can still revive the PR at any time or
discuss it on the dev@druid.apache.org list.
Thank you for your contributions.

@github-actions github-actions Bot added the stale label Jan 28, 2025
@gianm gianm merged commit f694255 into apache:master Feb 4, 2025
@gianm gianm deleted the sql-null-types branch February 4, 2025 17:37
@kgyrtkirk kgyrtkirk added this to the 33.0.0 milestone Apr 14, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants