remove extractionFn from equality, null, and range filters#14612
Merged
gianm merged 5 commits intoapache:masterfrom Jul 19, 2023
Merged
remove extractionFn from equality, null, and range filters#14612gianm merged 5 commits intoapache:masterfrom
gianm merged 5 commits intoapache:masterfrom
Conversation
changes: * EqualityFilter, NullFilter, and RangeFilter no longer support extractionFn * SQL planner will use ExpressionFilter in the small number of cases where an extractionFn would have been used if sqlUseBoundsAndSelectors is set to false instead of equality/null/range filters * fix bugs and add tests with serde, equals, and cache key for null, equality, and range filters
clintropolis
force-pushed
the
remove-extractionFn-for-new-filters
branch
from
313e81f to
95fab41
Compare
gianm
approved these changes
Jul 19, 2023
Contributor
gianm
left a comment
gianm
left a comment
There was a problem hiding this comment.
LGTM. Glad to see things moving away from extractionFn and towards expressions.
gianm
added a commit
to gianm/druid
that referenced
this pull request
Jul 19, 2023
1) Consolidate duplicate code related to Expressions#buildTimeFloorFilter. 2) Cleaner logic in Expressions#toSimpleLeafFilter: choose bounds vs range filter based solely on plannerContext.isUseBoundsAndSelectors, not also considering rhs kind. Use parsed rhs in both paths (except for numerics in the bound path). 3) Fix ArrayContains, ArrayOverlap to avoid equality filters when there is an extractionFn present. Fixes a bug introduced in apache#14612.
clintropolis
added a commit
to clintropolis/druid
that referenced
this pull request
Jul 19, 2023
) * remove extractionFn from equality, null, and range filters changes: * EqualityFilter, NullFilter, and RangeFilter no longer support extractionFn * SQL planner will use ExpressionFilter in the small number of cases where an extractionFn would have been used if sqlUseBoundsAndSelectors is set to false instead of equality/null/range filters * fix bugs and add tests with serde, equals, and cache key for null, equality, and range filters * test coverage fixes bugs * adjust * adjust again * so persnickety
clintropolis
added a commit
that referenced
this pull request
Jul 19, 2023
…14621) * remove extractionFn from equality, null, and range filters changes: * EqualityFilter, NullFilter, and RangeFilter no longer support extractionFn * SQL planner will use ExpressionFilter in the small number of cases where an extractionFn would have been used if sqlUseBoundsAndSelectors is set to false instead of equality/null/range filters * fix bugs and add tests with serde, equals, and cache key for null, equality, and range filters * test coverage fixes bugs * adjust * adjust again * so persnickety
gianm
added a commit
that referenced
this pull request
Jul 20, 2023
…14619) * Simplify bounds/range vs selectors/equality logic in SQL planning. 1) Consolidate duplicate code related to Expressions#buildTimeFloorFilter. 2) Cleaner logic in Expressions#toSimpleLeafFilter: choose bounds vs range filter based solely on plannerContext.isUseBoundsAndSelectors, not also considering rhs kind. Use parsed rhs in both paths (except for numerics in the bound path). 3) Fix ArrayContains, ArrayOverlap to avoid equality filters when there is an extractionFn present. Fixes a bug introduced in #14612. * Avoid sending nonprimitives down the bound path.
gianm
added a commit
to gianm/druid
that referenced
this pull request
Jul 20, 2023
…pache#14619) * Simplify bounds/range vs selectors/equality logic in SQL planning. 1) Consolidate duplicate code related to Expressions#buildTimeFloorFilter. 2) Cleaner logic in Expressions#toSimpleLeafFilter: choose bounds vs range filter based solely on plannerContext.isUseBoundsAndSelectors, not also considering rhs kind. Use parsed rhs in both paths (except for numerics in the bound path). 3) Fix ArrayContains, ArrayOverlap to avoid equality filters when there is an extractionFn present. Fixes a bug introduced in apache#14612. * Avoid sending nonprimitives down the bound path.
abhishekagarwal87
pushed a commit
that referenced
this pull request
Jul 21, 2023
…14619) (#14624) * Simplify bounds/range vs selectors/equality logic in SQL planning. 1) Consolidate duplicate code related to Expressions#buildTimeFloorFilter. 2) Cleaner logic in Expressions#toSimpleLeafFilter: choose bounds vs range filter based solely on plannerContext.isUseBoundsAndSelectors, not also considering rhs kind. Use parsed rhs in both paths (except for numerics in the bound path). 3) Fix ArrayContains, ArrayOverlap to avoid equality filters when there is an extractionFn present. Fixes a bug introduced in #14612. * Avoid sending nonprimitives down the bound path.
sergioferragut
pushed a commit
to sergioferragut/druid
that referenced
this pull request
Jul 21, 2023
) * remove extractionFn from equality, null, and range filters changes: * EqualityFilter, NullFilter, and RangeFilter no longer support extractionFn * SQL planner will use ExpressionFilter in the small number of cases where an extractionFn would have been used if sqlUseBoundsAndSelectors is set to false instead of equality/null/range filters * fix bugs and add tests with serde, equals, and cache key for null, equality, and range filters * test coverage fixes bugs * adjust * adjust again * so persnickety
sergioferragut
pushed a commit
to sergioferragut/druid
that referenced
this pull request
Jul 21, 2023
…pache#14619) * Simplify bounds/range vs selectors/equality logic in SQL planning. 1) Consolidate duplicate code related to Expressions#buildTimeFloorFilter. 2) Cleaner logic in Expressions#toSimpleLeafFilter: choose bounds vs range filter based solely on plannerContext.isUseBoundsAndSelectors, not also considering rhs kind. Use parsed rhs in both paths (except for numerics in the bound path). 3) Fix ArrayContains, ArrayOverlap to avoid equality filters when there is an extractionFn present. Fixes a bug introduced in apache#14612. * Avoid sending nonprimitives down the bound path.
teyeheimans
pushed a commit
to level23/druid-client
that referenced
this pull request
Mar 19, 2024
- Removed GroupByV1. GroupBy v1 is a legacy engine and has not been supported since 2021. - **Removed extraction functions.** Extraction functions are deprecated in druid and are NOT build in the new NullFilter, EqualityFilter and RangeFilter. These are deprecated and should be replaced by expressions. This is poorly documented, but can be found here: apache/druid#14612 Therefore, we have decided to remove the extraction functionality completely. You can replace the functionality of extraction functions with expressions (and virtual columns). See below for some examples. - Added `RangeFilter` which replaces the `BoundFilter`. See NULL usage sector below. - Added `EqualityFilter` which replaces the `SelectorFilter`. See NULL usage sector below. - `whereBetween` now uses the `range` filter. The `whereBetween` doesn't accept an SortingOrder anymore, but a DataType. - The `whereFlags()` method does not accept `$useJavascript` anymore, as it used an expression filter. - Removed `orWhereNotColumn`. This was a left-over and should have been removed in v3. - Added support for authentication in the druid client.
Merged
teyeheimans
added a commit
to level23/druid-client
that referenced
this pull request
Mar 19, 2024
* v4.0.0 - Removed GroupByV1. GroupBy v1 is a legacy engine and has not been supported since 2021. - **Removed extraction functions.** Extraction functions are deprecated in druid and are NOT build in the new NullFilter, EqualityFilter and RangeFilter. These are deprecated and should be replaced by expressions. This is poorly documented, but can be found here: apache/druid#14612 Therefore, we have decided to remove the extraction functionality completely. You can replace the functionality of extraction functions with expressions (and virtual columns). See below for some examples. - Added `RangeFilter` which replaces the `BoundFilter`. See NULL usage sector below. - Added `EqualityFilter` which replaces the `SelectorFilter`. See NULL usage sector below. - `whereBetween` now uses the `range` filter. The `whereBetween` doesn't accept an SortingOrder anymore, but a DataType. - The `whereFlags()` method does not accept `$useJavascript` anymore, as it used an expression filter. - Removed `orWhereNotColumn`. This was a left-over and should have been removed in v3. - Added support for authentication in the druid client. * Added whereArrayContains
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Follow up to #14542 with a few modifications and fixes
changes:
This PR has: