new SCALAR_IN_ARRAY function analogous to DRUID_IN#16306
Merged
gianm merged 3 commits intoapache:masterfrom Apr 19, 2024
Merged
new SCALAR_IN_ARRAY function analogous to DRUID_IN#16306gianm merged 3 commits intoapache:masterfrom
gianm merged 3 commits intoapache:masterfrom
Conversation
kgyrtkirk
reviewed
Apr 18, 2024
| | array_to_string(arr,str) | joins all elements of arr by the delimiter specified by str | | ||
| | string_to_array(str1,str2) | splits str1 into an array on the delimiter specified by str2, which is a regular expression | | ||
| | function | description | | ||
| |------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| |
Member
There was a problem hiding this comment.
I believe .md files are processed and rendered during page generation - is it really needed to add whitespace change to to unrelated lines - to make it a nice ascii table?
even the preious version was not caring at all if its a nice ascii or not by using | --- | --- |
| } | ||
| } | ||
|
|
||
| class ArrayScalarInFunction extends ArrayScalarFunction |
Member
There was a problem hiding this comment.
I wonder if it would be possible to instead of adding a new native function - delegate all this work to the new type-aware native in filter @clintropolis have added?
EDIT: now I see that the new typed in filter may not be available in all circumstances - as its not supported in defaultValue mode
gianm
reviewed
Apr 18, 2024
| | array_slice(arr,start,end) | return the subarray of arr from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end| | ||
| | array_to_string(arr,str) | joins all elements of arr by the delimiter specified by str | | ||
| | string_to_array(str1,str2) | splits str1 into an array on the delimiter specified by str2, which is a regular expression | | ||
| | function | description | |
Contributor
There was a problem hiding this comment.
Please don't format these markdown tables like this- it makes diffs hard to read because of the whitespace-only changes. Please turn off the feature in your IDE; if you're using IntelliJ, I think the code style file in the repo should do it.
| | array_ordinal(arr,long) | returns the array element at the 1 based index supplied, or null for an out of range index | | ||
| | array_contains(arr,expr) | returns 1 if the array contains the element specified by expr, or contains all elements specified by expr if expr is an array, else 0 | | ||
| | array_overlap(arr1,arr2) | returns 1 if arr1 and arr2 have any elements in common, else 0 | | ||
| | scalar_in(expr,arr) | returns 1 if the array contains the scalar specified by expr, else 0. | | |
Contributor
There was a problem hiding this comment.
scalar_in_array may be a better name. Wondering what you / others think?
Should also add a reference to the function in the SQL docs.
| } | ||
|
|
||
| @Override | ||
| ExprEval doApply(ExprEval arrayExpr, ExprEval scalarExpr) |
Contributor
There was a problem hiding this comment.
You don't need to do this now, but as a follow up you could add an asSingleThreaded impl that creates a Set to avoid the Arrays.asList(array).contains, similar to the technique used in OverlapConstantArray.
sreemanamala
changed the title
10 tasks
gianm
approved these changes
Apr 19, 2024
gianm
added a commit
to gianm/druid
that referenced
this pull request
Apr 19, 2024
Three changes to scalar_in_array as follow-ups to apache#16306: 1) Rename the class to more closely match the function name. 2) Add a specialization for constant arrays, where we build a HashSet. 3) Use castForEqualityComparison to properly handle cross-type comparisons. Additional tests verify comparisons between LONG and DOUBLE are now handled properly.
gianm
added a commit
to gianm/druid
that referenced
this pull request
Apr 20, 2024
1) Align behavior for `null` scalars to the behavior of the native `in` and `inType` filters: return `true` if the array itself contains null, else return `null`. 2) Rename the class to more closely match the function name. 3) Add a specialization for constant arrays, where we build a `HashSet`. 4) Use `castForEqualityComparison` to properly handle cross-type comparisons. Additional tests verify comparisons between LONG and DOUBLE are now handled properly.
gianm
added a commit
that referenced
this pull request
Apr 26, 2024
* Four changes to scalar_in_array as follow-ups to #16306: 1) Align behavior for `null` scalars to the behavior of the native `in` and `inType` filters: return `true` if the array itself contains null, else return `null`. 2) Rename the class to more closely match the function name. 3) Add a specialization for constant arrays, where we build a `HashSet`. 4) Use `castForEqualityComparison` to properly handle cross-type comparisons. Additional tests verify comparisons between LONG and DOUBLE are now handled properly. * Fix spelling. * Adjustments from review.
adarshsanjeev
pushed a commit
to adarshsanjeev/druid
that referenced
this pull request
May 6, 2024
* Four changes to scalar_in_array as follow-ups to apache#16306: 1) Align behavior for `null` scalars to the behavior of the native `in` and `inType` filters: return `true` if the array itself contains null, else return `null`. 2) Rename the class to more closely match the function name. 3) Add a specialization for constant arrays, where we build a `HashSet`. 4) Use `castForEqualityComparison` to properly handle cross-type comparisons. Additional tests verify comparisons between LONG and DOUBLE are now handled properly. * Fix spelling. * Adjustments from review.
cryptoe
pushed a commit
that referenced
this pull request
May 8, 2024
…6398) * Four changes to scalar_in_array as follow-ups to #16306: 1) Align behavior for `null` scalars to the behavior of the native `in` and `inType` filters: return `true` if the array itself contains null, else return `null`. 2) Rename the class to more closely match the function name. 3) Add a specialization for constant arrays, where we build a `HashSet`. 4) Use `castForEqualityComparison` to properly handle cross-type comparisons. Additional tests verify comparisons between LONG and DOUBLE are now handled properly. * Fix spelling. * Adjustments from review. Co-authored-by: Gian Merlino <gianmerlino@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
creates a new function
SCALAR_IN_ARRAY(expr, arr)to check if the scalar exprexpris present in the array exprarror notRelease note
Key changed/added classes in this PR
ScalarInOperatorConversion.javaFunction.javaThis PR has: