fix array_agg to work with complex types and bugs with expression aggregator complex array handling#13781
Merged
clintropolis merged 8 commits intoapache:masterfrom Feb 13, 2023
Conversation
…regator complex array handling
…nsistently honor druid.generic.useDefaultValueForNull, fix array_ordinal sql output type
abhishekagarwal87
approved these changes
Feb 13, 2023
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR removes an overly restrictive validation check on
ARRAY_AGGwhich was preventing use with druidCOMPLEXtypes, such asCOMPLEX<json>. This check is a relic from when this function was added, which was before the native expression system supportedCOMPLEXtypes.After removing this check, I ran into a couple of rough edges, and a bug in
ObjectStrategyComplexTypeStrategy, which was missing a check to ensure that the positionalreadmethod actually moved the buffer position forward by the number of bytes read (useful for array_agg which is serializing arrays) instead of relying on the underlyingObjectStrategyto move the buffer.After the bugs, the result seems pretty useful, particularly for

COMPLEX<json>(though certainly not the speediest thing around since its implemented as an expression aggregator):even distinct works:
While here, I also fixed a bug with the output type inference of
ARRAY_OFFSETandARRAY_ORDINALfunctions, as well as made nativeARRAYtyped expressions more consistently honordruid.generic.useDefaultValueForNull.This PR has: