-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
Is your feature request related to a problem or challenge?
As we port over functions out of the datafusion-physical-expr crate, we have discovered several parts of ExprSimplif that rely on semantics of the array functions themselves
For example, the rewrite from array1 @> array2 to array_has_all(array1, array2)in rewrite_array_has_all_operator_to_func`
([source link](https://github.com/apache/arrow-datafusion/blob/5537572820977b38719e2253f601a159deef5bc6/datafusion/optimizer/src/analyzer/rewrite_expr.rs#L137
As @jayzhan22 noted on #9496 #9496 (comment), this rewrite is not correct if users supply their own implementation of array_has_all as that implementation may not be equal to DataFusion's array_has_all function. Thus the rule only makes sense when using the datafusion implementation
Describe the solution you'd like
Move the array_function specific rules such as rewrite_array_has_all_operator_to_func to the dataufusion_array_functions
Describe alternatives you've considered
- Create a a User Defined
Analyzerpass (defined indatafusion_functions_array) with the desired rewrite - Register the
datafusion_function_arrayspecific rewrite when the array functions indatafusion_array_functionare registered (somewhere near here
Note you can register an analyzer rule via SessionState::add_analyzer_rule
Additional context
No response