Skip to content

Move datafusion_array_function specific rewrite rules like to datafusion_functions_array crate #9519

@alamb

Description

@alamb

Is your feature request related to a problem or challenge?

As we port over functions out of the datafusion-physical-expr crate, we have discovered several parts of ExprSimplif that rely on semantics of the array functions themselves

For example, the rewrite from array1 @> array2 to array_has_all(array1, array2)in rewrite_array_has_all_operator_to_func`
([source link](https://github.com/apache/arrow-datafusion/blob/5537572820977b38719e2253f601a159deef5bc6/datafusion/optimizer/src/analyzer/rewrite_expr.rs#L137

As @jayzhan22 noted on #9496 #9496 (comment), this rewrite is not correct if users supply their own implementation of array_has_all as that implementation may not be equal to DataFusion's array_has_all function. Thus the rule only makes sense when using the datafusion implementation

Describe the solution you'd like

Move the array_function specific rules such as rewrite_array_has_all_operator_to_func to the dataufusion_array_functions

Describe alternatives you've considered

  1. Create a a User Defined Analyzer pass (defined in datafusion_functions_array) with the desired rewrite
  2. Register the datafusion_function_array specific rewrite when the array functions in datafusion_array_function are registered (somewhere near here

Note you can register an analyzer rule via SessionState::add_analyzer_rule

Additional context

No response

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions