Skip to content

Use arrow bitwise operation rather than reimplementation #6066

@RTEnzyme

Description

@RTEnzyme

Is your feature request related to a problem or challenge?

Currently, binary bitwise array operations are being reimplemented using a loop in repository(https://github.com/apache/arrow-datafusion/blob/main/datafusion/physical-expr/src/expressions/binary/kernels.rs#L30):

/// The binary_bitwise_array_op macro only evaluates for integer types
/// like int64, int32.
/// It is used to do bitwise operation.
macro_rules! binary_bitwise_array_op {
    ($LEFT:expr, $RIGHT:expr, $METHOD:expr, $ARRAY_TYPE:ident) => {{
        let len = $LEFT.len();
        let left = $LEFT.as_any().downcast_ref::<$ARRAY_TYPE>().unwrap();
        let right = $RIGHT.as_any().downcast_ref::<$ARRAY_TYPE>().unwrap();
        let result = (0..len)
            .into_iter()
            .map(|i| {
                if left.is_null(i) || right.is_null(i) {
                    None
                } else {
                    Some($METHOD(left.value(i), right.value(i)))
                }
            })
            .collect::<$ARRAY_TYPE>();
        Ok(Arc::new(result))
    }};
}

I think arrow-rs bitwise operation(https://github.com/apache/arrow-rs/blob/master/arrow-arith/src/bitwise.rs#L24) can be used to achieve better performance.

Describe the solution you'd like

Use arrow-arith bitwise opeartion(https://github.com/apache/arrow-rs/blob/master/arrow-arith/src/bitwise.rs#L24)

Describe alternatives you've considered

No response

Additional context

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions