Skip to content

Conversation

@jorgecarleitao
Copy link
Member

Like the title says, another helper.

Note that, contrarily to horizontal operations, that we can represent as f(&ArrayRef) -> ArrayRef, in vertical operations like min, and sum we do not have a dynamically-typed struct to return, and thus need to have a function signature for each type. :(

I wonder if we could have in Rust a dynamically struct for single values, like DataFusion has ScalarValue.

@github-actions
Copy link

@nevi-me
Copy link
Contributor

nevi-me commented Sep 11, 2020

There's a lot of value in creating StructValue in arrow, like cpp has. One of the potential benefits would be showing the compute kennels to either take an array or scalar

I've tried doing this previously, but threw away the work as I couldn't get it quite right.

@jorgecarleitao
Copy link
Member Author

@nevi-me , to understand, the goal would we a scalar that we could safely convert Vec<Scalar> from and to an array? What is the issue with

enum ScalarValue {
    Int32(Option<i32>)
    ...
}

@andygrove
Copy link
Member

I think that @nevi-me 's suggestion addresses something that I've been trying to solve as well. Our kernels operate on arrays, so calling a + b is fine when both are arrays, but if we're evaluating a + 1 against a batch of 1000 rows then we have to create an array of 1000 literal ones to pass to the kernel.

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants