[Variant]: Implement DataType::Decimal32/Decimal64/Decimal128/Decimal256 support for cast_to_variant kernel#8101
Conversation
…l256` support for `cast_to_variant` kernel
c1feb1b to
ffb3509
Compare
alamb
left a comment
There was a problem hiding this comment.
Thank you @liamzwbao -- this makes sense to me
| Decimal256Type, | ||
| as_primitive, | ||
| |v: i256| { | ||
| // Since `i128::MAX` is larger than the max value of `VariantDecimal16`, |
There was a problem hiding this comment.
Since DataType::Decimal256 can store values larger than can be stored in VariantDecimal16, I don't think this comment is accurate -- don't we need another check for overflow?
But I see that the tests covert the case of i256::MAX so this seems reasonable to me
There was a problem hiding this comment.
Improved the comments to make it clearer. What I was trying to say here is that if an i256 cannot fit into i128, then it can not fit into VariantDecimal16 either.
So here we can first convert i256 to i128 and process it like i128.
| Arc::new( | ||
| Decimal128Array::from(vec![ | ||
| Some(i128::MIN), | ||
| Some(-max_unscaled_value!(128, DECIMAL128_MAX_PRECISION)), |
There was a problem hiding this comment.
Shouldn't this be DECIMAL256_MAX_PRECISION?
| Some(-max_unscaled_value!(128, DECIMAL128_MAX_PRECISION)), | |
| Some(-max_unscaled_value!(128, DECIMAL256_MAX_PRECISION)), |
Same question for the other uses below too
There was a problem hiding this comment.
I guess you meant the Decimal256 tests. The reason I didn't use DECIMAL256_MAX_PRECISION is that DECIMAL128_MAX_PRECISION can already cause the overflow in the test cases.
I have updated the test cases to include the min/max value that can be cast from Decimal to VariantDecimal so these should be clearer
Which issue does this PR close?
DataType::Decimal32/Decimal64/Decimal128/Decimal256support forcast_to_variantkernel #8059.Rationale for this change
See the linked issue.
What changes are included in this PR?
Created a new macro to convert Arrow decimal to variant decimal. Support
Decimal32/Decimal64/Decimal128/Decimal256forcast_to_variant.Are these changes tested?
Yes
Are there any user-facing changes?
Yes, new variant casting supported