[VARIANT] Validate precision in VariantDecimalXX structs and add missing tests#7776
Merged
alamb merged 2 commits intoapache:mainfrom Jun 25, 2025
Merged
[VARIANT] Validate precision in VariantDecimalXX structs and add missing tests#7776alamb merged 2 commits intoapache:mainfrom
alamb merged 2 commits intoapache:mainfrom
Conversation
Contributor
Author
scovich
commented
Jun 25, 2025
| @@ -1,5 +1,3 @@ | |||
| use std::ops::Deref; | |||
Contributor
Author
There was a problem hiding this comment.
not sure how this ended up here...
Contributor
There was a problem hiding this comment.
we can blame the IDE perhaps
| #[derive(Debug, Clone, Copy, PartialEq)] | ||
| pub struct ShortString<'a>(pub(crate) &'a str); | ||
|
|
||
| /// Represents a 4-byte decimal value in the Variant format. |
Contributor
Author
There was a problem hiding this comment.
Moved to decimal.rs
Comment on lines
+34
to
+35
| const MAX_PRECISION: u32 = 9; | ||
| const MAX_UNSCALED_VALUE: u32 = 10_u32.pow(Self::MAX_PRECISION) - 1; |
Contributor
Author
There was a problem hiding this comment.
Hoisted up and renamed the existing constant, and used it to define the other constant
Comment on lines
+41
to
+43
| "Scale {} of a 4-byte decimal cannot exceed the max precision {}", | ||
| scale, | ||
| Self::MAX_PRECISION, |
Contributor
Author
There was a problem hiding this comment.
Updated the error message to reference the constant instead of a magic number
Comment on lines
+47
to
+54
| // Validate that the integer value fits within the precision | ||
| if integer.unsigned_abs() > Self::MAX_UNSCALED_VALUE { | ||
| return Err(ArrowError::InvalidArgumentError(format!( | ||
| "{} is too large to store in a 4-byte decimal with max precision {}", | ||
| integer, | ||
| Self::MAX_PRECISION | ||
| ))); | ||
| } |
Contributor
Author
There was a problem hiding this comment.
The newly added validation
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
VariantBuilder#7697Rationale for this change
As a follow-up to #7738, we should verify that the unscaled integer value fits in the max precision (scale factor was already validated).
What changes are included in this PR?
Add the missing checking, and add missing unit tests for both precision and scale.
Also move the
VariantDecimalXXstructs to their own mod.Are these changes tested?
Yes, see above.
Are there any user-facing changes?
No. Public re-rexport of the moved structs.