[Variant] Simpler shredding state#8444
Merged
mbrobbel merged 18 commits intoapache:mainfrom Sep 26, 2025
Merged
Conversation
github-actions
Bot
added
parquet
mbrobbel
approved these changes
Sep 25, 2025
Contributor
Author
scovich
commented
Sep 25, 2025
| /// Create a new `ShreddingState` from the given `value` and `typed_value` fields | ||
| /// | ||
| /// Note you can create a `ShreddingState` from a &[`StructArray`] using | ||
| /// `ShreddingState::try_from(&struct_array)`, for example: |
Contributor
Author
There was a problem hiding this comment.
ShreddingState constructor became infallible some time ago, and it changed to impl From instead of TryFrom at the same time. But this example kept working because of a blanket impl TryFrom in rust that leverages From under the hood.
alamb
approved these changes
Sep 25, 2025
| /// | non-NULL | non-NULL | The value is present and is a partially shredded object | | ||
| /// | ||
| /// | ||
| /// Applying the above rules to entire columns, we obtain the following: |
Contributor
There was a problem hiding this comment.
thank you for preserving this information
Contributor
Author
|
Any reason not to merge? |
Contributor
|
🚀 |
klion26
reviewed
Sep 29, 2025
| /// | ||
| /// | value | typed_value | Meaning | | ||
| /// |--------|-------------|---------| | ||
| /// | -- | -- | **Missing**: The value is always missing; only valid for shredded object fields | |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Struct#8336Rationale for this change
While working on #8438, I noticed that the enum variants of
ShreddingStateactually made the code (a lot) more complex than if I just referenced the (optional) value and typed_value columns directly. That made me wonder ifShreddingStatewould be better as a simple two-field struct.What changes are included in this PR?
Change
ShreddingStateto a two-field struct and update the few call sites that noticed.While we're at it, improve the docs about how shredding works.
Are these changes tested?
Existing tests cover what is mostly an internal change
Are there any user-facing changes?
ShreddingStateis pub and changed from enum to struct.