fix: remove unreasonable nullable check for data types in hash_joiner during merge operation#5784
Merged
wjones127 merged 10 commits intolance-format:mainfrom Jan 23, 2026
Merged
Conversation
Contributor
|
ACTION NEEDED The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. For details on the error please inspect the "PR Title Check" action. |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
wjones127
approved these changes
Jan 23, 2026
Contributor
wjones127
left a comment
There was a problem hiding this comment.
This looks good. Thanks for working on this!
Contributor
Author
Thanks @wjones127 :) |
vivek-bharathan
pushed a commit
to vivek-bharathan/lance
that referenced
this pull request
Feb 2, 2026
… during merge operation (lance-format#5784) During the invocation of add columns at https://github.com/lance-format/lance/blob/8c1842906bd275c32d403e4ddc78c7f77fa3de1f/rust/lance/src/dataset/fragment.rs#L1589, if the left table contains 100 rows of data and the right table contains 10 rows of data (with the remaining 90 rows being null), the validation at https://github.com/lance-format/lance/blob/8c1842906bd275c32d403e4ddc78c7f77fa3de1f/rust/lance/src/dataset/hash_joiner.rs#L192 will fail. **This is unreasonable because the validation here is a legacy validation that is no longer applicable to V2_0 and above.** ``` ValueError: Invalid user input: Found rows on LHS that do not match any rows on RHS. Lance would need to write nulls on the RHS, but Lance does not yet support nulls for type Float32., /Users/xxx/xxx/lance/rust/lance/src/dataset/hash_joiner.rs:197:36 ```
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
During the invocation of add columns at
lance/rust/lance/src/dataset/fragment.rs
Line 1589 in 8c18429
lance/rust/lance/src/dataset/hash_joiner.rs
Line 192 in 8c18429
This is unreasonable because the validation here is a legacy validation that is no longer applicable to V2_0 and above.