chore: cherry-pick fixes for v3.0.1 release#6217
Merged
westonpace merged 5 commits intorelease/v3.0from Mar 18, 2026
Merged
Conversation
…6160) Previously, `adjust_child_validity` would call `ArrayData::try_new` with a null bitmap on a `DataType::Null` array, causing an `.unwrap()` panic with `InvalidArgumentError("Arrays of type Null cannot contain a null bitmask")`. The trigger: when a user inserts rows where a struct sub-field has only null values, Arrow infers `DataType::Null` for that column. If a subsequent fragment omits that nullable sub-field, Lance inserts a `NullReader` to fill it in. `MergeStream` then merges the real batch (with null struct rows) and the `NullReader` batch (all-null struct), recursing into the struct where `adjust_child_validity` is called with the `Null`-typed child and a non-empty parent validity — triggering the panic. Fix: skip the bitmask operation when `child.data_type() == DataType::Null`. A `Null` array is always entirely null by definition and needs no validity adjustment. Closes #6159 --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
PR ReviewClean cherry-pick of two targeted bug fixes for the v3.0.1 patch release. Both fixes are well-scoped and well-tested. No issues found. Both fixes are correct:
Test coverage is thorough: unit tests at the Arrow and encoding layers, plus an end-to-end dataset test that exercises the full read path with schema evolution and null structs. LGTM. 🤖 Generated with Claude Code |
#6143) Change conflict_retries from 0 to 5 in insert_into_manifest so that cross-process races are handled correctly. When two processes concurrently insert the same object_id, the second one hits a commit version conflict. With conflict_retries > 0, MergeInsert retries by re-evaluating the full plan against the latest data, where the join detects the existing row and WhenMatched::Fail fires properly. Previously, conflict_retries=0 meant the second operation would fail with a generic TooMuchWriteContention error, but in some cases both commits could succeed creating duplicate manifest entries ("Expected exactly 1 table...found 2"). Add test with two independent ManifestNamespace instances racing on the same directory to verify no duplicates are created. Here's an example I run into occasionally ``` ... File "/home/runner/work/geneva/geneva/src/geneva/state/manager.py", line 35, in __init__ self.table = alter_or_create_table( File "/home/runner/work/geneva/geneva/src/geneva/utils/schema.py", line 138, in alter_or_create_table return db.create_table(table_name, schema=schema, namespace=namespace) File "/home/runner/work/geneva/geneva/src/geneva/db.py", line 403, in create_table return Table(self, name, namespace=namespace, storage_options=storage_options) File "/home/runner/work/geneva/geneva/src/geneva/table.py", line 489, in __init__ self._ltbl # noqa File "/home/runner/.local/share/uv/python/cpython-3.10-linux-x86_64-gnu/lib/python3.10/functools.py", line 981, in __get__ val = self.func(instance) File "/home/runner/work/geneva/geneva/src/geneva/table.py", line 543, in _ltbl tbl = inner.open_table(self.name, namespace=self._namespace) File "/home/runner/work/geneva/geneva/.venv/lib/python3.10/site-packages/lancedb/namespace.py", line 392, in open_table response = self._ns.describe_table(request) File "/home/runner/work/geneva/geneva/.venv/lib/python3.10/site-packages/lance/namespace.py", line 362, in describe_table response_dict = self._inner.describe_table(request.model_dump()) OSError: LanceError(IO): Expected exactly 1 table with id 'default$geneva_manifests', found 2, /home/runner/work/lance/lance/rust/lance-namespace-impls/src/dir/manifest.rs:642:21 ``` --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This updates both transitive `lz4_flex` lines to their latest patch releases so the lockfile no longer carries the yanked `0.11.5` release. The `0.11.x` line comes from `tantivy`, and the `0.12.x` line comes from the Arrow/Parquet/DataFusion stack. Validated with `cargo check -p lance --locked`.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
fix CI error: `FAILED python/tests/test_integration.py::test_duckdb_pushdown_extension_types - _duckdb.Error: DeprecationWarning: fetch_arrow_table() is deprecated, use to_arrow_table() instead.`
Xuanwo
approved these changes
Mar 18, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cherry-picks bug fixes onto
release/v3.0for the v3.0.1 patch release:DataType::Nullinadjust_child_validityto prevent panicfetch_arrow_tablewithto_arrow_table#6146 - fix: replace fetch_arrow_table with to_arrow_tableTest plan