Skip to content

refactor: add store_prefix to lance-io's ObjectStore#5468

Merged
westonpace merged 5 commits intolance-format:mainfrom
cmccabe:add_store_prefix_to_dataset
Dec 18, 2025
Merged

refactor: add store_prefix to lance-io's ObjectStore#5468
westonpace merged 5 commits intolance-format:mainfrom
cmccabe:add_store_prefix_to_dataset

Conversation

@cmccabe
Copy link
Copy Markdown
Contributor

@cmccabe cmccabe commented Dec 13, 2025

Add the store_prefix to lance-io's ObjectStore. This is helpful when caching ObjectStore objects, so that we can tell which ones map to the same data.

Add the store_prefix to lance-io's ObjectStore. This is helpful when
caching ObjectStore objects, so that we can tell which ones map to the
same data.
@chatgpt-codex-connector
Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

Copy link
Copy Markdown
Member

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems reasonable to me. This object is somewhat indecisive between using public properties (list_is_lexically_ordered, use_constant_size_upload_parts) and accessors (block_size, max_iop_size) so I guess both approaches are fine 😆

@Xuanwo
Copy link
Copy Markdown
Collaborator

Xuanwo commented Dec 16, 2025

Seems tests case around the following failed, maybe worth another look, cc @cmccabe


failures:
    scheduler::tests::test_backpressure
    scheduler::tests::test_priority

@cmccabe
Copy link
Copy Markdown
Contributor Author

cmccabe commented Dec 18, 2025

Should be fixed now. I added a fallback path for object stores that aren't in the registry but that's we're "wrapping" via the deprecated API. For those we have to guess the storage prefix, unfortunately, since nothing can tell us about it. It should be OK to just use scheme+authority there.

@codecov
Copy link
Copy Markdown

codecov Bot commented Dec 18, 2025

Codecov Report

❌ Patch coverage is 58.33333% with 5 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance-io/src/object_store.rs 54.54% 5 Missing ⚠️

📢 Thoughts on this report? Let us know!

@westonpace westonpace merged commit 303ee1e into lance-format:main Dec 18, 2025
28 of 29 checks passed
wjones127 pushed a commit to wjones127/lance that referenced this pull request Dec 30, 2025
Add the store_prefix to lance-io's ObjectStore. This is helpful when
caching ObjectStore objects, so that we can tell which ones map to the
same data.
jackye1995 pushed a commit to jackye1995/lance that referenced this pull request Jan 21, 2026
Add the store_prefix to lance-io's ObjectStore. This is helpful when
caching ObjectStore objects, so that we can tell which ones map to the
same data.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants