Skip to content

feat(python): add storage_options to IvfModel and PqModel save/load#6312

Merged
Xuanwo merged 1 commit intolance-format:mainfrom
hushengquan:api_param
Mar 27, 2026
Merged

feat(python): add storage_options to IvfModel and PqModel save/load#6312
Xuanwo merged 1 commit intolance-format:mainfrom
hushengquan:api_param

Conversation

@hushengquan
Copy link
Copy Markdown
Contributor

Closes #6311

What

Add an optional storage_options keyword argument to IvfModel.save(), IvfModel.load(), PqModel.save(), and PqModel.load(), and forward it to the underlying LanceFileWriter / LanceFileReader.

Why

These methods accept cloud storage URIs (e.g. s3://, gs://) but previously had no way to pass credentials or backend-specific configuration. Users were forced to rely on environment variables for authentication, which breaks down when dealing with multiple storage backends that require different credentials.

Both LanceFileWriter and LanceFileReader already support storage_options — this change simply threads the parameter through.

Change Summary

  • python/python/lance/indices/ivf.py: Add storage_options parameter to IvfModel.save() and IvfModel.load(), forward to LanceFileWriter / LanceFileReader.
  • python/python/lance/indices/pq.py: Same change for PqModel.save() and PqModel.load().

Usage

from lance.indices.ivf import IvfModel
from lance.indices.pq import PqModel

opts = {"aws_access_key_id": "...", "aws_secret_access_key": "...", "region": "us-east-1"}

# Save
ivf_model.save("s3://bucket/ivf.lance", storage_options=opts)
pq_model.save("s3://bucket/pq.lance", storage_options=opts)

# Load
ivf_model = IvfModel.load("s3://bucket/ivf.lance", storage_options=opts)
pq_model = PqModel.load("s3://bucket/pq.lance", storage_options=opts)

@github-actions github-actions Bot added enhancement New feature or request python labels Mar 27, 2026
@github-actions
Copy link
Copy Markdown
Contributor

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

@hushengquan hushengquan changed the title feat(python): Add storage_options to IvfModel and PqModel save/load feat(python): add storage_options to IvfModel and PqModel save/load Mar 27, 2026
Copy link
Copy Markdown
Collaborator

@Xuanwo Xuanwo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@Xuanwo Xuanwo merged commit 55311f8 into lance-format:main Mar 27, 2026
15 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IvfModel.save/load and PqModel.save/load do not support storage_options

2 participants