feat(oss): add sts token support for aliyun oss via storage_options#5632
Conversation
|
ACTION NEEDED The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification. For details on the error please inspect the "PR Title Check" action. |
hh23485
changed the title
|
Hi @wjones127, I’m not entirely sure, but it seems that the Claude Code Review CI here requires the current actor to have write permission to this repo, so I’m unable to make this check pass. This change is quite small. Could you please help review it when you have a chance? |
hh23485
changed the title
|
Is there any reviewer interested in this feature? We rely on this capability for using Lance in our production environment |
Hi @Xuanwo thanks! I ran into some CI failures (cargo-deny and aws-integtest). Since I haven't modified any dependencies and the AWS test seems unrelated to my OSS changes, I believe these are not caused by my code. I've merged the latest main hoping to resolve any environment issues. If they fail again, could you please help check? Links: https://github.com/lance-format/lance/actions/runs/20742942831/job/59776109503?pr=5632 |
|
Hi @Xuanwo Can you help on this? |
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
|
Hi @Xuanwo, I'd like to know when this commit will be released with a new version of Pylance. I see that the latest 1.0.2 version doesn't include it, so what's the release plan here. |
…ance-format#5632) ## Summary This PR adds support for passing an OSS STS security token through `storage_options` when using Lance with Aliyun OSS or other lake‑based storage systems. In multi‑tenant environments, operating on OSS with short‑lived STS credentials is a critical use case. Previous versions only extracted `access_key_id` and `access_key_secret`, so the security token was not recognized. This change ensures that the token can be provided explicitly via `storage_options`. --- ## Motivation While environment variables can technically be used, they are not suitable in multi‑tenant scenarios due to process‑level leakage risks and the inability to safely refresh credentials at runtime. Passing credentials through `storage_options` remains the correct and secure approach. --- ## Implementation Notes The change is intentionally minimal. Although we could introduce a whitelist to iterate over allowed OSS fields, this does not appear necessary at the moment given the current scope and usage patterns. Co-authored-by: Xuanwo <github@xuanwo.io>
2 participants
Summary
This PR adds support for passing an OSS STS security token through
storage_optionswhen using Lance with Aliyun OSS or other lake‑based storage systems.In multi‑tenant environments, operating on OSS with short‑lived STS credentials is a critical use case. Previous versions only extracted
access_key_idandaccess_key_secret, so the security token was not recognized. This change ensures that the token can be provided explicitly viastorage_options.Motivation
While environment variables can technically be used, they are not suitable in multi‑tenant scenarios due to process‑level leakage risks and the inability to safely refresh credentials at runtime. Passing credentials through
storage_optionsremains the correct and secure approach.Implementation Notes
The change is intentionally minimal. Although we could introduce a whitelist to iterate over allowed OSS fields, this does not appear necessary at the moment given the current scope and usage patterns.