feat: serialize storage options in table identifier proto

[LuQQiu]

add storage options in table identifier proto to allow pass in storage credentials or information.
Make dataset parameter optional in filtered_read_exec_from_proto, falls back to opening from the proto's table identifier

[github-actions]

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

[github-actions]

Code Review

P0 - Security Concern: Credentials in Serialized Proto

The addition of storage_options to TableIdentifier proto introduces a security concern. According to the codebase, storage_options can contain sensitive credentials:

• AWS access keys and secret keys
• Security tokens (OSS, Tencent Cloud, HuggingFace)
• API keys and authentication tokens
• Credentials from secret managers

These are now being serialized into the proto message for distributed execution (planner → executor). This could expose credentials if:

1. Proto messages are logged - common in distributed systems for debugging
2. Network transport is not encrypted - credentials sent in plaintext
3. Proto messages are cached/persisted - credentials stored insecurely

Suggestions:

• Consider whether storage options should be passed out-of-band (e.g., via a secure credential provider on the executor side)
• At minimum, add documentation warning that this proto should only be sent over encrypted channels
• Consider filtering out sensitive keys before serialization, or use a dedicated credentials field with explicit handling

If this is intentional for specific distributed execution use cases with proper security controls, please document the expected security model.

[codecov]

Codecov Report

❌ Patch coverage is 56.09756% with 18 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance/src/io/exec/filtered_read_proto.rs	56.09%	15 Missing and 3 partials ⚠️

📢 Thoughts on this report? Let us know!

[jackye1995]

I think this should call latest_storage_options()?

[LuQQiu]

oh that's good!

[jackye1995]

I feel at this point it's more like a table state than table identifier, not sure if we can still change ethe name though

[LuQQiu]

emmm, we mention not make the proto backward compatible, this may be merged within one week... emm

[jackye1995]

2 comments, other parts looks good to me