Skip to content

feat: add native oss support for lance#4247

Merged
Xuanwo merged 3 commits intolance-format:mainfrom
Xuanwo:4205-oss-support
Jul 17, 2025
Merged

feat: add native oss support for lance#4247
Xuanwo merged 3 commits intolance-format:mainfrom
Xuanwo:4205-oss-support

Conversation

@Xuanwo
Copy link
Copy Markdown
Collaborator

@Xuanwo Xuanwo commented Jul 17, 2025

Fix #4205

Signed-off-by: Xuanwo <github@xuanwo.io>
@github-actions github-actions Bot added the enhancement New feature or request label Jul 17, 2025
Signed-off-by: Xuanwo <github@xuanwo.io>
Copy link
Copy Markdown
Contributor

@BubbleCal BubbleCal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

have you tried this on aliyun OSS?

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

Attention: Patch coverage is 11.11111% with 64 lines in your changes missing coverage. Please review.

Project coverage is 80.20%. Comparing base (146176e) to head (f26c46e).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
rust/lance-io/src/object_store/providers/oss.rs 9.85% 64 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4247      +/-   ##
==========================================
- Coverage   80.24%   80.20%   -0.05%     
==========================================
  Files         297      298       +1     
  Lines      105211   105283      +72     
  Branches   105211   105283      +72     
==========================================
+ Hits        84429    84440      +11     
- Misses      17706    17769      +63     
+ Partials     3076     3074       -2     
Flag Coverage Δ
unittests 80.20% <11.11%> (-0.05%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Xuanwo
Copy link
Copy Markdown
Collaborator Author

Xuanwo commented Jul 17, 2025

have you tried this on aliyun OSS?

Yep, tested with an oss example:

Using dataset URI: oss://opendal-testing/test/dataset
Created test data with 5 rows

Writing data to OSS...
Data written successfully!

Reading data from OSS...
Dataset opened successfully!
Schema: Schema { fields: [Field { name: "id", id: 0, parent_id: -1, logical_type: LogicalType("int32"), metadata: {}, encoding: Some(Plain), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }, Field { name: "name", id: 1, parent_id: -1, logical_type: LogicalType("string"), metadata: {}, encoding: Some(VarBinary), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }, Field { name: "score", id: 2, parent_id: -1, logical_type: LogicalType("float"), metadata: {}, encoding: Some(Plain), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }], metadata: {} }
Current version: 1
Number of rows: 5

Scanning data:
Batch 0: 5 rows
Sample data:
  Row 0: id=1, name=Alice, score=85.5
  Row 1: id=2, name=Bob, score=92.3
  Row 2: id=3, name=Charlie, score=78.9
  Row 3: id=4, name=David, score=88.1
  Row 4: id=5, name=Eve, score=95.7

Example completed successfully!

Signed-off-by: Xuanwo <github@xuanwo.io>
@Xuanwo Xuanwo merged commit 8936bdd into lance-format:main Jul 17, 2025
25 of 27 checks passed
@AshDyh1999
Copy link
Copy Markdown

have you tried this on aliyun OSS?

Yep, tested with an oss example:

Using dataset URI: oss://opendal-testing/test/dataset
Created test data with 5 rows

Writing data to OSS...
Data written successfully!

Reading data from OSS...
Dataset opened successfully!
Schema: Schema { fields: [Field { name: "id", id: 0, parent_id: -1, logical_type: LogicalType("int32"), metadata: {}, encoding: Some(Plain), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }, Field { name: "name", id: 1, parent_id: -1, logical_type: LogicalType("string"), metadata: {}, encoding: Some(VarBinary), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }, Field { name: "score", id: 2, parent_id: -1, logical_type: LogicalType("float"), metadata: {}, encoding: Some(Plain), nullable: false, children: [], dictionary: None, storage_class: Default, unenforced_primary_key: false }], metadata: {} }
Current version: 1
Number of rows: 5

Scanning data:
Batch 0: 5 rows
Sample data:
  Row 0: id=1, name=Alice, score=85.5
  Row 1: id=2, name=Bob, score=92.3
  Row 2: id=3, name=Charlie, score=78.9
  Row 3: id=4, name=David, score=88.1
  Row 4: id=5, name=Eve, score=95.7

Example completed successfully!

Hi @Xuanwo , thanks for adding this feature!
Could you please provide an example or documentation on how to use this new feature?
I'd love to try it out but I'm not sure about the correct usage.
Thanks in advance!

Xuanwo pushed a commit that referenced this pull request Mar 12, 2026
Improvements #4247 alicloud
storage config doc.

Signed-off-by: FarmerChillax <farmerchillax@outlook.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Concurrent commit overwrites + OSS If-None-Match write failures (Lance ≥ 0.24) – any mature commit_lock solution?

4 participants