Skip to content

refactor: rework scalar index loading, training, and parsing into a plugin trait#4584

Merged
westonpace merged 10 commits intolance-format:mainfrom
westonpace:refactor/scalar-index-plugins
Sep 2, 2025
Merged

refactor: rework scalar index loading, training, and parsing into a plugin trait#4584
westonpace merged 10 commits intolance-format:mainfrom
westonpace:refactor/scalar-index-plugins

Conversation

@westonpace
Copy link
Copy Markdown
Member

These operations were previously all done via match statements. This made it difficult to create new index types.

Copy link
Copy Markdown
Contributor

@wjones127 wjones127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool to see the index types behind a trait.

Approved pending CI.

@westonpace westonpace force-pushed the refactor/scalar-index-plugins branch 2 times, most recently from 9ec646b to 28c3e0c Compare August 28, 2025 22:47
@westonpace westonpace force-pushed the refactor/scalar-index-plugins branch from 1d8599d to d34c875 Compare August 29, 2025 12:15
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Aug 29, 2025

Codecov Report

❌ Patch coverage is 73.91842% with 422 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.82%. Comparing base (0b82c81) to head (b86b246).
⚠️ Report is 4 commits behind head on main.

Files with missing lines Patch % Lines
rust/lance-index/src/scalar/zonemap.rs 61.66% 64 Missing and 5 partials ⚠️
rust/lance/src/index/scalar.rs 75.11% 43 Missing and 11 partials ⚠️
rust/lance-index/src/scalar/label_list.rs 48.42% 46 Missing and 3 partials ⚠️
rust/lance-index/src/scalar/inverted.rs 41.77% 43 Missing and 3 partials ⚠️
rust/lance-index/src/scalar/bitmap.rs 78.72% 23 Missing and 17 partials ⚠️
rust/lance/src/dataset/index.rs 42.30% 28 Missing and 2 partials ⚠️
rust/lance-index/src/scalar/ngram.rs 75.53% 12 Missing and 11 partials ⚠️
rust/lance-index/src/scalar/btree.rs 88.82% 6 Missing and 14 partials ⚠️
rust/lance-index/src/scalar.rs 60.46% 17 Missing ⚠️
rust/lance/src/index.rs 70.00% 11 Missing and 4 partials ⚠️
... and 11 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #4584      +/-   ##
==========================================
- Coverage   80.93%   80.82%   -0.11%     
==========================================
  Files         314      315       +1     
  Lines      117064   117713     +649     
  Branches   117064   117713     +649     
==========================================
+ Hits        94749    95146     +397     
- Misses      18931    19154     +223     
- Partials     3384     3413      +29     
Flag Coverage Δ
unittests 80.82% <73.91%> (-0.11%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions github-actions Bot added the java label Aug 29, 2025
@westonpace westonpace merged commit a40c17a into lance-format:main Sep 2, 2025
30 of 33 checks passed
@HaochengLIU
Copy link
Copy Markdown
Member

I like the PR, great simplification

jackye1995 pushed a commit to jackye1995/lance that referenced this pull request Jan 21, 2026
…lugin trait (lance-format#4584)

These operations were previously all done via match statements. This
made it difficult to create new index types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants