Skip to content

fix: vector search should respect target fragment#5080

Closed
yingjianwu98 wants to merge 1 commit intolance-format:mainfrom
yingjianwu98:vecotr_search_respect_fragments_scan
Closed

fix: vector search should respect target fragment#5080
yingjianwu98 wants to merge 1 commit intolance-format:mainfrom
yingjianwu98:vecotr_search_respect_fragments_scan

Conversation

@yingjianwu98
Copy link
Copy Markdown
Contributor

Vector search currently does not respect the target fragment list specified via Scanner.with_fragments().

Given a dataset with:

  • indexed_fragments = [1, 2, 3]
  • unindexed_fragments = [4, 5]

Scenario 1: Query targets only unindexed fragments [4, 5]

  • Bug: Still scans and returns results from indexed fragments [1, 2, 3]
  • Expected: Should only scan fragments [4, 5] using flat search

Scenario 2: Query targets mixed fragments [3, 4]

  • Bug: Scans indexed fragment 3 correctly, but also scans unindexed fragment 5 even though it's not in the target list
  • Expected: Should only scan fragments [3, 4]

@github-actions github-actions Bot added the bug Something isn't working label Oct 28, 2025
@yingjianwu98 yingjianwu98 changed the title fix: vector search should respect target fragment scanned fix: vector search should respect target fragment Oct 28, 2025
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Oct 28, 2025

Codecov Report

❌ Patch coverage is 89.28571% with 12 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/dataset/scanner.rs 89.28% 8 Missing and 4 partials ⚠️

📢 Thoughts on this report? Let us know!

@yingjianwu98
Copy link
Copy Markdown
Contributor Author

yingjianwu98 commented Oct 31, 2025

@westonpace @wjones127

Sorry for the direct ping, but I believe this bug could potentially have Lance to return results from unintended fragments. This might have broader implications so I want to bring this to your attention, thanks.

@yingjianwu98
Copy link
Copy Markdown
Contributor Author

closing this in favor of #5924

wjones127 pushed a commit that referenced this pull request Feb 17, 2026
…quested fragments (#5924)

Fixes bug where vector and FTS searches ignore `with_fragments() `filter
when querying unindexed fragments, which will return results from
indexed fragments that were not requested.

Note, this does not fix the issues where `with_fragments` contains both
indexed_fragement and non_indexed_fragment for FTS and vector search,
and I have a separate follow up PR to fix the behavior.


This PR combines my previous effort for fixing the issue:
#5081
#5080

---------

Co-authored-by: stevie9868 <yingjianwu2@email.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants