Skip to content

fix: fast_search limits full text search to indexed fragments#6006

Merged
BubbleCal merged 1 commit intomainfrom
yang/fastsearch-fts
Feb 25, 2026
Merged

fix: fast_search limits full text search to indexed fragments#6006
BubbleCal merged 1 commit intomainfrom
yang/fastsearch-fts

Conversation

@BubbleCal
Copy link
Copy Markdown
Contributor

@BubbleCal BubbleCal commented Feb 25, 2026

Summary

  • short-circuit FTS scans when fast_search is enabled and no indexed fragments exist so we return an empty plan instead of scanning unindexed data
  • skip the unindexed-match planning path entirely under fast_search, forcing only index-backed queries even when fragments exist
  • add plan verification and a regression test proving fast_search excludes rows appended after building the FTS index

@github-actions
Copy link
Copy Markdown
Contributor

ACTION NEEDED
Lance follows the Conventional Commits specification for release automation.

The PR title and description are used as the merge commit message. Please update your PR title and description to match the specification.

For details on the error please inspect the "PR Title Check" action.

@BubbleCal BubbleCal changed the title Ensure fast_search limits full text search to indexed fragments fix: fast_search limits full text search to indexed fragments Feb 25, 2026
@github-actions github-actions Bot added the bug Something isn't working label Feb 25, 2026
@github-actions
Copy link
Copy Markdown
Contributor

Code Review

Summary: This PR correctly implements fast_search behavior for full-text search by limiting results to indexed fragments only.

Assessment

The implementation is sound:

  • Uses existing patterns (EmptyExec::new(FTS_SCHEMA.clone())) consistent with the rest of the function
  • All three code paths (no index, all unindexed, mixed) are handled correctly
  • Regression test validates actual row counts, not just plan structure

No P0/P1 Issues Found

The code changes are well-scoped and follow the established patterns in the codebase.

Minor Suggestion (non-blocking)

The plan assertion test in the existing test case verifies the "mixed indexed/unindexed" scenario. Consider adding a plan assertion for the "no index at all + fast_search" path in a follow-up to ensure EmptyExec is returned, though the behavior is implicitly covered by the test_fts_fast_search_excludes_unindexed_rows test.

Copy link
Copy Markdown
Member

@westonpace westonpace left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome!

Comment on lines +3371 to +3373
if self.fast_search {
return Ok(Arc::new(EmptyExec::new(FTS_SCHEMA.clone())));
}
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly different from vector search. In vector search, if there is no index, then fast_search is just ignored (it still does an exhaustive KNN). However, I kind of like this behavior better. So I think I'm just pointing it out. We can update vector search later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ic, let me update the vector search behavior in next PR

@BubbleCal BubbleCal merged commit 8a87852 into main Feb 25, 2026
33 of 36 checks passed
@BubbleCal BubbleCal deleted the yang/fastsearch-fts branch February 25, 2026 13:12
wjones127 pushed a commit to wjones127/lance that referenced this pull request Feb 25, 2026
…format#6006)

Summary
- short-circuit FTS scans when `fast_search` is enabled and no indexed
fragments exist so we return an empty plan instead of scanning unindexed
data
- skip the unindexed-match planning path entirely under `fast_search`,
forcing only index-backed queries even when fragments exist
- add plan verification and a regression test proving `fast_search`
excludes rows appended after building the FTS index
wjones127 pushed a commit to wjones127/lance that referenced this pull request Feb 25, 2026
…format#6006)

Summary
- short-circuit FTS scans when `fast_search` is enabled and no indexed
fragments exist so we return an empty plan instead of scanning unindexed
data
- skip the unindexed-match planning path entirely under `fast_search`,
forcing only index-backed queries even when fragments exist
- add plan verification and a regression test proving `fast_search`
excludes rows appended after building the FTS index
wjones127 pushed a commit that referenced this pull request Feb 26, 2026
Summary
- short-circuit FTS scans when `fast_search` is enabled and no indexed
fragments exist so we return an empty plan instead of scanning unindexed
data
- skip the unindexed-match planning path entirely under `fast_search`,
forcing only index-backed queries even when fragments exist
- add plan verification and a regression test proving `fast_search`
excludes rows appended after building the FTS index
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants