feat(docs): enhance search with TF-IDF relevance ranking (#40)#64
Closed
feat(docs): enhance search with TF-IDF relevance ranking (#40)#64
Conversation
aa56ed4 to
9015924
Compare
e125f1f to
e2ff25a
Compare
- Add relevance ranking toggle to existing Pagefind search (Ctrl+K) - When enabled: re-ranks results using TF-IDF cosine similarity with title/heading boost - Toggle persists in localStorage, off by default - Build-time index: chunks ~108 markdown files for relevance scoring - 9 search quality tests (schema, coverage, relevance, data quality) - Zero new dependencies — pure JS scoring enhancement Relates to #40 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
99421cc to
1b51247
Compare
The docs-search.test.ts requires docs/dist/search-index.json which is generated by the docs build. Add a lightweight CI step that runs the search index generator script and copies the output to docs/dist/ where the tests expect it. This avoids a full Astro build while still providing the artifact needed for search index validation tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
4321343 to
6d85ddd
Compare
Owner
Author
|
Closing: upstream PR exists on bradygaster/squad. Work continues there. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this improves
The docs site has Pagefind (⌘K) for full-text keyword search. This PR enhances that existing search with an optional TF-IDF relevance ranking toggle — no second search, no separate overlay.
Example: Searching "model pinning" with Pagefind returns every page mentioning "model". With relevance ranking ON, the Model Selection page ranks #1 because "pinning" appears in its title and headings (1.3x boost).
How it works
Visual Change
The existing ⌘K search dialog now includes a "✨ Relevance ranking" checkbox below the search input:
📸 Screenshot — Search with Relevance Toggle
Summary
Adds an optional relevance ranking enhancement to the existing Pagefind search:
build-search-index.mjs)No external APIs, no database, no runtime dependencies — everything runs in the browser.
Architecture (single search, not two)
SemanticSearch.astroas a separate componentSearch.astroas a re-ranking functionbuild-search-index.mjs— still generates the TF-IDF index at build timeFiles Changed
docs/scripts/build-search-index.mjsdocs/src/components/Search.astrodocs/src/components/SemanticSearch.astrodocs/src/layouts/DocsLayout.astrotest/docs-search.test.tsTesting
npx vitest run test/docs-search.test.ts)cd docs && npm run build)Decisions
Relates to #40