Skip to content

Vector index never populated — hybrid search always falls back to BM25-only #22

@rohitg00

Description

@rohitg00

Problem

The system creates an embedding provider and a VectorIndex, but only SearchIndex (BM25) is populated during the compress pipeline. There is no code path that automatically generates embeddings and adds them to the vector index.

The HybridSearch can search both BM25 and vector indices, but the vector index will always be empty unless manually populated.

Locations

  • src/functions/compress.ts — only adds to BM25 search index, never to vector index
  • src/state/vector-index.ts — exists but never populated
  • src/state/hybrid-search.ts — falls back to BM25-only when vector index is empty

Suggested Fix

Add embedding generation in the compress pipeline when an embedding provider is available. After compressing an observation, generate an embedding from its title + facts + narrative, then add to the vector index.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workinghighHigh priority

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions