Skip to content

Add semantic vector search via Qdrant + Ollama#11

Open
renezander030 wants to merge 1 commit intokvanland:mainfrom
renezander030:feat/vector-search
Open

Add semantic vector search via Qdrant + Ollama#11
renezander030 wants to merge 1 commit intokvanland:mainfrom
renezander030:feat/vector-search

Conversation

@renezander030
Copy link
Copy Markdown

Problem

The current tasks search command fires N+1 API calls on every query (1 to list projects, then 1 per project to fetch and filter tasks). With hundreds of tasks across many projects, this takes 3-5 seconds per search. Worse, substring matching misses semantically related results: searching "deployment" won't find a task titled "push release to prod".

Solution

Add optional vector search using Qdrant (vector database) and Ollama (local embeddings with nomic-embed-text). Both run locally, no external API calls, no data leaves the machine.

How it works

  1. vector-sync fetches all active tasks, embeds them via Ollama, and stores vectors in Qdrant
  2. Incremental sync uses MD5 content hashing (title|content|tags) to skip unchanged tasks
  3. Metadata-only changes (priority, dueDate) update the Qdrant payload without re-embedding
  4. semantic queries embed the search string and match by cosine similarity in <100ms
  5. similar finds semantically related tasks for deduplication or discovering related work
  6. If Qdrant/Ollama are down, semantic search falls back to keyword search automatically

What I changed

  • lib/vector.js (new): Qdrant + Ollama integration, incremental sync, search, similar tasks, health checks
  • lib/tasks.js: Added semanticSearch, findSimilar, vectorSync, vectorStatus with keyword fallback
  • bin/ticktick.js: New subcommands: semantic, similar, vector-sync, vector-status
  • lib/mcp.js: 4 new MCP tools for AI assistant integration
  • lib/cli.js: Score-based result formatting, updated help text
  • README.md: Full documentation with prerequisites, usage, cron setup

Production context

I've been running this architecture (same stack: Qdrant + Ollama + nomic-embed-text, same incremental sync pattern) on my own TickTick MCP server for several months with ~500 tasks. The speed difference is significant, and the semantic matching catches things keyword search never will.

New CLI commands

ticktick tasks vector-sync                           # Build/update the index
ticktick tasks semantic "anything about deployments"  # Semantic search
ticktick tasks similar TASK_ID                        # Find similar tasks
ticktick tasks vector-status                          # Index health check

New MCP tools

Tool Description
ticktick_tasks_semantic_search Semantic search via vector similarity
ticktick_tasks_similar Find semantically similar tasks
ticktick_vector_sync Sync tasks into vector index
ticktick_vector_status Check vector index health

Prerequisites (optional)

Vector search is entirely optional. If Qdrant/Ollama aren't running, everything else works exactly as before.

docker run -d --name qdrant -p 6333:6333 qdrant/qdrant
docker run -d --name ollama -p 11434:11434 ollama/ollama
docker exec ollama ollama pull nomic-embed-text

Test plan

  • Existing tests pass (5 pre-existing failures unrelated to this change)
  • ticktick tasks vector-sync indexes tasks from the API
  • ticktick tasks semantic "query" returns scored results
  • ticktick tasks similar TASK_ID finds related tasks
  • ticktick tasks vector-status reports index health
  • Semantic search falls back to keyword when Qdrant is down
  • MCP tools appear and work in Claude Desktop/Code

Keyword search iterates every project via the API on each query (N+1 calls)
and only does substring matching. Vector search queries a local Qdrant index
in <100ms and finds semantically related tasks that keywords miss.

New CLI commands: semantic, similar, vector-sync, vector-status
New MCP tools: ticktick_tasks_semantic_search, ticktick_tasks_similar,
ticktick_vector_sync, ticktick_vector_status

Falls back to keyword search when Qdrant/Ollama are unavailable.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant