Epic: Core Context Provider

[prosdev]

Description

This epic focuses on building the essential context provider functionality that forms the foundation of the dev-agent system. The Core Context Provider analyzes code repositories, generates and stores vector embeddings, and provides relevant context to AI tools, significantly reducing hallucinations and improving the accuracy of AI responses when working with codebases.

Goals

1. ✅ Create a robust repository scanner that extracts components and relationships from TypeScript/JavaScript codebases and documentation (Markdown)
2. ✅ Implement vector storage and retrieval using LanceDB and Transformers.js for local-first semantic search
3. ✅ Develop a context assembly system that provides the most relevant code sections based on queries
4. ✅ Build a CLI interface for basic operations
5. 🔄 HTTP API Server deferred (not needed for local-first CLI workflow)

Architecture Decision: CLI-First

Original Plan: HTTP API server for AI tool integration

What We Built: CLI-first architecture with JSON output

• dev search with --json flag - Structured output for tools
• dev plan with --json flag - Machine-readable plans
• dev gh search with --json flag - GitHub context
• All commands support JSON for tool integration

Why CLI-First Works:

• ✅ Works with all AI tools (Claude Code, Cursor, Antigravity)
• ✅ No server management overhead
• ✅ Fast enough for local workflows (~200ms warm, ~1-2s cold)
• ✅ Simple deployment (just install CLI)
• ✅ Better for local-first philosophy

HTTP API Status:

• 🔄 Deferred until web UI or remote access is needed
• 🔄 Build when/if use case emerges
• Current stubs in packages/core/src/api/ preserved for future

Completed Work

✅ Repository Scanner (Issue #3)

• TypeScript scanning with ts-morph
• Markdown documentation parsing with Remark
• Component extraction (functions, classes, interfaces)
• Relationship mapping (imports, dependencies)
• Comprehensive test coverage

✅ Vector Storage (Issue #4)

• LanceDB for local-first storage
• Transformers.js for embeddings
• Semantic search capabilities
• High-performance retrieval
• Edge case handling

✅ Infrastructure (Issues #11, #12)

• Configuration management system
• Repository indexer integration layer
• Unified API across packages
• Type-safe interfaces

✅ CLI Tool (Issue #6)

• dev init - Initialize configuration
• dev scan - Scan repository
• dev search - Semantic search
• dev plan - Generate plans from issues
• dev explore - Pattern discovery and code exploration
• dev gh - GitHub context integration
• All commands support --json for tool integration

✅ Context Assembly

• Relevance ranking in indexer
• Query processing and filtering
• Result formatting (JSON, pretty, markdown)
• Integration with subagent system

Acceptance Criteria

• ✅ Repository scanner successfully identifies functions, classes, interfaces, and their relationships, plus documentation
• ✅ Vector storage system stores and retrieves code embeddings with high accuracy using local models
• ✅ Context assembly system ranks and formats context appropriately for AI tools (via CLI)
• ✅ CLI tool allows users to scan repositories, manage context, and perform operations with JSON output
• ✅ All components are thoroughly tested with >80% test coverage (557 tests passing!)
• ✅ Documentation is comprehensive and clear (4 comprehensive READMEs, 2,000+ lines)

Definition of Done

• ✅ All code passes linting, formatting, and type checking
• ✅ All tests are passing with >= 80% coverage (557 tests across 30 files!)
• ✅ Performance benchmarks meet targets for repository scanning and context retrieval
• ✅ Documentation is complete, including API reference and user guide
• ✅ Real-world validation through dogfooding

By The Numbers

• 557 tests passing across 30 test files
• 100% coverage for all utility functions
• 2,000+ lines of comprehensive documentation
• ~8,000 lines of production code (including tests)
• 4 subagents working together seamlessly

Next Steps

MCP Integration (Issue #26):
The recommended next step is implementing an MCP (Model Context Protocol) server to provide native integration with AI tools:

• Claude Desktop (native MCP support)
• Claude Code (native MCP support)
• Cursor (recently added MCP support)

This will provide better UX than shell commands while maintaining the local-first philosophy.

Priority: High (Core foundation complete)

Status: ✅ COMPLETE - READY FOR PRODUCTION