Skip to content

feat: add codetect-cli for MCP vs CLI token-cost A/B testing#85

Open
brian-lai wants to merge 10 commits into
mainfrom
para/cli-ab-testing
Open

feat: add codetect-cli for MCP vs CLI token-cost A/B testing#85
brian-lai wants to merge 10 commits into
mainfrom
para/cli-ab-testing

Conversation

@brian-lai
Copy link
Copy Markdown
Owner

Summary

  • Adds codetect-cli binary providing the same 4 capabilities as the MCP server via direct CLI subcommands
  • Enables A/B testing token consumption: MCP tools vs Bash tool invoking codetect-cli
  • Reuses 100% of existing tool handler code via server.CallTool() — identical code paths, only transport differs

CLI Interface

codetect-cli search <query> [--top-k N] [--detail minimal|standard|rich]
codetect-cli file <path> [--start-line N] [--end-line N]
codetect-cli symbols find <name> [--kind KIND] [--limit N]
codetect-cli symbols list <path>
codetect-cli hybrid <query> [--limit N] [--rerank] [--detail minimal|standard|rich]

Changes

  • cmd/codetect-cli/main.go — CLI entry point with subcommand routing (223 lines)
  • cmd/codetect-cli/main_test.go — 24 tests covering arg parsing and tool invocation
  • Makefile — Added build-cli target, updated build/install/uninstall

Test plan

  • 24 unit tests pass (go test ./cmd/codetect-cli/...)
  • make build-cli succeeds
  • All 4 subcommands produce valid JSON output matching MCP tool format
  • Graceful degradation when resources unavailable (symbols returns {"available": false})

🤖 Generated with Claude Code

brian-lai and others added 10 commits April 6, 2026 02:10
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… list modes)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… tests

- Add defer cleanup() for ResourcePool in all run* functions
- Add version sync comment to cliVersion const
- Add table-driven test for help/-h/--help variants
- Strengthen version test to check output contains version string
- Document test server config asymmetry (DefaultConfig vs production)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant