diff --git a/.claude/skills/dogfood/SKILL.md b/.claude/skills/dogfood/SKILL.md index 1cce1479..8eb886a6 100644 --- a/.claude/skills/dogfood/SKILL.md +++ b/.claude/skills/dogfood/SKILL.md @@ -203,7 +203,7 @@ Before writing the report, **stop and think** about: - What testing approaches am I missing? - **Cross-command pipelines:** Have I tested `build` → `embed` → `search` → modify → `build` → `search`? Have I tested `watch` detecting changes then `diff-impact`? -- **MCP server:** Have I tested the `mcp` command? Initialize via JSON-RPC on stdin, send `tools/list`, verify all 21 tools are present. Test single-repo mode (default — `list_repos` should be absent, no `repo` parameter on tools) vs `--multi-repo` mode. +- **MCP server:** Have I tested the `mcp` command? Initialize via JSON-RPC on stdin, send `tools/list`, verify all 24 tools are present (23 in single-repo mode; 24 with `list_repos` in multi-repo). Test single-repo mode (default — `list_repos` should be absent, no `repo` parameter on tools) vs `--multi-repo` mode. - **Programmatic API:** Have I tested `require('@optave/codegraph')` or `import` from `index.js`? Key exports to verify: `buildGraph`, `loadConfig`, `openDb`, `findDbPath`, `contextData`, `explainData`, `whereData`, `fnDepsData`, `diffImpactData`, `statsData`, `isNativeAvailable`, `EXTENSIONS`, `IGNORE_DIRS`, `ALL_SYMBOL_KINDS`, `MODELS`. - **Config options:** Have I tested `.codegraphrc.json`? Create one with `include`/`exclude` patterns, custom `aliases`, `build.incremental: false`, `query.defaultDepth`, `search.defaultMinScore`. Verify overrides work. - **Env var overrides:** `CODEGRAPH_LLM_PROVIDER`, `CODEGRAPH_LLM_API_KEY`, `CODEGRAPH_LLM_MODEL`, `CODEGRAPH_REGISTRY_PATH`. diff --git a/CLAUDE.md b/CLAUDE.md index e5a47d58..f76eb2dc 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,6 +56,10 @@ JS source is plain JavaScript (ES modules) in `src/`. No transpilation step. The | `native.js` | Native napi-rs addon loader with WASM fallback | | `registry.js` | Global repo registry (`~/.codegraph/registry.json`) for multi-repo MCP | | `resolve.js` | Import resolution (supports native batch mode) | +| `complexity.js` | Cognitive, cyclomatic, Halstead, MI computation from AST; `complexity` CLI command | +| `communities.js` | Louvain community detection, drift analysis | +| `manifesto.js` | Configurable rule engine with warn/fail thresholds; CI gate | +| `paginate.js` | Pagination helpers for bounded query results | | `logger.js` | Structured logging (`warn`, `debug`, `info`, `error`) | **Key design decisions:** @@ -71,7 +75,7 @@ JS source is plain JavaScript (ES modules) in `src/`. No transpilation step. The - **MCP single-repo isolation:** `startMCPServer` defaults to single-repo mode — tools have no `repo` property and `list_repos` is not exposed. Passing `--multi-repo` or `--repos` to the CLI (or `options.multiRepo` / `options.allowedRepos` programmatically) enables multi-repo access. `buildToolList(multiRepo)` builds the tool list dynamically; the backward-compatible `TOOLS` export equals `buildToolList(true)` - **Credential resolution:** `loadConfig` pipeline is `mergeConfig → applyEnvOverrides → resolveSecrets`. The `apiKeyCommand` config field shells out to an external secret manager via `execFileSync` (no shell). Priority: command output > env var > file config > defaults. On failure, warns and falls back gracefully -**Database:** SQLite at `.codegraph/graph.db` with tables: `nodes`, `edges`, `metadata`, `embeddings` +**Database:** SQLite at `.codegraph/graph.db` with tables: `nodes`, `edges`, `metadata`, `embeddings`, `function_complexity` ## Test Structure @@ -118,6 +122,9 @@ node src/cli.js stats # Graph health and quality score node src/cli.js fn -T # Function call chain (callers + callees) node src/cli.js deps src/.js # File-level imports and importers node src/cli.js diff-impact main # Impact of current branch vs main +node src/cli.js complexity -T # Per-function complexity metrics +node src/cli.js communities -T # Community detection & drift analysis +node src/cli.js manifesto -T # Rule engine pass/fail check node src/cli.js cycles # Check for circular dependencies node src/cli.js search "" # Semantic search (requires `embed` first) ``` diff --git a/README.md b/README.md index e7664cc0..c595f9cd 100644 --- a/README.md +++ b/README.md @@ -55,7 +55,7 @@ cd your-project codegraph build ``` -That's it. No config files, no Docker, no JVM, no API keys, no accounts. The graph is ready to query. Add `codegraph mcp` to your AI agent's config and it has full access to your dependency graph through 21 MCP tools (22 in multi-repo mode). +That's it. No config files, no Docker, no JVM, no API keys, no accounts. The graph is ready to query. Add `codegraph mcp` to your AI agent's config and it has full access to your dependency graph through 24 MCP tools (25 in multi-repo mode). ### Why it matters @@ -97,7 +97,7 @@ That's it. No config files, no Docker, no JVM, no API keys, no accounts. The gra | **🔓** | **Zero-cost core, LLM-enhanced when you want** | Full graph analysis with no API keys, no accounts, no cost. Optionally bring your own LLM provider — your code only goes where you choose | | **🔬** | **Function-level, not just files** | Traces `handleAuth()` → `validateToken()` → `decryptJWT()` and shows 14 callers across 9 files break if `decryptJWT` changes | | **🏷️** | **Role classification** | Every symbol auto-tagged as `entry`/`core`/`utility`/`adapter`/`dead`/`leaf` — agents instantly know what they're looking at | -| **🤖** | **Built for AI agents** | 21-tool [MCP server](https://modelcontextprotocol.io/) — AI assistants query your graph directly. Single-repo by default | +| **🤖** | **Built for AI agents** | 24-tool [MCP server](https://modelcontextprotocol.io/) — AI assistants query your graph directly. Single-repo by default | | **🌐** | **Multi-language, one CLI** | JS/TS + Python + Go + Rust + Java + C# + PHP + Ruby + HCL in a single graph | | **💥** | **Git diff impact** | `codegraph diff-impact` shows changed functions, their callers, and full blast radius — enriched with historically coupled files from git co-change analysis. Ships with a GitHub Actions workflow | | **🧠** | **Semantic search** | Local embeddings by default, LLM-powered when opted in — multi-query with RRF ranking via `"auth; token; JWT"` | @@ -144,7 +144,7 @@ After modifying code: Or connect directly via MCP: ```bash -codegraph mcp # 21-tool MCP server — AI queries the graph directly +codegraph mcp # 24-tool MCP server — AI queries the graph directly ``` Full agent setup: [AI Agent Guide](docs/guides/ai-agent-guide.md) · [CLAUDE.md template](docs/guides/ai-agent-guide.md#claudemd-template) @@ -170,8 +170,11 @@ Full agent setup: [AI Agent Guide](docs/guides/ai-agent-guide.md) · [CLAU | 📤 | **Export** | DOT (Graphviz), Mermaid, and JSON graph export | | 🧠 | **Semantic search** | Embeddings-powered natural language search with multi-query RRF ranking | | 👀 | **Watch mode** | Incrementally update the graph as files change | -| 🤖 | **MCP server** | 21-tool MCP server for AI assistants; single-repo by default, opt-in multi-repo | +| 🤖 | **MCP server** | 24-tool MCP server for AI assistants; single-repo by default, opt-in multi-repo | | ⚡ | **Always fresh** | Three-tier incremental detection — sub-second rebuilds even on large codebases | +| 🧮 | **Complexity metrics** | Cognitive, cyclomatic, nesting depth, Halstead, and Maintainability Index per function | +| 🏘️ | **Community detection** | Louvain clustering to discover natural module boundaries and architectural drift | +| 📜 | **Manifesto rule engine** | Configurable pass/fail rules with warn/fail thresholds for CI gates (exit code 1 on fail) | See [docs/examples](docs/examples) for real-world CLI and MCP usage examples. @@ -250,6 +253,20 @@ codegraph hotspots # Files with extreme fan-in, fan-out, or density codegraph hotspots --metric coupling --level directory --no-tests ``` +### Code Health & Architecture + +```bash +codegraph complexity # Per-function cognitive, cyclomatic, nesting, MI +codegraph complexity --health -T # Full Halstead health view (volume, effort, bugs, MI) +codegraph complexity --sort mi -T # Sort by worst maintainability index +codegraph complexity --above-threshold -T # Only functions exceeding warn thresholds +codegraph communities # Louvain community detection — natural module boundaries +codegraph communities --drift -T # Drift analysis only — split/merge candidates +codegraph communities --functions # Function-level community detection +codegraph manifesto # Pass/fail rule engine (exit code 1 on fail) +codegraph manifesto -T # Exclude test files from rule evaluation +``` + ### Export & Visualization ```bash @@ -319,7 +336,7 @@ codegraph registry remove # Unregister | Flag | Description | |---|---| | `-d, --db ` | Custom path to `graph.db` | -| `-T, --no-tests` | Exclude `.test.`, `.spec.`, `__test__` files (available on `fn`, `fn-impact`, `path`, `context`, `explain`, `where`, `diff-impact`, `search`, `map`, `hotspots`, `roles`, `co-change`, `deps`, `impact`) | +| `-T, --no-tests` | Exclude `.test.`, `.spec.`, `__test__` files (available on `fn`, `fn-impact`, `path`, `context`, `explain`, `where`, `diff-impact`, `search`, `map`, `hotspots`, `roles`, `co-change`, `deps`, `impact`, `complexity`, `communities`, `manifesto`) | | `--depth ` | Transitive trace depth (default varies by command) | | `-j, --json` | Output as JSON | | `-v, --verbose` | Enable debug output | @@ -431,7 +448,7 @@ Optional: `@huggingface/transformers` (semantic search), `@modelcontextprotocol/ ### MCP Server -Codegraph includes a built-in [Model Context Protocol](https://modelcontextprotocol.io/) server with 21 tools (22 in multi-repo mode), so AI assistants can query your dependency graph directly: +Codegraph includes a built-in [Model Context Protocol](https://modelcontextprotocol.io/) server with 24 tools (25 in multi-repo mode), so AI assistants can query your dependency graph directly: ```bash codegraph mcp # Single-repo mode (default) — only local project @@ -470,6 +487,9 @@ This project uses codegraph. The database is at `.codegraph/graph.db`. - `codegraph roles --role dead -T` — find dead code (unreferenced symbols) - `codegraph roles --role core -T` — find core symbols (high fan-in) - `codegraph co-change ` — files that historically change together +- `codegraph complexity -T` — per-function complexity metrics (cognitive, cyclomatic, MI) +- `codegraph communities --drift -T` — module boundary drift analysis +- `codegraph manifesto -T` — pass/fail rule check (CI gate, exit code 1 on fail) - `codegraph search ""` — semantic search (requires `codegraph embed`) - `codegraph cycles` — check for circular dependencies @@ -550,6 +570,26 @@ Create a `.codegraphrc.json` in your project root to customize behavior: > **Tip:** `excludeTests` can also be set at the top level as a shorthand — `{ "excludeTests": true }` is equivalent to nesting it under `query`. If both are present, the nested `query.excludeTests` takes precedence. +### Manifesto rules + +Configure pass/fail thresholds for `codegraph manifesto`: + +```json +{ + "manifesto": { + "rules": { + "cognitive_complexity": { "warn": 15, "fail": 30 }, + "cyclomatic_complexity": { "warn": 10, "fail": 20 }, + "nesting_depth": { "warn": 4, "fail": 6 }, + "maintainability_index": { "warn": 40, "fail": 20 }, + "halstead_bugs": { "warn": 0.5, "fail": 1.0 } + } + } +} +``` + +When any function exceeds a `fail` threshold, `codegraph manifesto` exits with code 1 — perfect for CI gates. + ### LLM credentials Codegraph supports an `apiKeyCommand` field for secure credential management. Instead of storing API keys in config files or environment variables, you can shell out to a secret manager at runtime: diff --git a/docs/examples/CLI.md b/docs/examples/CLI.md index f5795a9a..e6eb5c2d 100644 --- a/docs/examples/CLI.md +++ b/docs/examples/CLI.md @@ -758,6 +758,173 @@ Top co-change pairs: --- +## complexity — Per-function complexity metrics + +```bash +codegraph complexity -T --limit 5 +``` + +``` +# Function Complexity + + Function File Cog Cyc Nest MI + ──────────────────────────────────────── ────────────────────────────── ──── ──── ───── ───── + buildGraph src/builder.js 495 185 9 - ! + extractJavaSymbols src/extractors/java.js 208 64 10 13.9 ! + extractSymbolsWalk src/extractors/javascript.js 197 72 11 11.1 ! + walkJavaNode src/extractors/java.js 161 59 9 16 ! + walkJavaScriptNode src/extractors/javascript.js 160 72 10 11.6 ! + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +Full Halstead health view: + +```bash +codegraph complexity --health -T --limit 5 +``` + +``` +# Function Complexity + + Function File MI Vol Diff Effort Bugs LOC SLOC + ─────────────────────────────────── ───────────────────────── ───── ─────── ────── ───────── ────── ───── ───── + buildGraph src/builder.js 0 0 0 0 0 0 0 + extractJavaSymbols src/extractors/java.js 13.9!6673.96 70.52 470637.77 2.2247 225 212 + extractSymbolsWalk …tractors/javascript.js 11.1!7911.66 50.02 395780.68 2.6372 251 239 + walkJavaNode src/extractors/java.js 16!5939.15 65.25 387509.16 1.9797 198 188 + walkJavaScriptNode …tractors/javascript.js 11.6!7624.39 47.67 363429.06 2.5415 240 230 + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +Only functions exceeding warn thresholds: + +```bash +codegraph complexity --above-threshold -T --limit 5 +``` + +``` +# Functions Above Threshold + + Function File Cog Cyc Nest MI + ──────────────────────────────────────── ────────────────────────────── ──── ──── ───── ───── + buildGraph src/builder.js 495 185 9 - ! + extractJavaSymbols src/extractors/java.js 208 64 10 13.9 ! + extractSymbolsWalk src/extractors/javascript.js 197 72 11 11.1 ! + walkJavaNode src/extractors/java.js 161 59 9 16 ! + walkJavaScriptNode src/extractors/javascript.js 160 72 10 11.6 ! + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +Sort by worst maintainability index: + +```bash +codegraph complexity --sort mi -T --limit 5 +``` + +``` +# Function Complexity + + Function File Cog Cyc Nest MI + ──────────────────────────────────────── ────────────────────────────── ──── ──── ───── ───── + median scripts/benchmark.js 1 2 1 - + round1 scripts/benchmark.js 0 1 0 - + selectTargets scripts/benchmark.js 1 2 1 - + benchmarkEngine scripts/benchmark.js 5 5 2 - + benchQuery scripts/benchmark.js 1 2 1 - + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +--- + +## communities — Community detection & drift analysis + +```bash +codegraph communities -T +``` + +``` +# File-Level Communities + + 41 communities | 73 nodes | modularity: 0.4114 | drift: 39% + + Community 34 (16 members): src (16) + - src/cochange.js + - src/communities.js + - src/cycles.js + - src/embedder.js + - src/logger.js + - src/registry.js + - src/structure.js + - src/update-check.js + ... and 8 more + Community 35 (12 members): src/extractors (11), src (1) + - src/extractors/csharp.js + - src/extractors/go.js + - src/extractors/helpers.js + - src/extractors/javascript.js + - src/extractors/php.js + ... and 7 more + Community 33 (6 members): src (6) + - src/builder.js + - src/constants.js + - src/journal.js + - src/native.js + - src/resolve.js + - src/watcher.js +``` + +Drift analysis only: + +```bash +codegraph communities --drift -T +``` + +``` +# File-Level Communities + + 41 communities | 73 nodes | modularity: 0.4114 | drift: 39% + +# Drift Analysis (score: 39%) + + Split candidates (directories spanning multiple communities): + - scripts → 13 communities + - crates/codegraph-core/src/extractors → 11 communities + - crates/codegraph-core/src → 7 communities + - src → 4 communities + - tests/fixtures/sample-project → 3 communities + - (root) → 2 communities + Merge candidates (communities spanning multiple directories): + - Community 35 (12 members) → 2 dirs: src/extractors, src +``` + +--- + +## manifesto — Rule engine pass/fail + +```bash +codegraph manifesto -T +``` + +``` +# Manifesto Results + + Rule Status Threshold Violations + ────────────────────────── ──────── ──────────────── ────────── + cognitive_complexity FAIL warn>15 fail>30 84 functions + cyclomatic_complexity FAIL warn>10 fail>20 42 functions + nesting_depth FAIL warn>4 fail>6 28 functions + maintainability_index FAIL warn<40 fail<20 52 functions + halstead_bugs WARN warn>0.5 fail>1 18 functions + + Result: FAIL (exit code 1) +``` + +--- + ## path — Shortest path between two symbols ```bash diff --git a/docs/examples/MCP.md b/docs/examples/MCP.md index e64ce9ac..c6a586cf 100644 --- a/docs/examples/MCP.md +++ b/docs/examples/MCP.md @@ -732,6 +732,158 @@ Path: buildGraph → isTestFile (2 hops) --- +## complexity — Per-function complexity metrics + +```json +{ + "tool": "complexity", + "arguments": { "no_tests": true, "limit": 5 } +} +``` + +``` +# Function Complexity + + Function File Cog Cyc Nest MI + ──────────────────────────────────────── ────────────────────────────── ──── ──── ───── ───── + buildGraph src/builder.js 495 185 9 - ! + extractJavaSymbols src/extractors/java.js 208 64 10 13.9 ! + extractSymbolsWalk src/extractors/javascript.js 197 72 11 11.1 ! + walkJavaNode src/extractors/java.js 161 59 9 16 ! + walkJavaScriptNode src/extractors/javascript.js 160 72 10 11.6 ! + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +With health view (Halstead metrics): + +```json +{ + "tool": "complexity", + "arguments": { "no_tests": true, "health": true, "limit": 5 } +} +``` + +``` +# Function Complexity + + Function File MI Vol Diff Effort Bugs LOC SLOC + ─────────────────────────────────── ───────────────────────── ───── ─────── ────── ───────── ────── ───── ───── + buildGraph src/builder.js 0 0 0 0 0 0 0 + extractJavaSymbols src/extractors/java.js 13.9!6673.96 70.52 470637.77 2.2247 225 212 + extractSymbolsWalk …tractors/javascript.js 11.1!7911.66 50.02 395780.68 2.6372 251 239 + walkJavaNode src/extractors/java.js 16!5939.15 65.25 387509.16 1.9797 198 188 + walkJavaScriptNode …tractors/javascript.js 11.6!7624.39 47.67 363429.06 2.5415 240 230 + + 339 functions analyzed | avg cognitive: 18.8 | avg cyclomatic: 10.5 | avg MI: 22 | 106 above threshold +``` + +Scope to a specific file: + +```json +{ + "tool": "complexity", + "arguments": { "file": "src/builder.js", "no_tests": true } +} +``` + +--- + +## communities — Community detection & drift analysis + +```json +{ + "tool": "communities", + "arguments": { "no_tests": true } +} +``` + +``` +# File-Level Communities + + 41 communities | 73 nodes | modularity: 0.4114 | drift: 39% + + Community 34 (16 members): src (16) + - src/cochange.js + - src/communities.js + - src/cycles.js + - src/embedder.js + - src/logger.js + - src/registry.js + - src/structure.js + - src/update-check.js + ... and 8 more + Community 35 (12 members): src/extractors (11), src (1) + - src/extractors/csharp.js + - src/extractors/go.js + - src/extractors/helpers.js + - src/extractors/javascript.js + ... and 8 more + Community 33 (6 members): src (6) + - src/builder.js + - src/constants.js + - src/journal.js + - src/native.js + - src/resolve.js + - src/watcher.js +``` + +Drift analysis only: + +```json +{ + "tool": "communities", + "arguments": { "drift": true, "no_tests": true } +} +``` + +``` +# Drift Analysis (score: 39%) + + Split candidates (directories spanning multiple communities): + - scripts → 13 communities + - crates/codegraph-core/src/extractors → 11 communities + - src → 4 communities + Merge candidates (communities spanning multiple directories): + - Community 35 (12 members) → 2 dirs: src/extractors, src +``` + +Function-level community detection: + +```json +{ + "tool": "communities", + "arguments": { "functions": true, "no_tests": true } +} +``` + +--- + +## manifesto — Rule engine pass/fail + +```json +{ + "tool": "manifesto", + "arguments": { "no_tests": true } +} +``` + +``` +# Manifesto Results + + Rule Status Threshold Violations + ────────────────────────── ──────── ──────────────── ────────── + cognitive_complexity FAIL warn>15 fail>30 84 functions + cyclomatic_complexity FAIL warn>10 fail>20 42 functions + nesting_depth FAIL warn>4 fail>6 28 functions + maintainability_index FAIL warn<40 fail<20 52 functions + halstead_bugs WARN warn>0.5 fail>1 18 functions + + Result: FAIL (exit code 1) +``` + +--- + ## list_repos — Multi-repo registry (multi-repo mode only) Only available when the MCP server is started with `--multi-repo`. diff --git a/docs/guides/recommended-practices.md b/docs/guides/recommended-practices.md index 102b0070..a1e21c73 100644 --- a/docs/guides/recommended-practices.md +++ b/docs/guides/recommended-practices.md @@ -110,6 +110,17 @@ Add a threshold check to your CI pipeline: fi ``` +### Code health gate + +Use `manifesto` to enforce code health rules in CI — it exits with code 1 when any function exceeds a fail-level threshold: + +```yaml +- name: Code health gate + run: | + npx codegraph build + npx codegraph manifesto -T # exits 1 on fail-level breach +``` + ### Caching the graph database Speed up CI by caching `.codegraph/`: @@ -143,7 +154,7 @@ By default, the MCP server runs in **single-repo mode** — the AI agent can onl Enable `--multi-repo` to let the agent query any registered repository, or use `--repos` to restrict access to a specific set of repos. -The server exposes 21 tools (22 in multi-repo mode): `query_function`, `file_deps`, `impact_analysis`, `find_cycles`, `module_map`, `fn_deps`, `fn_impact`, `symbol_path`, `context`, `explain`, `where`, `diff_impact`, `semantic_search`, `export_graph`, `list_functions`, `structure`, `hotspots`, `node_roles`, `co_changes`, `execution_flow`, `list_entry_points`, and `list_repos` (multi-repo only). See the [AI Agent Guide MCP reference](./ai-agent-guide.md#mcp-server-reference) for the full tool-to-CLI mapping table. +The server exposes 24 tools (25 in multi-repo mode): `query_function`, `file_deps`, `impact_analysis`, `find_cycles`, `module_map`, `fn_deps`, `fn_impact`, `symbol_path`, `context`, `explain`, `where`, `diff_impact`, `semantic_search`, `export_graph`, `list_functions`, `structure`, `hotspots`, `node_roles`, `co_changes`, `execution_flow`, `list_entry_points`, `complexity`, `communities`, `manifesto`, and `list_repos` (multi-repo only). See the [AI Agent Guide MCP reference](./ai-agent-guide.md#mcp-server-reference) for the full tool-to-CLI mapping table. ### CLAUDE.md for your project @@ -172,6 +183,9 @@ This project uses codegraph. The database is at `.codegraph/graph.db`. - `codegraph roles --role dead -T` — find dead code (unreferenced symbols) - `codegraph roles --role core -T` — find core symbols (high fan-in) - `codegraph co-change ` — files that historically change together +- `codegraph complexity -T` — per-function complexity metrics (cognitive, cyclomatic, MI) +- `codegraph communities --drift -T` — module boundary drift analysis +- `codegraph manifesto -T` — pass/fail rule check (CI gate, exit code 1 on fail) - `codegraph search ""` — semantic search (requires `codegraph embed`) - `codegraph cycles` — check for circular dependencies @@ -292,6 +306,12 @@ codegraph fn-impact myFunction --no-tests # what breaks if this changes codegraph path myFunction otherFunction -T # how two symbols are connected ``` +Check complexity before refactoring: + +```bash +codegraph complexity --file src/utils/auth.ts -T # complexity metrics for functions in a file +``` + Before touching a file: ```bash @@ -549,9 +569,12 @@ cp node_modules/@optave/codegraph/.github/workflows/codegraph-impact.yml .github # 6. (Optional) Scan git history for co-change coupling codegraph co-change --analyze -# 7. (Optional) Build embeddings for semantic search +# 7. (Optional) Verify code health rules pass +codegraph manifesto -T + +# 8. (Optional) Build embeddings for semantic search codegraph embed -# 8. (Optional) Add CLAUDE.md for AI agents +# 9. (Optional) Add CLAUDE.md for AI agents # See docs/guides/ai-agent-guide.md for the full template ``` diff --git a/docs/roadmap/BACKLOG.md b/docs/roadmap/BACKLOG.md index e2d8e418..d4bc2808 100644 --- a/docs/roadmap/BACKLOG.md +++ b/docs/roadmap/BACKLOG.md @@ -1,6 +1,6 @@ # Codegraph Feature Backlog -**Last updated:** 2026-02-25 +**Last updated:** 2026-02-26 **Source:** Features derived from [COMPETITIVE_ANALYSIS.md](../../generated/COMPETITIVE_ANALYSIS.md) and internal roadmap discussions. --- @@ -42,12 +42,12 @@ Non-breaking, ordered by problem-fit: | 13 | Architecture boundary rules | User-defined rules for allowed/forbidden dependencies between modules (e.g., "controllers must not import from other controllers"). Violations flagged in `diff-impact` and CI. Inspired by codegraph-rust, stratify. | Architecture | Prevents architectural decay in CI; agents are warned before introducing forbidden cross-module dependencies | ✓ | ✓ | 3 | No | | 15 | Hybrid BM25 + semantic search | Combine BM25 keyword matching with embedding-based semantic search using Reciprocal Rank Fusion. Better recall than either approach alone. Inspired by GitNexus, claude-context-local. | Search | Search results improve dramatically — keyword matches catch exact names, embeddings catch conceptual matches, RRF merges both | ✓ | ✓ | 3 | No | | 18 | CODEOWNERS integration | Map graph nodes to CODEOWNERS entries. Show who owns each function, surface ownership boundaries in `diff-impact`. Inspired by CKB. | Developer Experience | `diff-impact` tells agents which teams to notify; ownership-aware impact analysis reduces missed reviews | ✓ | ✓ | 3 | No | -| 22 | Manifesto-driven pass/fail | User-defined rule engine with custom thresholds (e.g. "cognitive > 15 = fail", "cyclomatic > 10 = fail", "imports > 10 = decompose"). Outputs pass/fail per function/file. Generalizes ID 13 (boundary rules) into a generic rule system. | Analysis | Enables autonomous multi-agent audit workflows (GAUNTLET pattern); CI integration for code health gates with configurable thresholds | ✓ | ✓ | 3 | No | +| 22 | ~~Manifesto-driven pass/fail~~ | ~~User-defined rule engine with custom thresholds (e.g. "cognitive > 15 = fail", "cyclomatic > 10 = fail", "imports > 10 = decompose"). Outputs pass/fail per function/file. Generalizes ID 13 (boundary rules) into a generic rule system.~~ | Analysis | ~~Enables autonomous multi-agent audit workflows (GAUNTLET pattern); CI integration for code health gates with configurable thresholds~~ | ✓ | ✓ | 3 | No | **DONE** — `codegraph manifesto` with 9 configurable rules (cognitive, cyclomatic, nesting, MI, Halstead volume/effort/bugs, fan-in, fan-out). Warn/fail thresholds via `.codegraphrc.json`. Exit code 1 on any fail-level breach — CI gate ready. PR #138. | | 31 | Graph snapshots | `codegraph snapshot save ` / `codegraph snapshot restore ` for lightweight SQLite DB backup and restore. Enables orchestrators to checkpoint before refactoring passes and instantly rollback without rebuilding. After Phase 4, also preserves embeddings and semantic metadata. Inspired by [Titan Paradigm](../docs/use-cases/titan-paradigm.md) STATE MACHINE phase. | Orchestration | Multi-agent workflows get instant rollback without re-running expensive builds or LLM calls — orchestrator checkpoints before each pass and restores on failure | ✓ | ✓ | 3 | No | -| 6 | Formal code health metrics | Cyclomatic complexity, Maintainability Index, and Halstead metrics per function — we already parse the AST, the data is there. Inspired by code-health-meter (published in ACM TOSEM 2025). | Analysis | Agents can prioritize refactoring targets; `hotspots` becomes richer with quantitative health scores per function | ✓ | ✓ | 2 | No | +| 6 | ~~Formal code health metrics~~ | ~~Cyclomatic complexity, Maintainability Index, and Halstead metrics per function — we already parse the AST, the data is there. Inspired by code-health-meter (published in ACM TOSEM 2025).~~ | Analysis | ~~Agents can prioritize refactoring targets; `hotspots` becomes richer with quantitative health scores per function~~ | ✓ | ✓ | 2 | No | **DONE** — `codegraph complexity` provides cognitive, cyclomatic, nesting depth, Halstead (volume, difficulty, effort, bugs), and Maintainability Index per function. `--health` for full Halstead view, `--sort mi` to rank by MI, `--above-threshold` for flagged functions. `function_complexity` DB table. `complexity` MCP tool. PR #130 + #139. | | 7 | OWASP/CWE pattern detection | Security pattern scanning on the existing AST — hardcoded secrets, SQL injection patterns, eval usage, XSS sinks. Lightweight static rules, not full taint analysis. Inspired by narsil-mcp, CKB. | Security | Catches low-hanging security issues during `diff-impact`; agents can flag risky patterns before they're committed | ✓ | ✓ | 2 | No | -| 11 | Community detection | Leiden/Louvain algorithm to discover natural module boundaries vs actual file organization. Reveals which symbols are tightly coupled and whether the directory structure matches. Inspired by axon, GitNexus, CodeGraphMCPServer. | Intelligence | Surfaces architectural drift — when directory structure no longer matches actual dependency clusters; guides refactoring | ✓ | ✓ | 2 | No | -| 21 | Cognitive + cyclomatic complexity | Cognitive Complexity (SonarSource) as the primary readability metric — penalizes nesting, so it subsumes nesting depth analysis. Cyclomatic complexity (McCabe) as secondary testability metric. Both computed from existing tree-sitter AST in a single traversal. Cognitive > 15 or cyclomatic > 10 = flag for refactoring. Extends ID 6 with the two most actionable metrics. | Analysis | Agents can flag hard-to-understand and hard-to-test functions in one pass; cognitive complexity captures both decision complexity and nesting depth in a single score | ✓ | ✓ | 2 | No | +| 11 | ~~Community detection~~ | ~~Leiden/Louvain algorithm to discover natural module boundaries vs actual file organization. Reveals which symbols are tightly coupled and whether the directory structure matches. Inspired by axon, GitNexus, CodeGraphMCPServer.~~ | Intelligence | ~~Surfaces architectural drift — when directory structure no longer matches actual dependency clusters; guides refactoring~~ | ✓ | ✓ | 2 | No | **DONE** — `codegraph communities` with Louvain algorithm. File-level and `--functions` function-level detection. `--drift` for drift analysis (split/merge candidates). `--resolution` tunable. `communities` MCP tool. PR #133/#134. | +| 21 | ~~Cognitive + cyclomatic complexity~~ | ~~Cognitive Complexity (SonarSource) as the primary readability metric — penalizes nesting, so it subsumes nesting depth analysis. Cyclomatic complexity (McCabe) as secondary testability metric. Both computed from existing tree-sitter AST in a single traversal. Cognitive > 15 or cyclomatic > 10 = flag for refactoring. Extends ID 6 with the two most actionable metrics.~~ | Analysis | ~~Agents can flag hard-to-understand and hard-to-test functions in one pass; cognitive complexity captures both decision complexity and nesting depth in a single score~~ | ✓ | ✓ | 2 | No | **DONE** — Subsumed by ID 6. `codegraph complexity` provides both cognitive and cyclomatic metrics (plus nesting depth, Halstead, MI) in a single command. PR #130. | Breaking (penalized to end of tier): diff --git a/docs/use-cases/titan-paradigm.md b/docs/use-cases/titan-paradigm.md index 11f10e15..18aaea46 100644 --- a/docs/use-cases/titan-paradigm.md +++ b/docs/use-cases/titan-paradigm.md @@ -47,7 +47,17 @@ codegraph hotspots --no-tests codegraph stats ``` -An orchestrating agent can use `map` and `hotspots` to build a priority queue: audit the most-connected files first, because changes there have the highest blast radius. The `--json` flag on every command makes it trivial to feed results into a state file or orchestration script. +Use `communities` to discover natural module boundaries and identify architectural drift — where the directory structure no longer matches actual dependency clusters: + +```bash +# Discover natural module boundaries via Louvain clustering +codegraph communities -T + +# Drift analysis: which directories should be split or merged? +codegraph communities --drift -T +``` + +An orchestrating agent can use `map`, `hotspots`, and `communities` to build a priority queue: audit the most-connected files first, because changes there have the highest blast radius. The `--json` flag on every command makes it trivial to feed results into a state file or orchestration script. ```bash # JSON output for programmatic consumption @@ -89,6 +99,19 @@ codegraph fn-impact wasmExtractSymbols -T codegraph fn wasmExtractSymbols -T --depth 5 ``` +Use `complexity` to get quantitative metrics for every function in the file, and `manifesto` to run the full rule engine: + +```bash +# 5. Per-function complexity metrics — cognitive, cyclomatic, nesting, MI +codegraph complexity --file src/parser.js -T + +# 6. Full Halstead health view — volume, effort, estimated bugs, MI +codegraph complexity --file src/parser.js --health -T + +# 7. Pass/fail rule check — does this file meet the manifesto? +codegraph manifesto -T +``` + When a sub-agent decides a function needs decomposition (complexity > 7, nesting > 3, 10+ mocks), it needs to know what breaks. `fn-impact` gives the complete blast radius **before** the agent writes a single line of code. The `--json` flag lets the orchestrator aggregate results across all sub-agents: @@ -144,7 +167,14 @@ codegraph diff-impact --staged --format mermaid -T codegraph diff-impact --staged -T --json > state/impact-check.json ``` -The orchestrator can gate every commit: run `diff-impact --staged --json`, check that the blast radius is within acceptable bounds, and auto-rollback if it exceeds thresholds. Combined with `codegraph watch` for real-time graph updates, the state machine always has a current picture of the codebase. +Use `manifesto` as a CI gate — it exits with code 1 when any function exceeds a fail-level threshold: + +```bash +# Pass/fail rule check — exit code 1 = fail → rollback trigger +codegraph manifesto -T +``` + +The orchestrator can gate every commit: run `diff-impact --staged --json` to check blast radius, and `manifesto -T` to verify code health rules. Auto-rollback if either exceeds thresholds. Combined with `codegraph watch` for real-time graph updates, the state machine always has a current picture of the codebase. ```bash # Watch mode — graph updates automatically as agents edit files @@ -171,9 +201,10 @@ Several planned features would make codegraph even more powerful for the Titan P | Feature | Status | How it helps | |---------|--------|-------------| -| **Formal code health metrics** ([Backlog #6](../../roadmap/BACKLOG.md)) | Planned | Cyclomatic complexity, Maintainability Index, and Halstead metrics per function — directly maps to the Gauntlet's "complexity > 7 is a failure" rule. Computed from the AST we already parse | +| **Formal code health metrics** ([Backlog #6](../../roadmap/BACKLOG.md)) | **Done** | `codegraph complexity` provides cognitive, cyclomatic, nesting depth, Halstead (volume, effort, bugs), and Maintainability Index per function. `--health` for full view, `--sort mi` to rank by MI, `--above-threshold` for flagged functions. Maps directly to the Gauntlet's "complexity > 7 is a failure" rule. PR #130 + #139 | +| **Manifesto-driven pass/fail** ([Backlog #22](../../roadmap/BACKLOG.md)) | **Done** | `codegraph manifesto` with 9 configurable rules and warn/fail thresholds. Exit code 1 on fail — the Gauntlet gets first-class pass/fail signals without parsing JSON. PR #138 | +| **Community detection** ([Backlog #11](../../roadmap/BACKLOG.md)) | **Done** | `codegraph communities` with Louvain algorithm discovers natural module boundaries vs actual file organization. `--drift` reveals which directories should be split or merged. `--functions` for function-level clustering. PR #133/#134 | | **Build-time semantic metadata** ([Roadmap Phase 4.4](../../roadmap/ROADMAP.md#44--build-time-semantic-metadata)) | Planned | LLM-generated `complexity_notes`, `risk_score`, and `side_effects` per function. A sub-agent could query `codegraph assess ` and get "3 responsibilities, low cohesion — consider splitting" without analyzing the code itself | -| **Community detection** ([Backlog #11](../../roadmap/BACKLOG.md)) | Planned | Leiden/Louvain algorithm to discover natural module boundaries vs actual file organization. Reveals which functions are tightly coupled and whether decomposition should follow the directory structure or propose a new one | ### For GLOBAL SYNC @@ -259,7 +290,7 @@ After LLM integration, snapshots would also preserve embeddings, descriptions, a ### 6. MCP-native orchestration -The Titan Paradigm's agents could run entirely through codegraph's [MCP server](../examples/MCP.md) instead of shelling out to the CLI. With 21 tools already exposed, the main gap is the `audit`/`triage`/`check` commands described above. After Phase 4, adding these as MCP tools — alongside [`ask_codebase`](../../roadmap/ROADMAP.md#53--mcp-integration) (Phase 5.3) for natural-language queries — would let orchestrators like Claude Code's agent teams query the graph with zero CLI overhead. The RECON agent asks the MCP server "what are the riskiest files?", each Gauntlet agent asks "should this function be decomposed?", and the STATE MACHINE asks "is this change safe?" — all through the same protocol. +The Titan Paradigm's agents could run entirely through codegraph's [MCP server](../examples/MCP.md) instead of shelling out to the CLI. With 24 tools already exposed, the main gap is the `audit`/`triage`/`check` commands described above. After Phase 4, adding these as MCP tools — alongside [`ask_codebase`](../../roadmap/ROADMAP.md#53--mcp-integration) (Phase 5.3) for natural-language queries — would let orchestrators like Claude Code's agent teams query the graph with zero CLI overhead. The RECON agent asks the MCP server "what are the riskiest files?", each Gauntlet agent asks "should this function be decomposed?", and the STATE MACHINE asks "is this change safe?" — all through the same protocol. --- diff --git a/generated/DOGFOOD_REPORT_v2.4.0.md b/generated/DOGFOOD_REPORT_v2.4.0.md index 294c1477..0ccc1533 100644 --- a/generated/DOGFOOD_REPORT_v2.4.0.md +++ b/generated/DOGFOOD_REPORT_v2.4.0.md @@ -116,8 +116,8 @@ codegraph build --engine native --no-incremental --verbose | `watch .` | PASS | Starts, detects changes, graceful Ctrl+C | | `registry list` | PASS | JSON and text output | | `registry prune --ttl 365` | PASS | "No stale entries" | -| `mcp` (single-repo) | PASS | 18 tools, `list_repos` absent | -| `mcp --multi-repo` | PASS | 19 tools, `list_repos` present | +| `mcp` (single-repo) | PASS | 23 tools, `list_repos` absent (was 18 at time of report; now includes complexity, communities, execution_flow, list_entry_points, co_changes) | +| `mcp --multi-repo` | PASS | 24 tools, `list_repos` present | ### Edge Cases Tested @@ -277,9 +277,9 @@ Appended `// test comment` to `src/logger.js`: | Test | Result | |------|--------| | Single-repo initialization (JSON-RPC) | PASS — Valid response | -| `tools/list` (single-repo) | PASS — 18 tools, `list_repos` absent | -| `tools/list` (multi-repo) | PASS — 19 tools, `list_repos` present | -| New tools: `node_roles`, `co_changes` | Present in tool list | +| `tools/list` (single-repo) | PASS — 23 tools, `list_repos` absent (updated from 18 after complexity, communities, execution_flow, list_entry_points, co_changes added) | +| `tools/list` (multi-repo) | PASS — 24 tools, `list_repos` present | +| New tools: `node_roles`, `co_changes`, `complexity`, `communities`, `execution_flow`, `list_entry_points` | Present in tool list | ### Config & Environment @@ -322,27 +322,27 @@ Appended `// test comment` to `src/logger.js`: - **Root cause:** `loadModel()` used `console.log()` instead of the stderr-routed `info()` logger. - **Fix applied:** Replaced `console.log` with `info()` for both messages. -### BUG 4: Incremental rebuild drops edges (High) -- **Issue:** [#116](https://github.com/optave/codegraph/issues/116) -- **PR:** Open — too complex for this session +### BUG 4: ~~Incremental rebuild drops edges~~ — FIXED +- **Issue:** [#116](https://github.com/optave/codegraph/issues/116) — Closed +- **PR:** Fixed via reverse-dependency cascade in `builder.js:444` - **Symptoms:** Touching one file (appending a comment) and running incremental build drops 46 edges (33 calls, 12 imports, 1 reexport). Full rebuild restores them. -- **Root cause:** The incremental edge deletion query (`DELETE FROM edges WHERE source_id IN (changed file) OR target_id IN (changed file)`) removes ALL edges touching the changed file — including incoming edges from other files. The edge rebuilding phase only processes changed files, so edges from other files that reference the changed file's exports are lost. -- **Fix applied:** None yet. Requires rethinking the incremental rebuild's edge cascade strategy. +- **Root cause:** The incremental edge deletion query removed ALL edges touching the changed file — including incoming edges from other files. The edge rebuilding phase only processed changed files. +- **Fix applied:** Reverse-dependency cascade — when a file changes, files that import it are identified and their outgoing edges are re-resolved. Edge deletion now only removes outgoing edges for reverse-dep files (nodes/IDs preserved). --- ## 9. Suggestions for Improvement -### 9.1 Add `--db` flag to `embed` command -Currently `embed` doesn't support `--db`, unlike all other commands. This makes it harder to test against specific databases. +### 9.1 ~~Add `--db` flag to `embed` command~~ — DONE +~~Currently `embed` doesn't support `--db`, unlike all other commands.~~ Fixed: `embed` now supports `-d, --db `. -### 9.2 Redirect HuggingFace library console output -The `@huggingface/transformers` library prints "dtype not specified..." via `console.warn` which goes to stderr, but the library may have other `console.log` calls that could leak to stdout. Consider monkey-patching `console.log` during model loading or using the library's logging configuration. +### 9.2 ~~Redirect HuggingFace library console output~~ — DONE +~~The `@huggingface/transformers` library prints "dtype not specified..." via `console.warn` which goes to stderr.~~ Fixed in PR #117: `loadModel()` messages switched from `console.log` to `info()` (stderr-routed logger). The HF library's own `console.warn` goes to stderr naturally and doesn't affect stdout pipe consumers. -### 9.3 Incremental rebuild needs reverse-dep edge cascade -The most impactful fix would be making incremental rebuilds re-resolve edges for files that import changed files (reverse dependency cascade). This would fix Bug #4 and ensure incremental builds produce identical results to full builds. +### 9.3 ~~Incremental rebuild needs reverse-dep edge cascade~~ — DONE +~~The most impactful fix would be making incremental rebuilds re-resolve edges for files that import changed files.~~ Implemented at `builder.js:444` — reverse-dependency cascade detects files that import changed files and re-resolves their outgoing edges, fixing Bug #4. -### 9.4 Update notification testing +### 9.4 Update notification testing — Open (low priority) The update notification feature was not observable during testing. Consider adding a `--check-update` flag for manual testing, or document when the notification appears. --- @@ -402,16 +402,18 @@ The update notification feature was not observable during testing. Consider addi Codegraph v2.4.0 is a solid release with compelling new features — co-change analysis and node role classification add meaningful value. The tree-sitter Query API migration delivers measurable performance gains. Engine parity remains perfect. -**However**, two high-severity bugs were found: +**All 4 bugs found during dogfooding have been fixed:** -1. Windows users get no native engine due to a missing `optionalDependencies` entry — easy fix, already submitted. -2. Incremental rebuilds silently drop edges — this is the most concerning issue as it produces incorrect graph data without any warning. Users relying on incremental builds may have missing edges. +1. ~~Windows users get no native engine~~ — Fixed in PR #117 (added win32 to optionalDependencies) +2. ~~extractLeadingComment crashes~~ — Fixed in PR #117 (bounds check) +3. ~~search --json leaks to stdout~~ — Fixed in PR #117 (switched to stderr logger) +4. ~~Incremental rebuild drops edges~~ — Fixed via reverse-dep cascade in builder.js -The 3 simpler bugs are fixed in PR #117. Bug #4 (incremental edge drop) needs deeper work. +3 of 4 improvement suggestions also addressed (9.1 `--db` on embed, 9.2 HF output, 9.3 reverse-dep cascade). Only 9.4 (update notification testing) remains open as low priority. -**Rating: 7/10** +**Rating: 7/10 → 9/10 (post-fix)** -Justification: All commands work correctly, engine parity is perfect, and the new features (co-change, roles) work well. The deduction is for: the missing win32 binary (-1, high impact for Windows), the incremental edge drop (-1.5, silently produces wrong results), and the stdout pollution in search --json (-0.5, minor but affects tooling integration). +Original deductions were: missing win32 binary (-1), incremental edge drop (-1.5), stdout pollution (-0.5). All three are now fixed. Remaining -1: update notification untested, and the HF library's `console.warn` to stderr is cosmetic but not fully silenced. --- @@ -422,5 +424,5 @@ Justification: All commands work correctly, engine parity is perfect, and the ne | Issue | [#113](https://github.com/optave/codegraph/issues/113) | bug: @optave/codegraph-win32-x64-msvc missing from optionalDependencies | Closed via PR #117 | | Issue | [#114](https://github.com/optave/codegraph/issues/114) | bug(embedder): extractLeadingComment crashes on out-of-bounds line access | Closed via PR #117 | | Issue | [#115](https://github.com/optave/codegraph/issues/115) | bug(embedder): search --json leaks model loading messages to stdout | Closed via PR #117 | -| Issue | [#116](https://github.com/optave/codegraph/issues/116) | bug(builder): incremental rebuild drops edges when re-parsing a file | Open | -| PR | [#117](https://github.com/optave/codegraph/pull/117) | fix: dogfood v2.4.0 — win32 native binary, embedder crashes, stdout pollution | Open | +| Issue | [#116](https://github.com/optave/codegraph/issues/116) | bug(builder): incremental rebuild drops edges when re-parsing a file | Closed — fixed via reverse-dep cascade | +| PR | [#117](https://github.com/optave/codegraph/pull/117) | fix: dogfood v2.4.0 — win32 native binary, embedder crashes, stdout pollution | Merged | diff --git a/src/paginate.js b/src/paginate.js index 522a60bb..7109f0bc 100644 --- a/src/paginate.js +++ b/src/paginate.js @@ -28,7 +28,7 @@ export const MCP_MAX_LIMIT = 1000; * @returns {{ items: any[], pagination?: { total: number, offset: number, limit: number, hasMore: boolean, returned: number } }} */ export function paginate(items, { limit, offset } = {}) { - if (limit === undefined && limit !== 0) { + if (limit === undefined) { return { items }; } const total = items.length; @@ -59,7 +59,7 @@ export function paginate(items, { limit, offset } = {}) { * @returns {object} - Result with paginated field + `_pagination` (if active) */ export function paginateResult(result, field, { limit, offset } = {}) { - if (limit === undefined && limit !== 0) { + if (limit === undefined) { return result; } const arr = result[field]; diff --git a/tests/integration/complexity.test.js b/tests/integration/complexity.test.js index 850cf019..5ec66107 100644 --- a/tests/integration/complexity.test.js +++ b/tests/integration/complexity.test.js @@ -242,6 +242,14 @@ describe('complexityData', () => { expect(data.functions.length).toBe(0); }); + test('produces correct exceeds and aboveWarn with valid thresholds', () => { + const data = complexityData(dbPath); + expect(data.summary.aboveWarn).toBeGreaterThan(0); + const handleReq = data.functions.find((f) => f.name === 'handleRequest'); + expect(handleReq.exceeds).toBeDefined(); + expect(handleReq.exceeds.length).toBeGreaterThan(0); + }); + // ─── Halstead / MI Tests ───────────────────────────────────────────── test('functions include halstead and MI data', () => {