From 1be1df88668c8aa3811d979b0c7117bec4e244bb Mon Sep 17 00:00:00 2001 From: Kirill Turanskiy Date: Sat, 28 Mar 2026 04:59:43 +0300 Subject: [PATCH 1/5] =?UTF-8?q?chore:=20mcp-tool-api-consolidation=20spec?= =?UTF-8?q?=20(61=E2=86=927=20tools)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Full SpecKit pipeline: specify → clarify → plan → tasks → analyze. Consolidates 61 MCP tools into 6 primary (recall/store/feedback/vault/docs/admin) + check_system_health. Backward-compatible dispatch aliases for all old names. Target: >80% context window reduction (~6100 → ~900 tokens). Also: 3 new dashboard bugs recorded in inbox. --- .../specs/mcp-tool-api-consolidation/plan.md | 147 ++++++++++ .../specs/mcp-tool-api-consolidation/spec.md | 270 ++++++++++++++++++ .../specs/mcp-tool-api-consolidation/tasks.md | 79 +++++ .agent/tasks/inbox.md | 19 ++ 4 files changed, 515 insertions(+) create mode 100644 .agent/specs/mcp-tool-api-consolidation/plan.md create mode 100644 .agent/specs/mcp-tool-api-consolidation/spec.md create mode 100644 .agent/specs/mcp-tool-api-consolidation/tasks.md create mode 100644 .agent/tasks/inbox.md diff --git a/.agent/specs/mcp-tool-api-consolidation/plan.md b/.agent/specs/mcp-tool-api-consolidation/plan.md new file mode 100644 index 00000000..5a64c9d2 --- /dev/null +++ b/.agent/specs/mcp-tool-api-consolidation/plan.md @@ -0,0 +1,147 @@ +# Implementation Plan: MCP Tool API Consolidation (61 → 6+1) + +**Spec:** .agent/specs/mcp-tool-api-consolidation/spec.md +**Created:** 2026-03-28 +**Status:** Draft + +## Tech Stack + +No new dependencies. Pure refactoring of `internal/mcp/server.go`. + +| Component | Choice | Rationale | +|-----------|--------|-----------| +| Tool dispatch | Go switch statement | Existing pattern, zero overhead | +| Schema definition | `map[string]any` literals | Existing pattern in server.go | +| Backward compat | Dual-path dispatch | Old names → extract action → call primary handler | + +## Architecture + +``` +tools/list (default) tools/list (cursor=all) + │ │ + ▼ ▼ + 7 primary tools 7 primary + 61 alias Tool objects + │ │ + └─────────┬───────────────┘ + │ + callTool(name, args) + │ + ┌────────┴─────────┐ + │ Primary name? │ Alias name? + │ (recall, store, │ (search, find_by_file, + │ feedback, ...) │ store_memory, ...) + ▼ ▼ + handleRecall(args) inject action param + handleStore(args) ────────────────────► handleRecall(args') + handleFeedback(args) handleStore(args') + handleVault(args) ... + handleDocs(args) + handleAdmin(args) +``` + +Each primary handler: reads `action` from args → switch → delegates to existing handler function. +Alias dispatch: sets `action` in args map → calls primary handler. + +## File Structure + +``` +internal/mcp/ + server.go — modified: primary tool registrations + dispatch + tools_recall.go — NEW: handleRecall (routes to existing search handlers) + tools_store_consolidated.go — NEW: handleStore (routes to existing store handlers) + tools_feedback.go — NEW: handleFeedback (routes to existing feedback handlers) + tools_vault_consolidated.go — NEW: handleVault (routes to existing vault handlers) + tools_docs_consolidated.go — NEW: handleDocs (routes to existing doc handlers) + tools_admin.go — NEW: handleAdmin (routes to existing admin handlers) + server_test.go — modified: test expectations for 7 primary tools + tools_memory.go — unchanged (handler implementations stay) + tools_search.go — unchanged + tools_observations.go — unchanged + ... — all other tools_*.go unchanged +``` + +## Phases + +### Phase 1: Create 6 Primary Handler Files (FR-1 through FR-6) + +For each primary tool, create a `tools_.go` file containing: +1. A handler function: `handleRecall(ctx, args) (string, error)` +2. An `action` switch that routes to existing handler functions +3. Validation: unknown action → descriptive error listing valid actions + +**Order:** recall → store → feedback → vault → docs → admin (largest first, validates pattern) + +Each handler is a thin routing layer — NO new business logic. Example: + +```go +func (s *Server) handleRecall(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + action := coerceString(m["action"], "search") // default: search + switch action { + case "search": + return s.handleSearch(ctx, args) + case "preset": + // inject preset into args, call handleSearch + case "by_file": + return s.handleFindByFile(ctx, args) + // ... etc + default: + return "", fmt.Errorf("unknown recall action: %q (valid: search, preset, by_file, ...)", action) + } +} +``` + +### Phase 2: Register Primary Tools in server.go (FR-8) + +Replace the 61-entry `tools` slice with 7 entries: +- `recall`, `store`, `feedback`, `vault`, `docs`, `admin`, `check_system_health` +- Each with flat schema: `action` enum + all params optional +- Keep the old 61-entry slice as `aliasTools` for `cursor=all` + +### Phase 3: Update Dispatch Switch (FR-7) + +In `callTool`, replace the current 80+ case switch with: +1. Primary tool names → call primary handler directly +2. Alias names → inject `action` param → call primary handler +3. Legacy aliases (doc_update, get_recent_context, etc.) → same routing + +### Phase 4: Update Tests + +- Update `TestHandleToolsList` to expect 7 tools +- Update `TestCallTool_ToolNameRecognition` to test primary + alias +- Add tests: each primary tool with each action +- Add tests: alias dispatch produces same results as primary call + +### Phase 5: Version Bump + Release + +- Bump to v2.1.0 (MINOR — new API surface, no breaking changes) +- Update openclaw-engram to 2.1.0 +- Update Constitution #12 rationale with new tool count + +## Library Decisions + +| Component | Library | Rationale | +|-----------|---------|-----------| +| All | Custom (Go stdlib) | Pure routing logic, no external deps needed | + +## Unknowns and Risks + +| Unknown | Impact | Resolution | +|---------|--------|------------| +| Schema token count for 6 primary tools | MED | Measure after Phase 2 — each tool ~150-200 tokens with flat enum schema | +| Some handlers accept args differently (parseArgs vs json.Unmarshal) | LOW | Survey in Phase 1, normalize in handler | +| `search` handler has special preset logic | LOW | Recall handler delegates directly, preset param already supported | + +## Constitution Compliance + +| Principle | Compliance | +|-----------|-----------| +| #1 Server-Only | OK — server-side only refactoring | +| #3 Non-Blocking Hooks | N/A — no hook changes | +| #7 Bump Plugin Version | OK — openclaw bumped to match | +| #8 Complete Implementations | OK — every action fully implemented via existing handlers | +| #12 Tool Count Budget | **DIRECTLY IMPLEMENTS** — 61 → 7 tools | +| #15 Version Tracking | OK — unified bump to 2.1.0 | diff --git a/.agent/specs/mcp-tool-api-consolidation/spec.md b/.agent/specs/mcp-tool-api-consolidation/spec.md new file mode 100644 index 00000000..48e8e707 --- /dev/null +++ b/.agent/specs/mcp-tool-api-consolidation/spec.md @@ -0,0 +1,270 @@ +# Feature: MCP Tool API Consolidation (61 → 6+1) + +**Slug:** mcp-tool-api-consolidation +**Created:** 2026-03-28 +**Status:** Draft +**Author:** AI Agent (reviewed by user) +**Predecessor:** plugin-tool-consolidation (cleanup phase — removed 7 duplicates, expanded OpenClaw) + +## Overview + +Consolidate engram's 61 MCP tools into 6 primary tools plus `check_system_health`. Each +primary tool absorbs multiple current tools as parameter modes/actions. Backward-compatible +dispatch aliases ensure existing clients continue to work unchanged. This directly implements +Constitution Principle #12 (Tool Count Is a Budget): reducing ~6100 context tokens per session +to ~700 tokens (6 tools × ~100 tokens each). + +## Context + +### Problem +Agents use 2 of 61 tools (store_memory, search). Root cause: 17 search variants, 12 doc tools, +5 vault tools are separate tools instead of parameters to unified operations. The tool list +overwhelms agent context windows, hiding useful capabilities behind a wall of similar-sounding names. + +### Current State (v2.0.9) +- 61 registered MCP tools in 3 tiers: Core (9), Useful (14), Admin (38) +- 7 additional dispatch aliases for removed tools (from plugin-tool-consolidation PR #112) +- Each tool adds ~100 tokens to every MCP `tools/list` response +- Agents default to store_memory + search because they're the most obviously named + +### Classification (from audit 2026-03-28) + +| Primary Tool | Absorbs | Count | +|-------------|---------|-------| +| `recall` | search, decisions, changes, how_it_works, find_by_file, find_by_concept, find_by_type, find_similar_observations, recall_memory, search_sessions, explain_search_ranking, timeline, find_related_observations, get_observation, get_patterns, list_sessions | 16 | +| `store` | store_memory, edit_observation, merge_observations, import_instincts | 4 | +| `feedback` | rate_memory, suppress_memory, set_session_outcome | 3 | +| `vault` | store_credential, get_credential, list_credentials, delete_credential, vault_status | 5 | +| `docs` | doc_create, doc_read, doc_list, doc_history, doc_comment, list_collections, list_documents, get_document, remove_document, ingest_document, search_collection | 11 | +| `admin` | bulk_delete_observations, bulk_mark_superseded, bulk_boost_observations, tag_observation, get_observations_by_tag, batch_tag_by_pattern, graph_query, get_graph_stats, get_memory_stats, get_temporal_trends, get_data_quality_report, analyze_observation_importance, analyze_search_patterns, get_observation_quality, get_observation_scoring_breakdown, suggest_consolidations, trigger_maintenance, get_maintenance_stats, run_consolidation, export_observations, backfill_status | 21 | +| `check_system_health` | (stays as-is) | 1 | +| **Total** | | **61** | + +## Functional Requirements + +### FR-1: Create `recall` Tool (absorbs 16 tools) +The system must provide a single `recall` tool that supports all search/retrieval operations +via a required `action` parameter: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `search` | search | query, project, limit | +| `preset` | decisions, changes, how_it_works | query, preset (decisions/changes/how_it_works) | +| `by_file` | find_by_file | files, project | +| `by_concept` | find_by_concept | concept, project | +| `by_type` | find_by_type | type, project | +| `similar` | find_similar_observations | query, min_similarity | +| `timeline` | timeline | mode (recent/anchor/query), anchor_id, query | +| `related` | find_related_observations | id, min_confidence | +| `patterns` | get_patterns | project, type | +| `get` | get_observation | id | +| `sessions` | search_sessions, list_sessions | query (search) or omit (list) | +| `explain` | explain_search_ranking | query, project | + +Default action: `search` (when action omitted, treat as `search`). +`recall_memory` alias maps to action `search` with format param. + +### FR-2: Create `store` Tool (absorbs 4 tools) +The system must provide a single `store` tool with actions: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `create` | store_memory | content, title, type, tags, scope, always_inject, ttl_days | +| `edit` | edit_observation | id, title, narrative, facts, concepts, status, always_inject | +| `merge` | merge_observations | source_id, target_id, boost | +| `import` | import_instincts | path, project | + +Default action: `create`. + +### FR-3: Create `feedback` Tool (absorbs 3 tools) +The system must provide a single `feedback` tool with actions: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `rate` | rate_memory | id, useful (boolean) | +| `suppress` | suppress_memory | id | +| `outcome` | set_session_outcome | outcome (success/partial/failure/abandoned), reason | + +No default — action required. + +### FR-4: Create `vault` Tool (absorbs 5 tools) +The system must provide a single `vault` tool with actions: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `store` | store_credential | name, value, scope, project | +| `get` | get_credential | name | +| `list` | list_credentials | (none) | +| `delete` | delete_credential | name | +| `status` | vault_status | (none) | + +No default — action required. + +### FR-5: Create `docs` Tool (absorbs 11 tools) +The system must provide a single `docs` tool with actions: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `create` | doc_create | path, project, content, doc_type | +| `read` | doc_read | path, project, version | +| `list` | doc_list | project, doc_type | +| `history` | doc_history | path, project | +| `comment` | doc_comment | path, project, comment, line | +| `collections` | list_collections | (none) | +| `documents` | list_documents | collection | +| `get_doc` | get_document | collection, id | +| `remove` | remove_document | collection, id | +| `ingest` | ingest_document | collection, content, metadata | +| `search_docs` | search_collection | collection, query | + +No default — action required. + +### FR-6: Create `admin` Tool (absorbs 21 tools) +The system must provide a single `admin` tool with actions: + +| Action | Replaces | Key Parameters | +|--------|----------|----------------| +| `bulk_delete` | bulk_delete_observations | ids | +| `bulk_supersede` | bulk_mark_superseded | ids | +| `bulk_boost` | bulk_boost_observations | ids, amount | +| `tag` | tag_observation | id, add, remove | +| `by_tag` | get_observations_by_tag | tag | +| `batch_tag` | batch_tag_by_pattern | pattern, tag, action | +| `graph` | graph_query | id, mode | +| `graph_stats` | get_graph_stats | (none) | +| `stats` | get_memory_stats | (none) | +| `trends` | get_temporal_trends | project, days | +| `quality` | get_data_quality_report | project | +| `importance` | analyze_observation_importance | project | +| `search_analytics` | analyze_search_patterns | project | +| `obs_quality` | get_observation_quality | id | +| `scoring` | get_observation_scoring_breakdown | id | +| `consolidations` | suggest_consolidations | project | +| `maintenance` | trigger_maintenance | (none) | +| `maintenance_stats` | get_maintenance_stats | (none) | +| `consolidation` | run_consolidation | project | +| `export` | export_observations | project, format | +| `backfill_status` | backfill_status | (none) | + +No default — action required. + +### FR-7: Backward-Compatible Dispatch Aliases +All 61 original tool names must continue to work when called via `callTool`. The dispatch +switch maps old names to the new primary tool + action. Clients calling `find_by_file(files="x")` +get identical results to `recall(action="by_file", files="x")`. + +### FR-8: Tiered Registration +Default `tools/list` returns only 6+1 primary tools (recall, store, feedback, vault, docs, +admin, check_system_health). With `cursor=all` or `include_all=true`, also returns all 61 +original tools as full Tool objects with their original schemas (same format as pre-consolidation). +Primary tools appear first in the list, aliases after. + +## Non-Functional Requirements + +### NFR-1: Zero Client Breakage +Every existing MCP client must continue to work without modification. Response formats +must be identical for alias calls vs primary tool calls with equivalent parameters. + +### NFR-2: Context Window Reduction +Default `tools/list` response must be under 1000 tokens total (7 tools × ~130 tokens each). +Current: ~6100 tokens for 61 tools. + +### NFR-3: Response Time Parity +Primary tool calls must not add measurable latency compared to direct tool calls. +The action routing must be a simple string switch, not a search/lookup. + +### NFR-4: Input Schema Completeness +Each primary tool's input schema uses a flat structure: `action` as required enum + all +parameters optional. Each parameter description notes which action(s) it applies to. +Target: under 200 tokens per tool schema. No discriminated unions (they bloat beyond savings). + +## User Stories + +### US1: Agent Uses 7 Tools Instead of 61 (P1) +**As an** AI agent using engram via MCP, **I want** to see only 7 well-described tools, +**so that** I can quickly identify the right tool for any memory operation. + +**Acceptance Criteria:** +- [ ] Default `tools/list` returns exactly 7 tools +- [ ] Each tool description explains ALL available actions +- [ ] Agent can perform any operation available in the old 61-tool set + +### US2: Existing Client Continues Working (P1) +**As an** existing MCP client calling old tool names, **I want** my calls to work unchanged, +**so that** I don't need to update any code. + +**Acceptance Criteria:** +- [ ] `search(query="x")` returns same result as `recall(action="search", query="x")` +- [ ] `store_memory(content="x")` returns same result as `store(action="create", content="x")` +- [ ] All 61 old tool names dispatch correctly +- [ ] All 7 backward compat aliases from PR #112 still work + +### US3: Context Window Savings (P1) +**As a** system operator, **I want** reduced MCP tool payload, +**so that** agents have more context window for actual work. + +**Acceptance Criteria:** +- [ ] Default tools/list payload < 1000 tokens +- [ ] Previously: ~6100 tokens → now: ~900 tokens (>80% reduction) + +## Edge Cases + +- Old tool called with parameters matching new schema (e.g., `search(action="by_file")`) — must work, action param passed through +- New tool called without `action` param — use default action or return clear error +- `recall` called with both `action="search"` and `preset="decisions"` — preset takes precedence (matches current behavior) +- `admin` called with unknown action — return descriptive error listing valid actions +- Alias `doc_update` (removed in PR #112) — still dispatches to `docs(action="create")` +- `cursor=all` returns old tool names alongside new primary tools — no duplicate functionality shown + +## Out of Scope + +None — this is the complete consolidation. No items deferred. + +## Dependencies + +- v2.0.9 (PR #112) must be merged — it removed 7 duplicate tools and established dispatch alias pattern +- Constitution Principle #12 — this spec directly implements it + +## Success Criteria + +- [ ] `tools/list` returns 7 tools (6 primary + health) +- [ ] `tools/list?cursor=all` returns 7 primary + 61 aliases = 68 entries +- [ ] All existing tests pass with zero modification to test assertions about tool behavior (only tool name expectations change) +- [ ] `go test ./internal/mcp/...` passes +- [ ] Context window reduction verified: <1000 tokens for default tools/list + +## Clarifications + +### Session 2026-03-28 + +| # | Category | Question | Resolution | Date | +|---|----------|----------|------------|------| +| C1 | UX Flow | How to structure input schema for multi-action tools? | Flat: action enum + all params optional. Descriptions note applicable actions. Target <200 tokens/tool. No discriminated unions. | 2026-03-28 | +| C2 | Integration | How should aliases appear in cursor=all? | Full Tool objects with original schemas (backward compat). Primary tools first, aliases after. | 2026-03-28 | + +## Clarification Summary + +| Category | Status | +|----------|--------| +| Functional Scope | Clear | +| User Roles | Clear | +| Domain/Data Model | Clear | +| Data Lifecycle | Clear | +| Interaction & UX Flow | Resolved (C1) | +| Non-Functional: Perf/Scale | Clear | +| Non-Functional: Reliability | Clear | +| Non-Functional: Security | Clear | +| Integration | Resolved (C2) | +| Edge Cases | Clear | +| Constraints & Tradeoffs | Clear | +| Terminology | Clear | +| Completion Signals | Clear | +| Miscellaneous | Clear | + +**Questions asked/answered:** 2/2 +**Spec status:** Ready for planning +**Next:** /speckit-plan + +## Open Questions + +None. diff --git a/.agent/specs/mcp-tool-api-consolidation/tasks.md b/.agent/specs/mcp-tool-api-consolidation/tasks.md new file mode 100644 index 00000000..476fe40e --- /dev/null +++ b/.agent/specs/mcp-tool-api-consolidation/tasks.md @@ -0,0 +1,79 @@ +# Tasks: MCP Tool API Consolidation (61 → 6+1) + +**Spec:** .agent/specs/mcp-tool-api-consolidation/spec.md +**Plan:** .agent/specs/mcp-tool-api-consolidation/plan.md +**Generated:** 2026-03-28 + +## Phase 1: Create Primary Handler Files (FR-1 through FR-6) + +**Goal:** 6 new handler files, each routing actions to existing handlers +**Independent Test:** Each handler callable directly with action param, returns same results as old tool + +- [ ] T001 [P] [US1] Create `handleRecall` router in `internal/mcp/tools_recall.go` — 13 actions routing to existing search/timeline/pattern handlers +- [ ] T002 [P] [US1] Create `handleStore` router in `internal/mcp/tools_store_consolidated.go` — 4 actions routing to existing store/edit/merge/import handlers +- [ ] T003 [P] [US1] Create `handleFeedback` router in `internal/mcp/tools_feedback.go` — 3 actions routing to existing rate/suppress/outcome handlers +- [ ] T004 [P] [US1] Create `handleVault` router in `internal/mcp/tools_vault_consolidated.go` — 5 actions routing to existing credential handlers +- [ ] T005 [P] [US1] Create `handleDocs` router in `internal/mcp/tools_docs_consolidated.go` — 11 actions routing to existing doc/collection handlers +- [ ] T006 [P] [US1] Create `handleAdmin` router in `internal/mcp/tools_admin.go` — 21 actions routing to existing bulk/tag/graph/maintenance/analytics handlers + +--- + +**Checkpoint:** All 6 primary handlers compile and route correctly. `go build ./...` passes. + +## Phase 2: Register Primary Tools + Alias Tier (FR-7, FR-8) + +**Goal:** Default tools/list returns 7 tools; cursor=all returns 7 + 61 aliases +**Independent Test:** `tools/list` returns 7; `tools/list?cursor=all` returns 68; all alias calls work + +- [ ] T007 [US1] [US3] Replace `allTools` slice in `internal/mcp/server.go` with 7 primary tool definitions (flat schema: action enum + optional params) +- [ ] T008 [US1] Move current 61 tool definitions to `aliasTools` slice in `internal/mcp/server.go` — returned only with `cursor=all` +- [ ] T009 [US2] Update `handleCallTool` dispatch in `internal/mcp/server.go` — primary names call primary handlers; alias names inject action and call primary handlers +- [ ] T010 [US2] Verify all 7 backward compat aliases from PR #112 (get_context_timeline, get_timeline_by_query, get_recent_context, find_by_file_context, get_observation_relationships, get_graph_neighbors, doc_update) still dispatch correctly in `internal/mcp/server.go` +- [ ] T011 Run `go build ./...` to verify compilation in `internal/mcp/` +- [ ] T011a [US3] Measure default tools/list token count (target: <1000 tokens) — count JSON bytes of 7-tool response +- [ ] T011b [US1] Verify aliasTools preserves original schemas exactly (diff old vs new serialization) + +--- + +**Checkpoint:** 7 primary tools registered. 61 aliases in cursor=all. All dispatch paths working. Token count verified. + +## Phase 3: Update Tests (US1, US2, US3) + +**Goal:** All tests pass with new tool structure +**Independent Test:** `go test ./internal/mcp/ -count=1` passes + +- [ ] T012 [US1] Update `TestHandleToolsList` in `internal/mcp/server_test.go` — expect 7 default tools, 68 with cursor=all +- [ ] T013 [US2] Update `TestCallTool_ToolNameRecognition` in `internal/mcp/server_test.go` — test primary + alias names +- [ ] T014 [P] [US2] Add `TestAliasDispatchParity` in `internal/mcp/server_test.go` — verify alias call produces same result as primary+action call for 5 representative tools +- [ ] T015 [US1] Add `TestPrimaryToolActions` in `internal/mcp/server_test.go` — verify each primary tool rejects unknown actions with descriptive error +- [ ] T015a [P] [US1] Spot-check latency: time 10 recall(action="search") calls vs 10 old search() calls, verify <5% overhead in `internal/mcp/server_test.go` +- [ ] T016 Run `go test ./internal/mcp/ -count=1 -timeout 120s` to verify all tests pass + +--- + +**Checkpoint:** All MCP tests pass. Primary tools and aliases verified. + +## Phase 4: Version Bump + Release + +- [ ] T017 Bump openclaw-engram to 2.1.0 in `plugin/openclaw-engram/package.json` +- [ ] T018 Update `engramInstructions` string in `internal/mcp/server.go` to reference 7 primary tools instead of legacy tool names +- [ ] T019 Create PR, run review, merge +- [ ] T020 Tag v2.1.0, create GitHub release with structured notes +- [ ] T021 Verify `tools/list` returns exactly 7 tools on deployed server +- [ ] T022 Measure context token reduction (target: >80% reduction from ~6100 tokens) + +## Dependencies + +``` +Phase 1 ── T001-T006 all parallel (separate files) +Phase 2 ── T007-T009 sequential (same file) ── T010-T011 +Phase 3 ── T012-T015 partially parallel ── T016 +Phase 4 ── depends on Phase 3 complete +``` + +## Execution Strategy + +- **MVP scope:** Phase 1-3 (all tools consolidated + tested) +- **Parallel opportunities:** T001-T006 (6 independent files); T014 parallel with T012-T013 +- **Commit strategy:** One commit per phase +- **Review gates:** `/code-review lite` after Phase 2, full review before merge diff --git a/.agent/tasks/inbox.md b/.agent/tasks/inbox.md new file mode 100644 index 00000000..1fe391eb --- /dev/null +++ b/.agent/tasks/inbox.md @@ -0,0 +1,19 @@ + +- [ ] **[investigate]** Session tracking audit: Active Sessions=0 despite active session; openclaw heartbeat creates empty sessions (0 messages each); Telegram agent dialog not tracked as session; Sessions page shows all zeros. Need audit of: session-start hook init logic, session counting in SessionManager, openclaw plugin session ID computation, Telegram/OpenClaw agent session lifecycle. _2026-03-24_ + +- [ ] **[investigate]** Engram + OpenClaw integration architecture: hook receives ALL messages (heartbeat, Telegram, agent-to-agent, real user prompts) through single UserPromptSubmit entry point. Current approach = regex content filtering (whack-a-mole). Correct approach = message classification at entry: ctx/input metadata should indicate message type (user_prompt, heartbeat, system, agent, external). Requires openclaw audit + engram hook redesign. _2026-03-24_ + +- [ ] **[idea]** UI: add memory notes viewer in dashboard (browse observations as notes) _2026-03-24_ +- [ ] **[idea]** Memory: tree structure + note linking (Obsidian-style graph of connected observations with bidirectional references) _2026-03-24_ +- [ ] **[idea]** Memory: consistency checker + auto-repair for leaf nodes on failures (orphan vectors, broken relations, missing embeddings) _2026-03-24_ +- [ ] **[idea]** Memory: search indexes for instant retrieval (FTS + vector pre-warm + materialized views for common queries) _2026-03-24_ +- [ ] **[idea]** Plugin: memory_get as backward-compatible Markdown bridge — reads existing agent .md files from disk, transparently imports into engram on first access, serves from engram on subsequent reads _2026-03-24_ +- [ ] **[investigate]** Audit incomplete specs: self-learning.md (14/24), and all other specs in .agent/specs/ — find partially implemented features, gaps, abandoned work _2026-03-27_ +- [x] **[debt]** ~~Missing MCP tools: tag_observation~~ Already implemented (server.go line 890). _2026-03-24_ → verified 2026-03-28 +- [ ] **[bug]** OpenClaw engram v1.4.0 — 90s init delay regression. Code analysis shows no blocking imports added. chokidar lazy-load still in place. formatter.ts and message-classifier.ts are lightweight. Need Gateway-side profiling to determine where time is spent. Possible jiti caching issue or npm install delay. _2026-03-25_ +- [x] **[debt]** ~~store_memory without always-inject concept~~ Fixed: added always_inject param (PR #98). _2026-03-28_ +- [ ] **[bug]** CC bug #19225: Stop hooks in plugin hooks.json don't fire. Workaround: registered engram stop.js in global ~/.claude/settings.json. Need to document this for other plugin developers and track upstream fix. _2026-03-28_ +- [ ] **[bug]** Dashboard: Concept filter shows "No items to display" for ALL concepts (how-it-works, pattern, architecture, etc.). Dropdown populated but filtering returns empty. 853 observations exist. Likely client-side filtering bug or API not passing concept param. _2026-03-28_ +- [ ] **[bug]** Dashboard: "50 obs · 50 prompts" hardcoded on home page — should show real counts (853 observations visible in sidebar). _2026-03-28_ +- [ ] **[bug]** Dashboard Summaries tab: "No items to display" — 0 summaries generated in 24h+ despite 7 active sessions and 894 observations. Session summarization (stop.js → /sessions/{id}/summarize) may not be firing or server LLM extraction failing silently. _2026-03-28_ +- [ ] **[idea]** Engram CC plugin user commands: current 3 (setup/doctor/restart) are low-value admin tools. Add user-facing commands: `/engram:retro` (retrospective session analysis — what was injected, what was useful, effectiveness), `/engram:stats` (personal memory stats, learning curve), `/engram:cleanup` (review + suppress low-quality observations), `/engram:export` (export observations as markdown). Consider making existing skills (memory, retrospective-eval) into commands. _2026-03-28_ From ca82b2e56849750f7ae83d29fb3fe5f9de9c0a5e Mon Sep 17 00:00:00 2001 From: Kirill Turanskiy Date: Sat, 28 Mar 2026 05:04:30 +0300 Subject: [PATCH 2/5] =?UTF-8?q?feat:=20create=206=20primary=20tool=20route?= =?UTF-8?q?rs=20(Phase=201=20=E2=80=94=20FR-1=20through=20FR-6)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit New handler files that route consolidated tool actions to existing handlers: - tools_recall.go: 12 actions (search, preset, by_file, by_concept, etc.) - tools_store_consolidated.go: 4 actions (create, edit, merge, import) - tools_feedback.go: 3 actions (rate, suppress, outcome) - tools_vault_consolidated.go: 5 actions (store, get, list, delete, status) - tools_docs_consolidated.go: 11 actions (create, read, list, history, etc.) - tools_admin.go: 21 actions (bulk ops, tags, graph, maintenance, etc.) Each is a thin routing layer — NO new business logic. All delegate to existing handler functions via action parameter dispatch. --- internal/mcp/tools_admin.go | 65 ++++++++++++++++++++ internal/mcp/tools_docs_consolidated.go | 47 +++++++++++++++ internal/mcp/tools_feedback.go | 31 ++++++++++ internal/mcp/tools_recall.go | 76 ++++++++++++++++++++++++ internal/mcp/tools_store_consolidated.go | 30 ++++++++++ internal/mcp/tools_vault_consolidated.go | 35 +++++++++++ 6 files changed, 284 insertions(+) create mode 100644 internal/mcp/tools_admin.go create mode 100644 internal/mcp/tools_docs_consolidated.go create mode 100644 internal/mcp/tools_feedback.go create mode 100644 internal/mcp/tools_recall.go create mode 100644 internal/mcp/tools_store_consolidated.go create mode 100644 internal/mcp/tools_vault_consolidated.go diff --git a/internal/mcp/tools_admin.go b/internal/mcp/tools_admin.go new file mode 100644 index 00000000..fcc4c353 --- /dev/null +++ b/internal/mcp/tools_admin.go @@ -0,0 +1,65 @@ +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +func (s *Server) handleAdmin(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + action := coerceString(m["action"], "") + if action == "" { + return "", fmt.Errorf("action required for admin tool (valid: bulk_delete, bulk_supersede, bulk_boost, tag, by_tag, batch_tag, graph, graph_stats, stats, trends, quality, importance, search_analytics, obs_quality, scoring, consolidations, maintenance, maintenance_stats, consolidation, export, backfill_status)") + } + + switch action { + case "bulk_delete": + return s.handleBulkDeleteObservations(ctx, args) + case "bulk_supersede": + return s.handleBulkMarkSuperseded(ctx, args) + case "bulk_boost": + return s.handleBulkBoostObservations(ctx, args) + case "tag": + return s.handleTagObservation(ctx, args) + case "by_tag": + return s.handleGetObservationsByTag(ctx, args) + case "batch_tag": + return s.handleBatchTagByPattern(ctx, args) + case "graph": + return s.callTool(ctx, "graph_query", args) + case "graph_stats": + return s.handleGetGraphStats(ctx) + case "stats": + return s.handleGetMemoryStats(ctx) + case "trends": + return s.handleGetTemporalTrends(ctx, args) + case "quality": + return s.handleGetDataQualityReport(ctx, args) + case "importance": + return s.handleAnalyzeObservationImportance(ctx, args) + case "search_analytics": + return s.handleAnalyzeSearchPatterns(ctx, args) + case "obs_quality": + return s.handleGetObservationQuality(ctx, args) + case "scoring": + return s.handleGetObservationScoringBreakdown(ctx, args) + case "consolidations": + return s.handleSuggestConsolidations(ctx, args) + case "maintenance": + return s.handleTriggerMaintenance(ctx) + case "maintenance_stats": + return s.handleGetMaintenanceStats(ctx) + case "consolidation": + return s.handleRunConsolidation(ctx, args) + case "export": + return s.handleExportObservations(ctx, args) + case "backfill_status": + return s.handleBackfillStatus() + default: + return "", fmt.Errorf("unknown admin action: %q (valid: bulk_delete, bulk_supersede, bulk_boost, tag, by_tag, batch_tag, graph, graph_stats, stats, trends, quality, importance, search_analytics, obs_quality, scoring, consolidations, maintenance, maintenance_stats, consolidation, export, backfill_status)", action) + } +} diff --git a/internal/mcp/tools_docs_consolidated.go b/internal/mcp/tools_docs_consolidated.go new file mode 100644 index 00000000..526c890a --- /dev/null +++ b/internal/mcp/tools_docs_consolidated.go @@ -0,0 +1,47 @@ +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +// handleDocsConsolidated routes docs tool actions to the appropriate handler. +func (s *Server) handleDocsConsolidated(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + + action := coerceString(m["action"], "") + if action == "" { + return "", fmt.Errorf("action required for docs tool (valid: create, read, list, history, comment, collections, documents, get_doc, remove, ingest, search_docs)") + } + + switch action { + case "create": + return s.handleDocCreate(ctx, args) + case "read": + return s.handleDocRead(ctx, args) + case "list": + return s.handleDocList(ctx, args) + case "history": + return s.handleDocHistory(ctx, args) + case "comment": + return s.handleDocComment(ctx, args) + case "collections": + return s.handleListCollections(ctx) + case "documents": + return s.handleListDocuments(ctx, args) + case "get_doc": + return s.handleGetDocument(ctx, args) + case "remove": + return s.handleRemoveDocument(ctx, args) + case "ingest": + return s.handleIngestDocument(ctx, args) + case "search_docs": + return s.handleSearchCollection(ctx, args) + default: + return "", fmt.Errorf("unknown docs action: %q (valid: create, read, list, history, comment, collections, documents, get_doc, remove, ingest, search_docs)", action) + } +} diff --git a/internal/mcp/tools_feedback.go b/internal/mcp/tools_feedback.go new file mode 100644 index 00000000..6910fee7 --- /dev/null +++ b/internal/mcp/tools_feedback.go @@ -0,0 +1,31 @@ +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +// handleFeedbackConsolidated routes feedback tool actions to the appropriate handler. +func (s *Server) handleFeedbackConsolidated(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + + action := coerceString(m["action"], "") + if action == "" { + return "", fmt.Errorf("action required for feedback tool (valid: rate, suppress, outcome)") + } + + switch action { + case "rate": + return s.handleRateMemory(ctx, args) + case "suppress": + return s.handleSuppressMemory(ctx, args) + case "outcome": + return s.handleSetSessionOutcomeMCP(ctx, args) + default: + return "", fmt.Errorf("unknown feedback action: %q (valid: rate, suppress, outcome)", action) + } +} diff --git a/internal/mcp/tools_recall.go b/internal/mcp/tools_recall.go new file mode 100644 index 00000000..ad216d44 --- /dev/null +++ b/internal/mcp/tools_recall.go @@ -0,0 +1,76 @@ +// Package mcp — tools_recall.go routes consolidated "recall" tool actions +// to existing handler functions on *Server. This is the single entry point +// for all memory retrieval operations, dispatching by action parameter. +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +// handleRecall is the consolidated recall tool handler. It parses the "action" +// parameter and delegates to the appropriate existing handler or callTool dispatch. +func (s *Server) handleRecall(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", fmt.Errorf("recall: %w", err) + } + + action := coerceString(m["action"], "search") + + switch action { + case "search": + // Delegate to the full search dispatch in callTool. + return s.callTool(ctx, "search", args) + + case "preset": + preset := coerceString(m["preset"], "") + switch preset { + case "decisions", "changes", "how_it_works": + return s.callTool(ctx, preset, args) + default: + return "", fmt.Errorf("recall: unknown preset %q (valid: decisions, changes, how_it_works)", preset) + } + + case "by_file": + return s.callTool(ctx, "find_by_file", args) + + case "by_concept": + return s.callTool(ctx, "find_by_concept", args) + + case "by_type": + return s.callTool(ctx, "find_by_type", args) + + case "similar": + return s.handleFindSimilarObservations(ctx, args) + + case "timeline": + return s.callTool(ctx, "timeline", args) + + case "related": + return s.handleFindRelatedObservations(ctx, args) + + case "patterns": + return s.handleGetPatterns(ctx, args) + + case "get": + return s.handleGetObservation(ctx, args) + + case "sessions": + query := coerceString(m["query"], "") + if query != "" { + return s.handleSearchSessions(ctx, args) + } + return s.handleListSessions(ctx, args) + + case "explain": + return s.handleExplainSearchRanking(ctx, args) + + default: + return "", fmt.Errorf( + "unknown recall action: %q (valid: search, preset, by_file, by_concept, by_type, similar, timeline, related, patterns, get, sessions, explain)", + action, + ) + } +} diff --git a/internal/mcp/tools_store_consolidated.go b/internal/mcp/tools_store_consolidated.go new file mode 100644 index 00000000..d36dea2b --- /dev/null +++ b/internal/mcp/tools_store_consolidated.go @@ -0,0 +1,30 @@ +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +// handleStoreConsolidated routes store tool actions to the appropriate handler. +func (s *Server) handleStoreConsolidated(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + + action := coerceString(m["action"], "create") + + switch action { + case "create": + return s.handleStoreMemory(ctx, args) + case "edit": + return s.handleEditObservation(ctx, args) + case "merge": + return s.handleMergeObservations(ctx, args) + case "import": + return s.handleImportInstincts(ctx, args) + default: + return "", fmt.Errorf("unknown store action: %q (valid: create, edit, merge, import)", action) + } +} diff --git a/internal/mcp/tools_vault_consolidated.go b/internal/mcp/tools_vault_consolidated.go new file mode 100644 index 00000000..82c62020 --- /dev/null +++ b/internal/mcp/tools_vault_consolidated.go @@ -0,0 +1,35 @@ +package mcp + +import ( + "context" + "encoding/json" + "fmt" +) + +// handleVaultConsolidated routes vault tool actions to the appropriate handler. +func (s *Server) handleVaultConsolidated(ctx context.Context, args json.RawMessage) (string, error) { + m, err := parseArgs(args) + if err != nil { + return "", err + } + + action := coerceString(m["action"], "") + if action == "" { + return "", fmt.Errorf("action required for vault tool (valid: store, get, list, delete, status)") + } + + switch action { + case "store": + return s.handleStoreCredential(ctx, args) + case "get": + return s.handleGetCredential(ctx, args) + case "list": + return s.handleListCredentials(ctx, args) + case "delete": + return s.handleDeleteCredential(ctx, args) + case "status": + return s.handleVaultStatus(ctx, args) + default: + return "", fmt.Errorf("unknown vault action: %q (valid: store, get, list, delete, status)", action) + } +} From 9bf484fe9df4486c969e47d777dafe50586ddc37 Mon Sep 17 00:00:00 2001 From: Kirill Turanskiy Date: Sat, 28 Mar 2026 05:08:35 +0300 Subject: [PATCH 3/5] =?UTF-8?q?feat:=20register=207=20primary=20tools=20+?= =?UTF-8?q?=20alias=20dispatch=20(Phase=202=20=E2=80=94=20FR-7,=20FR-8)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add primaryTools() returning 6 consolidated tools with flat schemas - Default tools/list returns 7 tools (6 primary + check_system_health) - cursor=all returns primary + 61 legacy alias tools - callTool dispatch: primary names → consolidated handlers first, then fallthrough to legacy alias handlers - All 61 original tool names continue to work via alias dispatch --- internal/mcp/server.go | 177 ++++++++++++++++++++++++++++++++++++++--- 1 file changed, 165 insertions(+), 12 deletions(-) diff --git a/internal/mcp/server.go b/internal/mcp/server.go index 4b7968d5..0d9e17a9 100644 --- a/internal/mcp/server.go +++ b/internal/mcp/server.go @@ -461,6 +461,134 @@ Use file-based memory only for static instructions and user preferences. - Search BEFORE re-exploring code — someone already documented it. - Use specialized tools: ` + "`decisions`" + ` for architecture, ` + "`find_by_file`" + ` for code, ` + "`timeline`" + ` for history.` +// primaryTools returns the 7 consolidated primary tools shown by default. +func (s *Server) primaryTools() []Tool { + return []Tool{ + { + Name: "recall", + Description: "Search and retrieve memories. Actions: search (default), preset (decisions/changes/how_it_works), by_file, by_concept, by_type, similar, timeline, related, patterns, get, sessions, explain.", + tier: tierCore, + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "action": map[string]any{"type": "string", "enum": []string{"search", "preset", "by_file", "by_concept", "by_type", "similar", "timeline", "related", "patterns", "get", "sessions", "explain"}, "default": "search", "description": "Action to perform"}, + "query": map[string]any{"type": "string", "description": "Search query (for search, preset, similar, timeline:query, sessions, explain)"}, + "preset": map[string]any{"type": "string", "enum": []string{"decisions", "changes", "how_it_works"}, "description": "Search preset (for action=preset)"}, + "files": map[string]any{"type": "string", "description": "File paths (for action=by_file)"}, + "concept": map[string]any{"type": "string", "description": "Concept tag (for action=by_concept)"}, + "type": map[string]any{"type": "string", "description": "Observation type (for action=by_type)"}, + "id": map[string]any{"type": "number", "description": "Observation ID (for action=get, related)"}, + "project": map[string]any{"type": "string", "description": "Project name filter"}, + "limit": map[string]any{"type": "number", "description": "Max results"}, + "mode": map[string]any{"type": "string", "description": "Timeline mode: recent/anchor/query (for action=timeline)"}, + "min_similarity": map[string]any{"type": "number", "description": "Min similarity 0-1 (for action=similar)"}, + "min_confidence": map[string]any{"type": "number", "description": "Min confidence 0-1 (for action=related)"}, + "format": map[string]any{"type": "string", "description": "Output format: text/items/detailed"}, + }, + }, + }, + { + Name: "store", + Description: "Store, edit, or merge memories. Actions: create (default), edit, merge, import.", + tier: tierCore, + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{ + "action": map[string]any{"type": "string", "enum": []string{"create", "edit", "merge", "import"}, "default": "create", "description": "Action to perform"}, + "content": map[string]any{"type": "string", "description": "Observation content (for create)"}, + "title": map[string]any{"type": "string", "description": "Title (for create, edit)"}, + "id": map[string]any{"type": "number", "description": "Observation ID (for edit)"}, + "source_id": map[string]any{"type": "number", "description": "Source observation ID (for merge)"}, + "target_id": map[string]any{"type": "number", "description": "Target observation ID (for merge)"}, + "type": map[string]any{"type": "string", "description": "Observation type (for create)"}, + "tags": map[string]any{"type": "string", "description": "Comma-separated tags (for create)"}, + "scope": map[string]any{"type": "string", "description": "Scope: project/global/agent (for create)"}, + "always_inject": map[string]any{"type": "boolean", "description": "Always inject in context (for create, edit)"}, + "narrative": map[string]any{"type": "string", "description": "Narrative text (for edit)"}, + "path": map[string]any{"type": "string", "description": "File path (for import)"}, + "project": map[string]any{"type": "string", "description": "Project name"}, + }, + }, + }, + { + Name: "feedback", + Description: "Rate observations, suppress bad ones, or record session outcome. Actions: rate, suppress, outcome. Action required.", + tier: tierCore, + InputSchema: map[string]any{ + "type": "object", + "required": []string{"action"}, + "properties": map[string]any{ + "action": map[string]any{"type": "string", "enum": []string{"rate", "suppress", "outcome"}, "description": "Action to perform (required)"}, + "id": map[string]any{"type": "number", "description": "Observation ID (for rate, suppress)"}, + "useful": map[string]any{"type": "boolean", "description": "Was it helpful? (for rate)"}, + "outcome": map[string]any{"type": "string", "enum": []string{"success", "partial", "failure", "abandoned"}, "description": "Session outcome (for outcome action)"}, + "reason": map[string]any{"type": "string", "description": "Outcome reason (for outcome action)"}, + }, + }, + }, + { + Name: "vault", + Description: "Manage encrypted credentials. Actions: store, get, list, delete, status. Action required.", + tier: tierCore, + InputSchema: map[string]any{ + "type": "object", + "required": []string{"action"}, + "properties": map[string]any{ + "action": map[string]any{"type": "string", "enum": []string{"store", "get", "list", "delete", "status"}, "description": "Action to perform (required)"}, + "name": map[string]any{"type": "string", "description": "Credential name (for store, get, delete)"}, + "value": map[string]any{"type": "string", "description": "Credential value (for store)"}, + "scope": map[string]any{"type": "string", "description": "Scope: project/global (for store)"}, + "project": map[string]any{"type": "string", "description": "Project name (for store)"}, + }, + }, + }, + { + Name: "docs", + Description: "Versioned documents and collections. Actions: create, read, list, history, comment, collections, documents, get_doc, remove, ingest, search_docs. Action required.", + tier: tierUseful, + InputSchema: map[string]any{ + "type": "object", + "required": []string{"action"}, + "properties": map[string]any{ + "action": map[string]any{"type": "string", "enum": []string{"create", "read", "list", "history", "comment", "collections", "documents", "get_doc", "remove", "ingest", "search_docs"}, "description": "Action to perform (required)"}, + "path": map[string]any{"type": "string", "description": "Document path (for create, read, list, history, comment)"}, + "project": map[string]any{"type": "string", "description": "Project name"}, + "content": map[string]any{"type": "string", "description": "Document content (for create, ingest)"}, + "collection": map[string]any{"type": "string", "description": "Collection name (for documents, get_doc, remove, ingest, search_docs)"}, + "query": map[string]any{"type": "string", "description": "Search query (for search_docs)"}, + "version": map[string]any{"type": "number", "description": "Version number (for read)"}, + "comment": map[string]any{"type": "string", "description": "Comment text (for comment)"}, + "doc_type": map[string]any{"type": "string", "description": "Document type (for create, list)"}, + "id": map[string]any{"type": "string", "description": "Document ID (for get_doc, remove)"}, + }, + }, + }, + { + Name: "admin", + Description: "Administrative operations: bulk ops, tagging, graph, analytics, maintenance. Actions: bulk_delete, bulk_supersede, bulk_boost, tag, by_tag, batch_tag, graph, graph_stats, stats, trends, quality, importance, search_analytics, obs_quality, scoring, consolidations, maintenance, maintenance_stats, consolidation, export, backfill_status. Action required.", + tier: tierUseful, + InputSchema: map[string]any{ + "type": "object", + "required": []string{"action"}, + "properties": map[string]any{ + "action": map[string]any{"type": "string", "description": "Action to perform (required). See tool description for valid actions."}, + "ids": map[string]any{"type": "array", "items": map[string]any{"type": "number"}, "description": "Observation IDs (for bulk_delete, bulk_supersede, bulk_boost)"}, + "id": map[string]any{"type": "number", "description": "Observation ID (for tag, obs_quality, scoring, graph)"}, + "tag": map[string]any{"type": "string", "description": "Tag name (for by_tag, batch_tag)"}, + "project": map[string]any{"type": "string", "description": "Project name (for trends, quality, importance, etc.)"}, + "format": map[string]any{"type": "string", "description": "Export format: json/jsonl/markdown (for export)"}, + "mode": map[string]any{"type": "string", "description": "Graph mode (for graph action)"}, + "amount": map[string]any{"type": "number", "description": "Boost amount (for bulk_boost)"}, + "add": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "description": "Tags to add (for tag)"}, + "remove": map[string]any{"type": "array", "items": map[string]any{"type": "string"}, "description": "Tags to remove (for tag)"}, + "pattern": map[string]any{"type": "string", "description": "Search pattern (for batch_tag)"}, + "days": map[string]any{"type": "number", "description": "Days to analyze (for trends)"}, + }, + }, + }, + } +} + // handleToolsList returns the list of available tools. func (s *Server) handleToolsList(req *Request) *Response { tools := []Tool{ @@ -1387,9 +1515,7 @@ func (s *Server) handleToolsList(req *Request) *Response { ) } - // Tool tiering: parse optional cursor and include_all from request params. - // No cursor / empty → return T1+T2 tools only + nextCursor: "all" - // cursor: "all" OR include_all: true → return ALL tools + // Tool tiering: consolidated primary tools by default, all aliases with cursor=all. var listParams struct { Cursor string `json:"cursor"` IncludeAll bool `json:"include_all"` @@ -1398,14 +1524,20 @@ func (s *Server) handleToolsList(req *Request) *Response { _ = json.Unmarshal(req.Params, &listParams) } + primary := s.primaryTools() + + // Always include check_system_health with primary tools + primary = append(primary, Tool{ + Name: "check_system_health", + Description: "Comprehensive system health check. Returns status of all subsystems (database, vectors, cache, search) with actionable diagnostics.", + tier: tierCore, + InputSchema: map[string]any{ + "type": "object", + "properties": map[string]any{}, + }, + }) + if listParams.Cursor != "all" && !listParams.IncludeAll { - // Filter to primary tools (T1 + T2) - primary := make([]Tool, 0, len(tools)) - for _, t := range tools { - if t.tier <= tierUseful { - primary = append(primary, t) - } - } return &Response{ JSONRPC: "2.0", ID: req.ID, @@ -1416,11 +1548,16 @@ func (s *Server) handleToolsList(req *Request) *Response { } } + // cursor=all: primary tools first, then all legacy aliases + allTools := make([]Tool, 0, len(primary)+len(tools)) + allTools = append(allTools, primary...) + allTools = append(allTools, tools...) + return &Response{ JSONRPC: "2.0", ID: req.ID, Result: map[string]any{ - "tools": tools, + "tools": allTools, }, } } @@ -1476,7 +1613,23 @@ func (s *Server) handleToolsCall(ctx context.Context, req *Request) *Response { // callTool dispatches to the appropriate tool handler. func (s *Server) callTool(ctx context.Context, name string, args json.RawMessage) (string, error) { - // Special handlers for non-search tools + // Primary consolidated tool handlers + switch name { + case "recall": + return s.handleRecall(ctx, args) + case "store": + return s.handleStoreConsolidated(ctx, args) + case "feedback": + return s.handleFeedbackConsolidated(ctx, args) + case "vault": + return s.handleVaultConsolidated(ctx, args) + case "docs": + return s.handleDocsConsolidated(ctx, args) + case "admin": + return s.handleAdmin(ctx, args) + } + + // Legacy alias handlers for non-search tools switch name { case "graph_query": // Consolidated graph tool — routes by mode parameter From 366b1525c8bec70868fe0f78840baf8a4f971e99 Mon Sep 17 00:00:00 2001 From: Kirill Turanskiy Date: Sat, 28 Mar 2026 05:11:24 +0300 Subject: [PATCH 4/5] test: update MCP tests for 7 primary tools (Phase 3) - TestHandleToolsList: expect 7 primary tools by default, legacy in cursor=all - TestCallTool_ToolNameRecognition: verify primary + alias names in cursor=all - Account for conditional tools (store_memory etc.) not present with nil stores --- internal/mcp/server_test.go | 97 +++++++++++++++---------------------- 1 file changed, 38 insertions(+), 59 deletions(-) diff --git a/internal/mcp/server_test.go b/internal/mcp/server_test.go index 72b4bd32..adbe172d 100644 --- a/internal/mcp/server_test.go +++ b/internal/mcp/server_test.go @@ -405,36 +405,28 @@ func TestHandleToolsList(t *testing.T) { require.True(t, ok) assert.NotEmpty(t, tools) - // Default response (no cursor) should return only T1+T2 tools + // Default response (no cursor) should return only primary consolidated tools toolNames := make(map[string]bool) for _, tool := range tools { toolNames[tool.Name] = true } - // T1 (core) tools must be present - t1Tools := []string{ - "search", "decisions", "find_by_file", - "how_it_works", "check_system_health", + // Primary consolidated tools must be present + primaryTools := []string{ + "recall", "store", "feedback", "vault", "docs", "admin", "check_system_health", } - for _, name := range t1Tools { - assert.True(t, toolNames[name], "expected T1 tool %s to be present", name) + for _, name := range primaryTools { + assert.True(t, toolNames[name], "expected primary tool %s to be present", name) } + assert.Equal(t, len(primaryTools), len(tools), "default tools/list should return exactly %d tools", len(primaryTools)) - // T2 (useful) tools must be present - t2Tools := []string{ - "changes", "find_by_type", "find_by_concept", - "find_similar_observations", "timeline", - } - for _, name := range t2Tools { - assert.True(t, toolNames[name], "expected T2 tool %s to be present", name) - } - - // T3 (admin) tools must NOT be present in default listing - t3Tools := []string{ + // Legacy tools must NOT be present in default listing + legacyTools := []string{ + "search", "decisions", "find_by_file", "bulk_delete_observations", "trigger_maintenance", } - for _, name := range t3Tools { - assert.False(t, toolNames[name], "T3 tool %s should not be in default listing", name) + for _, name := range legacyTools { + assert.False(t, toolNames[name], "legacy tool %s should not be in default listing", name) } // Verify nextCursor is returned @@ -442,7 +434,7 @@ func TestHandleToolsList(t *testing.T) { assert.True(t, ok, "nextCursor should be present") assert.Equal(t, "all", nextCursor) - // Verify cursor: "all" returns ALL tools + // Verify cursor: "all" returns primary + all legacy alias tools reqAll := &Request{ JSONRPC: "2.0", ID: 2, @@ -454,13 +446,17 @@ func TestHandleToolsList(t *testing.T) { allTools, _ := resultAll["tools"].([]Tool) assert.Greater(t, len(allTools), len(tools), "cursor=all should return more tools than default") - // T3 tools should now be present + // Legacy tools should be present in cursor=all allToolNames := make(map[string]bool) for _, tool := range allTools { allToolNames[tool.Name] = true } - for _, name := range t3Tools { - assert.True(t, allToolNames[name], "T3 tool %s should be present with cursor=all", name) + for _, name := range legacyTools { + assert.True(t, allToolNames[name], "legacy tool %s should be present with cursor=all", name) + } + // Primary tools should also be in cursor=all + for _, name := range primaryTools { + assert.True(t, allToolNames[name], "primary tool %s should be present with cursor=all", name) } } @@ -1689,41 +1685,24 @@ func TestCallTool_ToolNameRecognition(t *testing.T) { result := resp.Result.(map[string]any) tools := result["tools"].([]Tool) - // Verify all expected tools are registered - expectedTools := map[string]bool{ - "search": true, - "timeline": true, - "decisions": true, - "changes": true, - "how_it_works": true, - "find_by_concept": true, - "find_by_file": true, - "find_by_type": true, - "find_related_observations": true, - "find_similar_observations": true, - "get_patterns": true, - "get_memory_stats": true, - "bulk_delete_observations": true, - "bulk_mark_superseded": true, - "bulk_boost_observations": true, - "trigger_maintenance": true, - "get_maintenance_stats": true, - "merge_observations": true, - "get_observation": true, - "edit_observation": true, - "get_observation_quality": true, - "suggest_consolidations": true, - "tag_observation": true, - "get_observations_by_tag": true, - "get_temporal_trends": true, - "get_data_quality_report": true, - "batch_tag_by_pattern": true, - "explain_search_ranking": true, - "export_observations": true, - "check_system_health": true, - "analyze_search_patterns": true, - "get_observation_scoring_breakdown": true, - "analyze_observation_importance": true, + // Verify primary consolidated tools are registered + primaryExpected := []string{ + "recall", "store", "feedback", "vault", "docs", "admin", "check_system_health", + } + // Verify key legacy alias tools are also registered (cursor=all) + // Note: store_memory, rate_memory, credentials, etc. are conditional (need observationStore != nil) + // Only check unconditionally registered tools here + aliasExpected := []string{ + "search", "timeline", "decisions", "find_by_file", + "bulk_delete_observations", "get_memory_stats", + "find_related_observations", "get_patterns", + } + expectedTools := make(map[string]bool) + for _, name := range primaryExpected { + expectedTools[name] = true + } + for _, name := range aliasExpected { + expectedTools[name] = true } foundTools := make(map[string]bool) From 93d2e50bfaccaad89b4c9232a4b71b152fc773cb Mon Sep 17 00:00:00 2001 From: Kirill Turanskiy Date: Sat, 28 Mar 2026 05:17:32 +0300 Subject: [PATCH 5/5] docs: update engramInstructions for 7 consolidated tools (T018) Replace 61 legacy tool references with 7 primary tools in the MCP server instructions string. Shows action-based API: recall(action=...), store(action=...), feedback(action=...), vault(action=...), docs(action=...), admin(action=...), check_system_health(). Includes backward compat note. --- internal/mcp/server.go | 165 ++++++++--------------------------------- 1 file changed, 32 insertions(+), 133 deletions(-) diff --git a/internal/mcp/server.go b/internal/mcp/server.go index 0d9e17a9..06ec53cb 100644 --- a/internal/mcp/server.go +++ b/internal/mcp/server.go @@ -315,151 +315,50 @@ func (s *Server) buildInstructions() string { // It teaches any agent how to effectively use engram's tools without needing a plugin. const engramInstructions = `# Engram — Persistent Memory for AI Agents -## MANDATORY Rules (non-negotiable) +## MANDATORY Rules -1. **BEFORE modifying any file** → call ` + "`find_by_file(files=\"path/to/file\")`" + ` to check what is known about it. -2. **BEFORE architectural decisions** → call ` + "`decisions(query=\"...\")`" + ` to check prior choices. -3. **BEFORE exploring unfamiliar code** → call ` + "`search(query=\"...\")`" + ` first — it may already be documented. -4. **Read injected context** — ` + "``" + ` and ` + "``" + ` blocks contain prior knowledge. Use it. +1. **BEFORE modifying any file** → ` + "`recall(action=\"by_file\", files=\"path/to/file\")`" + ` +2. **BEFORE architectural decisions** → ` + "`recall(action=\"preset\", preset=\"decisions\", query=\"...\")`" + ` +3. **BEFORE exploring unfamiliar code** → ` + "`recall(query=\"...\")`" + ` — it may already be documented. +4. **Read injected context** — ` + "``" + ` and ` + "``" + ` blocks contain prior knowledge. 5. Do NOT check env vars — call ` + "`check_system_health()`" + ` to verify connectivity. -Engram stores observations from coding sessions in PostgreSQL+pgvector and provides semantic search across them. -Hooks automatically capture knowledge from your sessions. Your job is to **retrieve, connect, and maintain** that knowledge. +## 7 Tools — What They Do + +| Tool | Purpose | Key Actions | +|------|---------|-------------| +| ` + "`recall`" + ` | **Search & retrieve** memories | search (default), preset, by_file, by_concept, by_type, similar, timeline, related, patterns, get, sessions, explain | +| ` + "`store`" + ` | **Save** memories, edit, merge | create (default), edit, merge, import | +| ` + "`feedback`" + ` | **Rate** quality, suppress bad data, record outcomes | rate, suppress, outcome | +| ` + "`vault`" + ` | **Credentials** — encrypted storage | store, get, list, delete, status | +| ` + "`docs`" + ` | **Documents** — versioned docs & collections | create, read, list, history, comment, collections, documents, get_doc, remove, ingest, search_docs | +| ` + "`admin`" + ` | **Bulk ops**, maintenance, analytics | bulk_delete, bulk_supersede, tag, graph, stats, trends, quality, export, maintenance, ... | +| ` + "`check_system_health`" + ` | **Health** check of all subsystems | (no action needed) | ## Quick Start 1. Verify connection: ` + "`check_system_health()`" + ` -2. Search existing knowledge: ` + "`search(query=\"...\")`" + ` -3. Before modifying code: ` + "`find_by_file(files=\"path/to/file\")`" + ` -4. Before architectural decisions: ` + "`decisions(query=\"...\")`" + ` -5. To explicitly remember something: ` + "`store_memory(content=\"...\", title=\"...\")`" + ` -6. To recall stored knowledge: ` + "`recall_memory(query=\"...\")`" + ` - -## Tool Categories - -### Memory Management (Tier 1 — use proactively) -| Tool | When to Use | -|------|-------------| -| ` + "`store_memory`" + ` | Explicitly remember something across sessions — decisions, patterns, preferences, insights. | -| ` + "`recall_memory`" + ` | Retrieve stored knowledge by semantic search. Supports text/items/detailed formats. | - -### Credential Management (secure storage) -| Tool | When to Use | -|------|-------------| -| ` + "`store_credential`" + ` | Securely store an API key, password, or token. Encrypted with AES-256-GCM. | -| ` + "`get_credential`" + ` | Retrieve and decrypt a stored credential by name. | -| ` + "`list_credentials`" + ` | List stored credentials (names and metadata only, no values). | -| ` + "`delete_credential`" + ` | Delete a stored credential by name. Scope-aware (project or global). | -| ` + "`vault_status`" + ` | Check vault encryption status: key configured, fingerprint, credential count, key source. | - -### Search & Retrieval (primary workflow) -| Tool | When to Use | -|------|-------------| -| ` + "`search`" + ` | General semantic search across observations, sessions, prompts. Start here. | -| ` + "`decisions`" + ` | Find past architecture/design decisions before making new ones. | -| ` + "`changes`" + ` | Find code modifications, refactorings, migration history. | -| ` + "`how_it_works`" + ` | Understand system design, patterns, implementation details. | -| ` + "`find_by_concept`" + ` | Browse by concept tag (e.g., "vector-search", "authentication"). | -| ` + "`find_by_file`" + ` | What's known about a file? Check BEFORE modifying unfamiliar code. | -| ` + "`find_by_type`" + ` | Filter by observation type (decision, bugfix, feature, etc.). | -| ` + "`find_similar_observations`" + ` | Pure vector similarity — detect duplicates before creating new ones. | -| ` + "`search_sessions`" + ` | Full-text search across indexed Claude Code session transcripts. | - -### Timeline & Context -| Tool | When to Use | -|------|-------------| -| ` + "`timeline`" + ` | Browse observations around a specific point in time. | -| ` + "`get_recent_context`" + ` | Quick dump of latest observations for a project. | -| ` + "`get_context_timeline`" + ` | Timeline around a specific observation ID. | -| ` + "`get_timeline_by_query`" + ` | Search + timeline combined — finds best match, shows surrounding context. | - -### Graph & Relationships -| Tool | When to Use | -|------|-------------| -| ` + "`find_related_observations`" + ` | Follow knowledge graph edges (causes, fixes, explains, contradicts). | -| ` + "`get_observation_relationships`" + ` | Multi-hop graph traversal with configurable depth. | -| ` + "`get_graph_neighbors`" + ` | FalkorDB graph neighbors (requires FalkorDB backend). | -| ` + "`get_graph_stats`" + ` | Graph backend status and statistics. | - -### Observation Management -| Tool | When to Use | -|------|-------------| -| ` + "`get_observation`" + ` | Fetch single observation by ID with full metadata. | -| ` + "`edit_observation`" + ` | Correct errors, add details, update scope. Only provided fields change. | -| ` + "`tag_observation`" + ` | Add/remove/set concept tags. Modes: add, remove, set. | -| ` + "`get_observations_by_tag`" + ` | List all observations with a specific tag. | -| ` + "`batch_tag_by_pattern`" + ` | Auto-tag observations matching a text pattern. Use dry_run=true first. | -| ` + "`merge_observations`" + ` | Combine duplicates — target kept and boosted, source superseded. | -| ` + "`bulk_delete_observations`" + ` | Batch delete by IDs. | -| ` + "`bulk_mark_superseded`" + ` | Mark observations as stale without deleting. | -| ` + "`bulk_boost_observations`" + ` | Adjust importance scores in bulk (-1.0 to 1.0). | -| ` + "`export_observations`" + ` | Export as JSON, JSONL, or Markdown. | - -### Quality & Analytics -| Tool | When to Use | -|------|-------------| -| ` + "`get_memory_stats`" + ` | System overview — counts, storage, health. | -| ` + "`get_observation_quality`" + ` | Quality score for a single observation with improvement suggestions. | -| ` + "`get_data_quality_report`" + ` | Comprehensive quality assessment across observations. | -| ` + "`get_observation_scoring_breakdown`" + ` | Debug why an observation has its current importance score. | -| ` + "`analyze_observation_importance`" + ` | Project-level importance analysis — top scored, most retrieved. | -| ` + "`get_temporal_trends`" + ` | Activity patterns over time (daily, weekly, hourly). | -| ` + "`explain_search_ranking`" + ` | Debug search result ordering for a query. | -| ` + "`analyze_search_patterns`" + ` | Search usage analytics — common queries, missed results. | -| ` + "`get_patterns`" + ` | Detected recurring patterns (workflow, best_practice, anti_pattern). | - -### Maintenance -| Tool | When to Use | -|------|-------------| -| ` + "`check_system_health`" + ` | Health check of all subsystems. Also verifies engram connectivity. | -| ` + "`suggest_consolidations`" + ` | Find observations that should be merged. | -| ` + "`run_consolidation`" + ` | Trigger decay, association discovery, and/or forgetting cycles. | -| ` + "`trigger_maintenance`" + ` | Run cleanup (old observations, DB optimization). | -| ` + "`get_maintenance_stats`" + ` | Last run time, cleanup counts, configuration. | - -### Sessions -| Tool | When to Use | -|------|-------------| -| ` + "`list_sessions`" + ` | List indexed sessions with workstation/project filters. | -| ` + "`search_sessions`" + ` | Full-text search within session transcripts. | - -### Collections & Documents -| Tool | When to Use | -|------|-------------| -| ` + "`list_collections`" + ` | Show configured collections with document counts. | -| ` + "`list_documents`" + ` | List documents in a collection. | -| ` + "`get_document`" + ` | Retrieve full document content. | -| ` + "`ingest_document`" + ` | Add document — chunks, embeds, stores. Idempotent (same hash = skip). | -| ` + "`search_collection`" + ` | Semantic search across document chunks. | -| ` + "`remove_document`" + ` | Soft-delete (deactivate) a document. | - -### Import -| Tool | When to Use | -|------|-------------| -| ` + "`import_instincts`" + ` | Import ECC instinct files as guidance observations. Idempotent. | +2. Search knowledge: ` + "`recall(query=\"...\")`" + ` +3. Before modifying code: ` + "`recall(action=\"by_file\", files=\"path/to/file\")`" + ` +4. Before decisions: ` + "`recall(action=\"preset\", preset=\"decisions\", query=\"...\")`" + ` +5. Remember something: ` + "`store(content=\"...\", title=\"...\")`" + ` +6. Rate a memory: ` + "`feedback(action=\"rate\", id=123, useful=true)`" + ` +7. Store a secret: ` + "`vault(action=\"store\", name=\"API_KEY\", value=\"...\")`" + ` ## Workflow Patterns -**Starting work:** Context is auto-injected by hooks. Use ` + "`search`" + ` or ` + "`get_recent_context`" + ` for more. -**Before modifying code:** ` + "`find_by_file`" + ` + ` + "`how_it_works`" + ` to understand what's known. -**Before architectural decisions:** ` + "`decisions`" + ` to check prior choices. -**Debugging:** ` + "`find_related_observations`" + ` to trace cause chains. -**Periodic cleanup:** ` + "`suggest_consolidations`" + ` → ` + "`merge_observations`" + ` → ` + "`trigger_maintenance`" + `. -**Storing secrets:** ` + "`store_credential`" + ` for API keys, passwords, tokens. ` + "`vault_status`" + ` to verify encryption is active. - -## Engram vs File-Based Memory - -Prefer ` + "`store_memory`" + ` over file-based memory for decisions, patterns, and insights. -Engram provides semantic search, cross-project visibility (global scope), and cross-machine access. -Use file-based memory only for static instructions and user preferences. +**Starting work:** Context is auto-injected by hooks. Use ` + "`recall(query=\"...\")`" + ` for more. +**Before modifying code:** ` + "`recall(action=\"by_file\")`" + ` + ` + "`recall(action=\"preset\", preset=\"how_it_works\")`" + ` +**Before architectural decisions:** ` + "`recall(action=\"preset\", preset=\"decisions\")`" + ` +**After using a memory:** ` + "`feedback(action=\"rate\", id=N, useful=true)`" + ` +**Debugging:** ` + "`recall(action=\"related\", id=N)`" + ` to trace cause chains. +**Cleanup:** ` + "`admin(action=\"consolidations\")`" + ` → ` + "`store(action=\"merge\")`" + ` → ` + "`admin(action=\"maintenance\")`" + ` +**Secrets:** ` + "`vault(action=\"store\")`" + ` for API keys. ` + "`vault(action=\"status\")`" + ` to verify encryption. -## Common Mistakes +## Backward Compatibility -- Do NOT check ENGRAM_URL/ENGRAM_API_TOKEN env vars — call ` + "`check_system_health()`" + ` instead. -- Use ` + "`store_memory`" + ` when you want to explicitly remember something. Hooks capture observations automatically, but ` + "`store_memory`" + ` lets you create memories on demand. -- Read injected ` + "``" + ` and ` + "``" + ` blocks — they contain prior knowledge. -- Search BEFORE re-exploring code — someone already documented it. -- Use specialized tools: ` + "`decisions`" + ` for architecture, ` + "`find_by_file`" + ` for code, ` + "`timeline`" + ` for history.` +All legacy tool names (search, store_memory, find_by_file, decisions, etc.) still work. +Use ` + "`cursor=all`" + ` in tools/list to see all 61 legacy aliases.` // primaryTools returns the 7 consolidated primary tools shown by default. func (s *Server) primaryTools() []Tool {