Fleet-Wide Pattern Learning System for Cross-Repo Knowledge Sharing

[github-actions]

Strategic Improvement

Extract successful patterns from pipeline runs (fix strategies, iteration sequences, test approaches, architecture decisions) and store in centralized fleet repository. New repos query pattern library to benefit from fleet-wide experience immediately. Enables true cross-repo learning where solutions discovered in one repo inform strategies across the entire fleet.

Acceptance Criteria

• Pattern extraction engine analyzes successful runs: events.jsonl + memory + git history + stage outcomes
• Centralized pattern storage at ~/.shipwright/fleet-patterns.json with versioned schema
• Pattern matching API: query by tech stack similarity, issue type, error signature, architecture
• Build loop and intelligence engine query patterns before making strategic decisions
• Pattern effectiveness tracking: success rate, cost metrics, usage frequency per pattern
• Fleet dashboard widget showing pattern library size, growth rate, and effectiveness trends
• Documentation in CLAUDE.md under intelligence layer and fleet mode sections

Context

• Priority: P2
• Complexity: full
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P2: Intelligence & Learning

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	full
Max Iterations	30
Triage	Directly extends Shipwright's intelligence and fleet systems with cross-repo pattern learning to improve strategy decisions and reduce iteration costs—aligns with core mission.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	25252015691

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (286 lines)

Implementation Plan: Fleet-Wide Pattern Learning System

Goal

Extract successful patterns from pipeline runs, store them in a centralized fleet repository, and expose a query API so new repos benefit from fleet-wide experience immediately.

Brainstorming / Socratic Refinement (resolved autonomously)

Requirements Clarity

• MVP: A bash module (sw-fleet-patterns.sh) with extract, query, record-outcome, and stats subcommands; storage at ~/.shipwright/fleet-patterns.json with versioned schema; integration hooks called from sw-loop.sh (build) and sw-intelligence.sh; dashboard widget reading the stats endpoint.
• Implicit requirements: Schema versioning + migration; bash 3.2 compatibility; atomic writes with locking; idempotent extraction (don't double-count the same run); non-fatal callers (pattern subsystem must never break a pipeline).
• Acceptance criteria (concrete, derived from issue):
  1. sw-fleet-patterns extract analyzes a completed run from events.jsonl + memory + git log + stage outcomes and writes/updates entries.
  2. sw-fleet-patterns query --tech-stack node --issue-type infrastructure [--error-sig <s>] returns ranked matches as JSON.
  3. sw-loop.sh queries patterns before iteration 1 and injects top-N into the prompt.
  4. sw-intelligence.sh consults patterns when scoring strategy options.
  5. record-outcome <pattern-id> <success|failure> <cost> updates effectiveness counters.
  6. Dashboard /api/fleet-patterns endpoint returns {size, growth_7d, top_effective}.
  7. CLAUDE.md updated under Intelligence Layer + Fleet Mode (Multi-Repo Operations).
  8. Test suite covers extract, query, record, schema migration, concurrent write.

Design Alternatives

• A. JSON file (chosen): ~/.shipwright/fleet-patterns.json with jq CRUD + flock. Pros: matches existing artifacts (memory, costs, baselines, machines.json); zero deps; backup-friendly. Cons: O(n) scans beyond ~10k entries.
• B. SQLite via sw-db.sh: Pros: indexed queries, ACID. Cons: heavier integration; cross-repo sync less obvious; new tables + tests.
• C. Hybrid (JSON canonical, SQLite cache): Cons: dual source of truth, more code.

Pick A for MVP — matches ~/.shipwright/baselines/, costs.json, memory/. If scale becomes a problem, Phase 2 shadows into SQLite. Blast radius minimized: new file, new module, additive hooks.

Risk Assessment

• False positives from over-eager extraction (a "successful" run that succeeded by accident).
• Tech-stack contamination: a pattern from a Go repo injected into a Node repo.
• Concurrent extraction from parallel pipelines corrupting fleet-patterns.json.
• Stale patterns: a strategy that worked 6 months ago is now wrong.

Dependency Analysis

• Reads: ~/.shipwright/events.jsonl, .claude/pipeline-state.md, .claude/pipeline-artifacts/*.json, ~/.shipwright/memory/<repo-hash>/*.json, git log.
• Writes: ~/.shipwright/fleet-patterns.json (+ .lock + tmp).
• Called by: sw-loop.sh (prompt composition + post-iteration outcome), sw-intelligence.sh (strategy scoring), sw-pipeline.sh (post-run extraction hook), dashboard/server.ts (stats widget).

Component Diagram

┌──────────────────────────────────────────────────────────────────┐
│  Pipeline Run (sw-pipeline.sh)                                   │
│   └── post-run hook ──▶ sw-fleet-patterns extract                │
└──────────────────────────────────────────────────────────────────┘
                                   │
                                   ▼
┌──────────────────────────────────────────────────────────────────┐
│  Pattern Extraction Engine (sw-fleet-patterns.sh: extract)       │
│   reads: events.jsonl, memory/*.json, git log, stage outcomes    │

</details>

## ✅ Task Checklist
- [ ] Task 1: Create `templates/fleet-patterns-schema-v1.json` and `scripts/lib/fleet-patterns.sh` with VERSION, schema constant, init/lock/load helpers
- [ ] Task 2: Implement atomic write + flock-based locking in lib (with macOS/Linux compat from `lib/compat.sh`)
- [ ] Task 3: Implement `fp_migrate_schema` with stepwise migrations
- [ ] Task 4: Implement `fp_extract_from_run` parsing `events.jsonl`, stage artifacts, and memory
- [ ] Task 5: Implement `fp_query` ranking + `fp_compute_score` + `fp_record_outcome` + `fp_prune_stale`
- [ ] Task 6: Create `scripts/sw-fleet-patterns.sh` CLI (extract/query/record-outcome/stats/migrate/prune), bash 3.2 compatible
- [ ] Task 7: Wire `fleet-patterns` subcommand into `scripts/sw` router
- [ ] Task 8: Add post-run extraction hook in `scripts/sw-pipeline.sh` (guarded, non-fatal); emit `pattern_extracted` event
- [ ] Task 9: Integrate query/record-outcome into `scripts/sw-loop.sh` (prompt composition + iteration outcome handler)
- [ ] Task 10: Integrate query into `scripts/sw-intelligence.sh` strategy scoring
- [ ] Task 11: Add `/api/fleet-patterns` endpoint to `dashboard/server.ts` and a stats widget in `dashboard/public/`
- [ ] Task 12: Write `scripts/sw-fleet-patterns-test.sh` (init, extract, query, record, migrate, concurrent upsert, prune, stats, corrupt-store)
- [ ] Task 13: Register new test suite in `package.json` and AUTO:test-suites; register scripts in AUTO:core-scripts; add file to AUTO:runtime-state
- [ ] Task 14: Update `.claude/CLAUDE.md` Intelligence Layer + Fleet Mode sections; add config knobs to feature flags table
- [ ] Task 15: Run `npm test`, `shipwright doctor`, `shipwright docs sync`; smoke-test extract+query against current repo
- [ ] All 15 tasks complete; new test suite green under `npm test`
- [ ] `shipwright doctor` clean; `shipwright docs check` clean (AUTO sections regenerated)
- [ ] Manual: extract on this repo's last pipeline produces ≥1 entry; subsequent query returns it
- [ ] Concurrent upsert test passes 10 consecutive runs (no JSON corruption)
- [ ] Dashboard widget renders with non-zero size after extraction

---
_Generated by `shipwright pipeline` at 2026-05-02T12:46:59Z_

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (124 lines)

Design: Fleet-Wide Pattern Learning System for Cross-Repo Knowledge Sharing

Context

Pipeline runs across the fleet (sethdford/shipwright and downstream daemon-managed repos) generate valuable strategy/outcome data, but each new repo currently starts cold — re-discovering test commands, file layouts, and infrastructure approaches that already worked elsewhere. Memory today is per-repo (~/.shipwright/memory/<repo-hash>/); there is no canonical fleet-wide pattern store, no extraction hook on pipeline completion, and no query API the build loop or intelligence layer can consult.

Codebase constraints that shape the design:

• Bash 3.2 target (no declare -A, readarray, ${var,,}); macOS + Linux compat helpers in scripts/lib/compat.sh.
• Existing artifacts already live in ~/.shipwright/ as JSON (costs.json, baselines/, machines.json, events.jsonl) — pattern store should match this convention.
• Parallel pipelines (--worktree, fleet mode) run concurrently — any shared state needs locking + atomic writes. CLAUDE.md "Common Pitfalls" already mandates tmp+mv.
• All pipeline-adjacent subsystems must be non-fatal to callers — a pattern subsystem failure cannot break a build.
• AUTO sections in .claude/CLAUDE.md (core-scripts, test-suites, runtime-state) are auto-generated; new files must register there.
• 102 existing test suites; new feature must add a suite to package.json and pass under npm test.

Decision

Build a JSON-backed pattern store at ~/.shipwright/fleet-patterns.json with a thin CLI (sw-fleet-patterns.sh) over a shared library (scripts/lib/fleet-patterns.sh). Five subsystems compose:

1. Store layer (lib/fleet-patterns.sh) — flock-protected read-modify-write with 5s timeout, jq empty validation before mv, schema-versioned envelope ({schema_version, entries[], stats{}}), stepwise migrations.
2. Extraction — post-run hook in sw-pipeline.sh calls sw-fleet-patterns extract --run-id $RUN_ID || true; extractor reads events.jsonl, stage artifacts, memory's success-patterns.json, computes a SHA1 fingerprint over canonical fields, upserts.
3. Query/scoring — Jaccard ≥0.5 on tech-stack tokens (language/framework/test_runner/package_manager) gates results; ranked by success_rate * log(uses+1) / cost_factor with age decay; floors at min_success_rate=0.6 and min_uses=3 before injection.
4. Outcome tracking — record-outcome increments success/failure counters atomically; prune retires patterns older than 90d.
5. Surfaces — sw-loop.sh injects top-N (default 3) into iteration-1 prompt only (avoid bloat); sw-intelligence.sh weights matched patterns into strategy scoring; dashboard/server.ts exposes GET /api/fleet-patterns shelling to stats --json.

Error model: distinct exit codes mapped to EFP_NO_STORE, EFP_LOCK_TIMEOUT, EFP_BAD_SCHEMA, EFP_INVALID_PATTERN. All callers wrap with || true. Errors emit pattern_* events to events.jsonl for observability without breaking pipelines.

Data flow: pipeline complete → extract (non-fatal) → upsert under flock → next run's loop calls query → top-N injected into prompt → iteration result → record-outcome → counters update → dashboard widget renders fresh stats.

Alternatives Considered

1. JSON file store (chosen) — Pros: matches existing ~/.shipwright/ artifacts; zero new deps; trivially backup/sync-able across fleet via rsync or future shared mount; jq is already a hard dep. Cons: O(n) scans past ~10k entries; concurrent writes need explicit locking.

2. SQLite via sw-db.sh — Pros: indexed lookups, ACID, scales to 100k+ entries naturally. Cons: heavier integration (new tables, migrations in two places); cross-repo sync semantics less obvious (DBs don't merge); doubles the test surface; over-engineered for MVP demand. Deferred as Phase 2 trigger when size > 10000.

3. Hybrid (JSON canonical + SQLite read cache) — Pros: query speed + simple sync. Cons: dual source of truth invites drift bugs; cache invalidation on every upsert; more code than either pure option. Rejected.

4. Per-repo memory only (status quo, scoped only to extraction polish) — Pros: zero new infra. Cons: doesn't solve the cross-repo cold-start problem the issue explicitly targets. Rejected.

Implementation Plan

Files to create:

• scripts/sw-fleet-patterns.sh — CLI: extract|query|record-outcome|stats|migrate|prune (~600 lines)
• scripts/lib/fleet-patterns.sh — store I/O, locking, scoring, migration (~350 lines)
• scripts/sw-fleet-patterns-test.sh — unit + integration + concurrent-write E2E (~400 lines)
• templates/fleet-patterns-schema-v1.json — empty store template

Files to modify:

• scripts/sw — register fleet-patterns subcommand
• scripts/sw-pipeline.sh — post-run extraction hook (guarded || true)
• scripts/sw-loop.sh — query in compose_prompt (iteration 1 only); record-outcome in iteration outcome handler
• scripts/sw-intelligence.sh — consult patterns in strategy scoring
• dashboard/server.ts — add GET /api/fleet-patterns (~30 lines)
• dashboard/public/ — stats card (size, growth_7d, top-3)
• .claude/CLAUDE.md — Intelligence Layer subsection + Fleet Mode reference + feature flags row; AUTO sections regenerate
• package.json — "test:fleet-patterns" script; included in aggregate npm test
• .claude/daemon-config.json template — fleet_patterns config block

Dependencies: none new. Uses existing jq, flock (with macOS path-based fallback already present in lib/compat.sh), sha1sum/shasum.

Risk areas:

---

Generated by shipwright pipeline design stage at 2026-05-02T12:49:25Z

[github-actions]

[github-actions]

🔨 Build started — shipwright loop with 30 max iterations, 1 agent(s), model: opus

[github-actions]

[github-actions]