decisions.md grows unbounded — agents waste context reading stale decisions

[diberry]

Problem

decisions.md grows unbounded in active Squad projects. A real-world Squad deployment (MMO project) reached 145KB / 2,219 lines / ~35K tokens — and every agent reads the entire file at spawn time.

Impact

• Context waste: Every agent burns ~35K tokens reading decisions before doing any work
• Slower spawns: More data to parse = slower agent startup
• Higher cost: Wasted tokens = wasted money on every agent interaction
• Reduced capacity: Less room in the context window for actual task work
• Compounding problem: Active projects generate 5-15 decisions/session; grows ~1-2KB/session

Root Cause Analysis

The current architecture has a write/read asymmetry:

• Writes are incremental: Agents write small decisions to decisions/inbox/, Scribe merges them — this works great
• Reads are monolithic: Every agent reads the ENTIRE decisions.md at spawn — this doesn't scale
• Archival is reactive: The 20KB/30-day rule in Scribe Task StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 exists but doesn't trigger reliably because:
  1. Scribe runs haiku (fast/cheap model) which sometimes skips the archival step
  2. The threshold check is a soft guideline, not enforced by code
  3. No automated mechanism exists outside of Scribe spawns

Evidence from User Report

"decisions.md is 149KB (~2,200 lines). Every agent reads it at spawn — that's ~35K tokens of context burned before any work starts."

The archive file was only 26KB despite the main file being 145KB — archival wasn't keeping pace with growth.

Current Mechanisms (and why they're insufficient)

Mechanism	Location	Issue
Scribe Task #5: Archive >20KB, >30 days	squad.agent.md:869	Soft guideline, not enforced by code; Scribe (haiku) often skips it
Scribe deduplication	scribe-charter.md:37-47	Handles exact dupes, but most decisions are unique
Nap skill	.squad/skills/nap/	On-demand only; user must know to invoke it
History 12KB rule	squad.agent.md:871	Only covers agent history.md, NOT decisions.md

Proposed Solutions

A. Changes to Squad Core (code/governance changes)

A1. Hard size enforcement in Scribe spawn prompt

Add explicit, measurable enforcement to the Scribe spawn template:

5. DECISIONS ARCHIVE (MANDATORY):
   a. Measure decisions.md size (wc -c or equivalent)
   b. If >20KB OR >100 entries:
      - Move entries older than 14 days to decisions-archive.md
      - Move entries with "Status: Implemented" regardless of age
      - Keep only entries from the last 14 days that are NOT implemented
   c. After archival, verify size is <20KB. If still over, archive entries >7 days old.
   d. Log: "📦 Archived {N} decisions ({old_size} → {new_size})"

A2. Decision index / summary section

Add a compact index at the top of decisions.md that agents can scan quickly:

## Decision Index (auto-generated by Scribe)
| ID | Date | Topic | Status | By |
|----|------|-------|--------|----|
| D-42 | 2026-03-20 | Use JWT for auth | Active | Ripley |
| D-41 | 2026-03-18 | Postgres over MySQL | Implemented | Dallas |
...

## Active Decisions (details below)
### 2026-03-20: Use JWT for auth
...

Agents reading decisions.md can scan the index (~2KB) and only deep-read entries relevant to their task.

A3. Scoped decision reads in spawn prompts

Instead of "Read .squad/decisions.md", change the spawn template to:

Read .squad/decisions.md — focus on the Decision Index at the top.
Only deep-read entries relevant to your current task domain: {task_domain}

This gives agents permission to skim rather than consuming every entry.

A4. decisions.md size check in squad nap

The nap skill already checks history.md (15KB) and total .squad/ (1MB). Add:

• decisions.md >20KB → trigger archival
• Make this part of the automated nap cycle, not just on-demand

A5. Configurable thresholds in .squad/config.json

Allow users to tune archival behavior:

{
  "decisions": {
    "maxSizeKB": 20,
    "archiveAfterDays": 14,
    "autoArchiveImplemented": true,
    "maxEntries": 100
  }
}

B. User-implementable solutions (ceremonies, skills, practices)

B1. "Decisions Cleanup" Ceremony

Users can add to .squad/ceremonies.md:

- name: Decisions Cleanup
  type: maintenance
  when: manual
  trigger: "decisions cleanup" or "clean decisions"
  facilitator: Scribe
  description: >
    Archive implemented and stale decisions. Rebuild decision index.
    Run when decisions.md exceeds 20KB or has >100 entries.
  steps:
    1. Scribe measures current size
    2. Scribe identifies implemented/stale entries (>30 days, status: implemented)
    3. Scribe moves them to decisions-archive.md
    4. Scribe rebuilds the Decision Index section
    5. Scribe reports: size before/after, entries archived

B2. "Decisions Hygiene" Skill

Users can create .squad/skills/decisions-hygiene/SKILL.md:

name: decisions-hygiene
description: Keep decisions.md under control
domain: squad-maintenance
confidence: medium
source: observed

Patterns:

• Mark decisions as Status: Implemented when work is done
• Use concise decision format (3-4 lines max per entry)
• Prefix decisions with domain tags: [auth], [api], [ui] for filtering
• Run decisions cleanup ceremony weekly in active projects

B3. Session-start size check

Users can add to their Lead's charter or a personal agent:

At session start, check the size of .squad/decisions.md.
If >20KB, recommend: "⚠️ decisions.md is {size}KB — consider running 'decisions cleanup'"

B4. Ralph integration

Add a decisions size check to Ralph's idle-watch:

When board is clear AND decisions.md > 20KB:
  "📦 decisions.md is {size}KB — spawning Scribe for archival"
  → Auto-spawn Scribe with archival task

Success Criteria

• Active decisions.md stays under 20KB (~5K tokens) in steady state
• Archival runs automatically, not just when users remember to invoke it
• Agents can efficiently read only relevant decisions
• Users have ceremony/skill patterns for manual cleanup when needed
• Config allows tuning thresholds per project

Labels

squad, enhancement, squad:lead

[github-actions]

🏗️ Squad Triage — Flight (Lead)

Issue: #20 — decisions.md grows unbounded — agents waste context reading stale decisions
Assigned to: FIDO (Quality Owner)
Reason: Issue relates to testing/quality work

@copilot evaluation: 🔴 Not suitable — issue involves work outside the coding agent's capability profile.

Team roster:

• Flight (Lead) → label: squad:flight
• Procedures (Prompt Engineer) → label: squad:procedures
• EECOM (Core Dev) → label: squad:eecom
• FIDO (Quality Owner) → label: squad:fido
• PAO (DevRel) → label: squad:pao
• CAPCOM (SDK Expert) → label: squad:capcom
• CONTROL (TypeScript Engineer) → label: squad:control
• Surgeon (Release Manager) → label: squad:surgeon
• Booster (CI/CD Engineer) → label: squad:booster
• GNC (Node.js Runtime) → label: squad:gnc
• Network (Distribution) → label: squad:network
• RETRO (Security) → label: squad:retro
• INCO (CLI UX & Visual Design) → label: squad:inco
• GUIDO (VS Code Extension) → label: squad:guido
• Telemetry (Aspire & Observability) → label: squad:telemetry
• VOX (REPL & Interactive Shell) → label: squad:vox
• DSKY (TUI Engineer) → label: squad:dsky
• Sims (E2E Test Engineer) → label: squad:sims
• Handbook (SDK Usability) → label: squad:handbook
• Ralph (Work Monitor) → label: squad:ralph
• @copilot (Coding Agent) → label: squad:copilot

To reassign, remove the current squad:* label and add the correct one.

[diberry]

🏗️ Full Team Review — 5 Agents, Synthesized Findings

Reviewers: Flight (Lead), EECOM (Core Dev), FIDO (Quality Owner), Procedures (Prompt Engineer), Telemetry (Observability)

---

🔴 Critical Finding: This Repo Is 2.4× Worse Than the Issue Describes

The issue cites 145KB as the motivating example. This repo's actual state:

File	Size	Status
decisions.md	345KB / 5,286 lines	17× the 20KB target
decisions-archive.md	233KB / 2,964 lines	Archive itself is unbounded
decisions/inbox/	1 file pending	Scribe merge incomplete

🔴 Critical Finding: archiveDecisions() Already Exists in nap.ts

EECOM + FIDO confirmed: nap.ts line 314 already implements deterministic archival via StorageProvider. The issue proposes building what partially exists. Two bugs found in the existing implementation:

1. Undated entries are immune to archival — daysAgoFromLine() returns null for entries without dates → they're kept forever. The ## Foundational Directives section uses ## headings (not ###) and is invisible to the parser entirely.
2. Age-only pruning fails for active projects — if all entries are <30 days old (realistic at 15 decisions/session), archival does nothing even at 345KB. The 7-day second-pass fallback proposed in A1 is not implemented.

🔴 Critical Finding: Union Merge Driver Is a Silent Landmine

EECOM + FIDO: .gitattributes specifies decisions.md merge=union (keep all lines from both sides). Archival removes lines. When branches merge, the union driver resurrects archived entries. This is a structural defect that makes naive line-deletion archival dangerous.

EECOM's solution: Annotate-in-place (mark entries as Status: Archived but keep the line) rather than deleting. This is union-merge-safe.

🔴 Critical Finding: Archive Has No Size Bound

The archive is 233KB and growing. The issue proposes moving entries from decisions.md → decisions-archive.md but never addresses archive pruning. We're moving the problem, not solving it.

🔴 Critical Finding: Zero Observability on Decisions

Telemetry: No squad.decisions.* metrics exist anywhere in the codebase. The subsystem is completely dark. You cannot verify archival works, measure token cost impact, or detect when archival stops working. "The first PR should not be the archival fix — it should be the metrics."

---

🟡 Key Architecture Questions Raised

1. Should decisions.md be a flat file? (Flight)
Not long-term. Keep it as human-readable view but drive from decisions.json. This is Phase 3/post-MVP but the schema must accommodate it from Phase 1.

2. Coordinator pre-spawn filtering is the highest-ROI fix (Flight + EECOM + Procedures)
The lifecycle spawn API already accepts decisions?: string but nobody populates it. Instead of giving agents the full 345KB file, the coordinator should filter by domain tags and inject only relevant decisions. This reduces 86K tokens/spawn to ~2K tokens immediately with zero archival risk.

3. Decisions need lifecycle states (Flight + Procedures)
proposed → active → implemented → superseded → archived. Without states, every fix is a blunt timestamp heuristic. Status-based archival (Status: Implemented → archive regardless of age) is more precise than age-based.

4. Scoped reads (A3) don't reduce token consumption (Flight + Procedures)
Once a file is in an agent's context, all tokens are consumed regardless of prompt instructions to "focus." A3 is cognitive guidance masquerading as an architectural fix. The real fix is coordinator-side injection filtering before the agent sees the file.

5. Prompt-based enforcement (A1) violates the project's own rule (Flight)
decisions.md already contains: "Security, PII, and file-write guards are implemented via hooks, NOT prompt instructions. Prompts can be ignored." A1 proposes the same class of fix that's already documented as unreliable.

---

🟡 Scribe Prompt Fix (Procedures)

Root cause of Scribe skipping archival: The Scribe charter has no archival step — only the spawn task list does. Under context pressure, Haiku follows charter identity and treats Task #5 as optional. Fixes:

1. Move archival to Task docs: add feature pages for link, nap, and scrub-emails commands #1 (before merge, while context is fresh)
2. Add HARD GATE label with exact byte thresholds (not ~20KB)
3. Add archival to the Scribe charter, not just the task list
4. Better yet: have Scribe call squad nap deterministically instead of doing archival via prompt

---

Recommended Revised Phase Plan

Phase	What	Owner	Priority
0	Stop the bleeding: run squad nap on 345KB corpus; fix undated-entry bug; fix ## vs ### parsing	EECOM	P0
1a	squad.decisions.* OTel metrics — instrument before fixing	Telemetry	P1
1b	Coordinator pre-spawn filtering — filterDecisionsForAgent() using existing lifecycle API	EECOM	P1
1c	Add status field to ParsedDecision + status-based archival in nap.ts	EECOM	P1
2a	decisions-index.md as separate file (not inside decisions.md — avoids union merge conflict)	EECOM + Procedures	P2
2b	Configurable thresholds in config.json	EECOM	P2
2c	Ralph idle-watch: auto-trigger archival when board clear + decisions >20KB	EECOM	P2
3	decisions.json structured backend (future, post-v1)	—	P3

Key Additions to PRD

• Non-goals: LLM-based conflict resolution, cross-squad federation, decisions versioning
• Bugs to fix first: Undated entry immunity, ## heading invisibility, archive unbounded
• Union merge safety: Either annotate-in-place OR move index to separate file OR accept solo-dev limitation
• Success metrics: Token consumption per spawn ≤5K (from ~86K); squad doctor clean on active project after 30 sessions; archival runs without user intervention weekly
• Acceptance criteria (FIDO): before_count === after_count + archived_count for every archival test; 353KB archival completes in <500ms; no entry ever lost during archive cycle

Cross-Reference

Companion issue: #21 provides detection (squad doctor), #20 provides remediation (archival + filtering). squad nap --repair is the shared implementation surface. Phase 0 of both should ship in the same PR.

[diberry]

Postmortem: archiveDecisions() Silent Failure — decisions.md Bloat to 345KB

Filed by: Flight
Date: 2026-07-22
Status: Bug fix in progress (squad/20-fix-nap-archival-bugs)

---

Incident

Two logic bugs in packages/squad-cli/src/cli/core/nap.ts archiveDecisions() caused it to silently do nothing for the actual .squad/decisions.md in this repository, allowing the file to grow to 345KB (353,414 bytes) — 17× over the 20KB hygiene threshold. Every agent spawn loaded the full file, wasting roughly 86K tokens per spawn in context that was supposed to be cleaned up automatically.

---

Impact

• decisions.md: 345KB (20KB threshold = 17× over budget)
• Token waste per agent spawn: ~86K tokens (345KB × 250 tokens/KB)
• Archival silently returned null on every nap invocation — no error, no warning
• The file has been growing unchecked since at least February 2026 (5 months)
• Active projects are the worst affected: the more a team uses Squad, the worse the bloat

---

Timeline

Date	Event
2026-02-28	archiveDecisions() introduced in commit aaefeb1 (PR bradygaster#635/bradygaster#636)
2026-02-28	2 archival tests shipped with the implementation — both using hardcoded 2024-01-XX dates
2026-02-28 → now	decisions.md grew unchecked; nap ran but silently did nothing for archival
2026-07-22	Bugs identified: undated-entry immunity (Bug 1) + all-recent no-op (Bug 2)
2026-07-22	squad/20-fix-nap-archival-bugs branch opened; fix implemented, tests added

---

Root Causes

RC1 — Bug 1: Undated entries are immune to archival

daysAgoFromLine() returns null when a line has no YYYY-MM-DD date. The split loop placed null-daysAgo entries into recent (the "keep" bucket) unconditionally:

// BUGGY — null goes to recent, never archived
if (e.daysAgo !== null && e.daysAgo > DECISION_MAX_AGE_DAYS) {
  old.push(e);
} else {
  recent.push(e);  // ← undated entries land here
}

The real decisions.md has 707 ##/### entries, many of which predate the date-stamping convention and carry no date. All of them were immune to archival.

RC2 — Bug 2: All-recent projects never archive

if (old.length === 0) return null;  // ← early exit for active teams

If every dated entry is <30 days old (true for any actively-developed project in its first month, or after a quiet period), archiveDecisions() returns null even if the file is 345KB. No secondary threshold, no count-based fallback.

RC3 — ## section headings invisible to the parser

The real decisions.md uses ## to group entries into sections (e.g., ## Foundational Directives) with ### entries inside. The parser only matches ^###\s — ## headings are silently skipped. This means section structure is lost when archiving (section headers become orphaned) and the entry count is understated.

---

Contributing Factors

CF1 — AI-authored code with AI-authored tests in a single pass

The entire feature — implementation + 38 tests — landed in one 1,492-line commit (aaefeb1), co-authored by Copilot. The tests were generated in the same mental model as the implementation, so they baked in the same assumption: entries always have dates. The generateDecisions() fixture helper used hardcoded 2024-01-XX dates — guaranteed >30 days old, guaranteed to have dates. The happy path worked perfectly. The adversarial cases were never considered.

CF2 — Test fixtures synthesized, not sampled from reality

Both archival tests used a for (let i = 0; i < 30; i++) loop generating ### 2024-01-XX: Decision N entries. The real decisions.md has:

• Undated entries (early directives like ### Type safety — strict mode non-negotiable)
• ## grouping headings (707 total)
• Entries from the past week (active project, all "recent")

No test ever ran against a realistic sample of the actual file.

CF3 — No format specification for decisions.md

The Scribe charter says "Merge inbox → decisions.md, delete inbox files" and "Deduplicate by ### blocks" but never mandates ### YYYY-MM-DD: format. The foundational directives at the top of decisions.md — written before the date convention emerged — have no dates. Nobody ever enforced the format, so the parser's assumption ("all entries have dates") was wrong from day one.

CF4 — Silent failure by design

archiveDecisions() returns null to signal "nothing to do." There is no distinction between "file is under threshold" and "file is over threshold but I couldn't archive anything." The caller (runNap) silently swallows the null. A user running squad nap with a 345KB decisions.md would see: 😴 Nap complete - nothing to clean up. — no indication the archival logic had been defeated.

CF5 — No coverage requirement for new hygiene functions

There was no team rule requiring archival/pruning functions to include:

• Undated entry tests
• All-recent entry tests
• Empty file / single entry tests
• Data-loss invariant test (entries_before = entries_in_main + entries_in_archive)

The PR description celebrated "38 tests" — which sounds thorough, but 36 of those covered history compression, log pruning, inbox cleanup, dry-run, and report formatting. Only 2 covered archival, and both were happy-path.

CF6 — No design review before implementation

The nap feature went straight from issue to code. There was no proposal in docs/proposals/ (which our own proposal-first directive requires for meaningful changes). A design review would have surfaced the question: "what happens with undated entries?" and "what's the fallback when everything is recent?"

---

Action Items

#	Item	Owner	Priority
A1	Merge the bug fix on squad/20-fix-nap-archival-bugs — three-bucket split (old/undated/recent) + count-based fallback	EECOM / Sims	P0 — immediate
A2	Add ## heading awareness to archiveDecisions() — preserve section structure during archival, don't lose ## groupings	EECOM	P1
A3	Add data-loss invariant test: entries_before === entries_in_main + entries_in_archive — fails if any entry is silently dropped	Sims	P1
A4	Update Scribe charter to mandate ### YYYY-MM-DD: format for all new inbox entries; Scribe should validate format before merging	Scribe / Procedures	P1
A5	Add silent-failure detection: when file is over threshold but archival returns null, emit a warning in the nap report	EECOM	P2
A6	Archival test checklist — add to CONTRIBUTING or the nap SKILL.md: any archival/pruning function must test (undated entries, all-recent, empty, single entry, data-loss invariant)	Procedures	P2
A7	Backfill proposal in docs/proposals/ for nap feature — documents design assumptions and invariants; satisfies proposal-first policy retroactively	PAO	P3
A8	Add contract comment to archiveDecisions() documenting: entry format assumptions, what happens with undated entries, size guarantee	EECOM	P2

---

Lessons Learned

1. AI writes the tests it believes in, not the tests that break it

When AI generates both the implementation and the tests in a single pass, the tests confirm the implementation's assumptions rather than challenging them. The happy-path fixture (2024-01-XX dated entries) was the only world the implementation was ever tested against. Adversarial test generation must be a separate, deliberate step — ideally by a different agent (Sims, Fido) than the one who wrote the implementation.

2. Synthetic test fixtures that don't reflect production data are a liability

generateDecisions() produced clean, perfectly-formatted, dated entries. The real decisions.md is messier. At least one test should use a realistic sample of the actual file for any function that processes .squad/ state. Fixture realism is not a luxury — it's a correctness requirement.

3. Silent null returns from hygiene functions are dangerous

A file hygiene function that silently does nothing is indistinguishable from a function that successfully cleaned up. The nap report should distinguish: "nothing needed" vs. "cleanup was attempted but conditions weren't met." Consider returning a { reason: string } when null is the result, or emitting a warn action type.

4. "Over threshold but can't archive" needs a fallback, always

Any size-based archival function operating on real-world data must handle: all entries recent, all entries undated, file growing faster than the archival window. The 30-day age threshold alone is not sufficient — it needs a count-based or byte-budget fallback. Document fallback strategy before implementing.

5. Format specs must precede parsers

The archival parser assumed ### YYYY-MM-DD: format. The format was never specified. Scribe merges inbox files with ### headings, but the early decisions had no dates and ## groupings were added organically. When writing a parser for team-managed files, lock the format spec first — in the Scribe charter, a template, or a validator.

6. Proposal-first policy exists for exactly this class of bug

The proposal-first rule isn't bureaucracy — it's a forcing function to surface design assumptions before they become bugs. The nap feature skipped the proposal step. A one-page proposal would have asked: "How do we handle undated entries?" "What's the fallback for active projects?" These questions have obvious answers once asked. Enforce the proposal gate for all hygiene/archival features.

---

Appendix: The Two Bugs, Annotated

// BUG 1: null daysAgo → recent (immune to archival)
for (const e of entries) {
  if (e.daysAgo !== null && e.daysAgo > DECISION_MAX_AGE_DAYS) {
    old.push(e);
  } else {
    recent.push(e); // ← undated entries end up here, never archived
  }
}

// BUG 2: if all entries are recent, early-return with no action
if (old.length === 0) return null; // ← 345KB file, returns null

Both bugs stem from the same assumption: "the file will always have dated entries older than 30 days." For a real, active team using Squad daily, this assumption is wrong within weeks of starting.