diff --git a/.squad/agents/pao/history.md b/.squad/agents/pao/history.md index 21d9f7fef..84df3254e 100644 --- a/.squad/agents/pao/history.md +++ b/.squad/agents/pao/history.md @@ -285,3 +285,14 @@ Completed full PRD based on research findings. **Document:** `docs/research/jsdo - Four-phase approach breaks large effort into digestible increments (Phase 0 validation before JSDoc audit helps mitigate risk of TypeDoc setup failing) **Decision:** PRD approved for handoff to implementation team. Ready for execution on next sprint. + +### HARD GATE archival docs (issue #69) +**Context:** Documented the HARD GATE archival mechanism in the Scribe workflow — two-tier thresholds (20KB/30 days, 50KB/7 days), heading-aware archival, count-based fallback, and the `archiveDecisions()` contract. + +**Learnings:** +- HARD GATE archival is an internal mechanism (Scribe workflow), not user-facing — placed in features/ alongside Memory, not in concepts/ +- The `archiveDecisions()` function lives in `packages/squad-cli/src/cli/core/nap.ts`, not in squad-sdk +- Two-tier approach: Tier 1 (age-based, 30 days) fires first at 20KB; Tier 2 (aggressive, 7 days) only if still over 50KB after Tier 1 +- Count-based fallback handles edge case where all entries are recent but file is still too large +- Undated entries (foundational directives without `YYYY-MM-DD` in heading) are always preserved — never archived +- The Scribe workflow order is PRE-CHECK → ARCHIVE [HARD GATE] → INBOX MERGE — archival runs before merge to prevent unbounded growth diff --git a/docs/src/content/docs/features/decision-archival.md b/docs/src/content/docs/features/decision-archival.md new file mode 100644 index 000000000..11a0e9538 --- /dev/null +++ b/docs/src/content/docs/features/decision-archival.md @@ -0,0 +1,133 @@ +# Decision archival + +> ⚠️ **Experimental** — Squad is alpha software. APIs, commands, and behavior may change between releases. + +Squad's shared decision log (`decisions.md`) grows with every session. Left unchecked, it consumes agent context windows and slows down every interaction. Decision archival keeps the log lean by moving stale entries to an archive file — automatically, before every Scribe merge. + +--- + +## The problem + +Every agent reads `decisions.md` at the start of every session. As the file grows, agents spend more context budget on old sprint artifacts, completed analysis docs, and one-time planning fragments instead of current, actionable decisions. + +Without intervention, `decisions.md` can exceed 50 KB in active projects — that's roughly 12,500 tokens of context consumed before an agent even starts working. + +--- + +## How HARD GATE archival works + +The Scribe agent enforces a **HARD GATE** on `decisions.md` size. "Hard gate" means this step is mandatory — it runs before every inbox merge and cannot be skipped or deferred. + +The workflow order is: + +1. **PRE-CHECK** — Measure `decisions.md` size and count inbox files +2. **ARCHIVE (HARD GATE)** — Run two-tier archival if thresholds are exceeded +3. **INBOX MERGE** — Merge new decisions from the inbox into `decisions.md` + +By archiving *before* merging, the file never grows unbounded — even when multiple agents write decisions in the same session. + +--- + +## Two-tier thresholds + +Archival uses two tiers that escalate based on file size: + +| Tier | Trigger | Action | Effect | +|------|---------|--------|--------| +| **Tier 1 (30-day)** | `decisions.md` ≥ 20 KB | Archive entries older than 30 days | Removes stale decisions while keeping recent context | +| **Tier 2 (7-day)** | `decisions.md` ≥ 50 KB after Tier 1 | Archive entries older than 7 days | Aggressive cleanup for runaway growth | + +Both tiers run in sequence. If Tier 1 brings the file under 50 KB, Tier 2 is skipped. + +### Count-based fallback + +When no entries exceed the age limit but the file still exceeds the 20 KB threshold, the archival engine falls back to **count-based archival** — it removes the oldest dated entries until the remaining content fits within the size budget. Undated entries (typically foundational directives) are always preserved. + +--- + +## Heading-aware archival + +Decision entries in `decisions.md` use the format: + +```markdown +### YYYY-MM-DD: Topic +**By:** Agent Name +**What:** Description of the decision +**Why:** Rationale +``` + +The archival engine parses `###` headings to identify entry boundaries. Each entry is treated as an atomic unit — archival never splits an entry mid-content. Entries without a parseable date in the heading are treated as undated and always kept. + +### Invariant + +The archival process guarantees: + +``` +entries_before = entries_kept + entries_archived +``` + +No decision data is ever silently dropped. Every archived entry is appended to `.squad/decisions-archive.md`. + +--- + +## Where archived decisions go + +Archived entries move to `.squad/decisions-archive.md`. This file is preserved for reference but is **not loaded into agent context**. Agents read only the lean, current `decisions.md`. + +``` +.squad/ +├── decisions.md # Active decisions (agents read this) +├── decisions-archive.md # Archived decisions (reference only) +└── decisions/ + └── inbox/ # New decisions waiting to be merged +``` + +--- + +## The `archiveDecisions()` contract + +The SDK exposes `archiveDecisions()` in the nap module (`packages/squad-cli/src/cli/core/nap.ts`). Key behaviors: + +| Behavior | Detail | +|----------|--------| +| **Threshold** | File must exceed 20 KB (`DECISION_THRESHOLD`) before any archival runs | +| **Age-based first** | Entries older than 30 days (`DECISION_MAX_AGE_DAYS`) are archived first | +| **Count-based fallback** | If no entries are old enough, the oldest dated entries are removed to fit the budget | +| **Undated entries preserved** | Entries without `YYYY-MM-DD` in the heading are never archived | +| **Atomic writes** | Archive content is appended to `decisions-archive.md`, then `decisions.md` is rewritten | +| **Dry run support** | Pass `dryRun: true` to calculate actions without writing to disk | +| **Return value** | Returns `null` if no archival is needed; otherwise returns a `NapAction` with bytes saved | + +--- + +## Health reports + +After archival runs, the Scribe emits a **HEALTH REPORT** to the session log (`.squad/log/`). The report includes: + +- `decisions.md` size before and after archival +- Number of inbox files processed +- Number of history files summarized + +This gives you visibility into how the team's shared memory is managed across sessions. + +--- + +## Practical example + +Here's what happens as `decisions.md` grows over the life of a project: + +| Project stage | File size | What happens | +|---------------|-----------|--------------| +| Early development | 5 KB | No archival — file is under threshold | +| Active sprint | 22 KB | **Tier 1** fires: entries older than 30 days move to archive | +| Heavy decision period | 55 KB after Tier 1 | **Tier 2** fires: entries older than 7 days move to archive | +| All entries are recent | 25 KB, nothing older than 30 days | **Count-based fallback**: oldest dated entries archived until file fits under 20 KB | +| Only foundational directives | 18 KB of undated entries | No archival — undated entries are always preserved | + +--- + +## Related pages + +- [Memory system](/docs/features/memory) — How Squad's three memory layers work +- [Context hygiene](/docs/features/context-hygiene) — User-facing commands for managing context growth (`squad nap`, `squad reskill`) +- [Memory & Knowledge](/docs/concepts/memory-and-knowledge) — Conceptual overview of how memory compounds over time diff --git a/docs/src/navigation.ts b/docs/src/navigation.ts index 76bfc212e..cffbcfe9e 100644 --- a/docs/src/navigation.ts +++ b/docs/src/navigation.ts @@ -46,6 +46,7 @@ export const NAV_SECTIONS: NavSection[] = [ { title: 'Response Modes', slug: 'features/response-modes' }, { title: 'Parallel Execution', slug: 'features/parallel-execution' }, { title: 'Memory', slug: 'features/memory' }, + { title: 'Decision Archival', slug: 'features/decision-archival' }, { title: 'Skills', slug: 'features/skills' }, { title: 'Directives', slug: 'features/directives' }, { title: 'Ceremonies', slug: 'features/ceremonies' }, @@ -87,6 +88,7 @@ export const NAV_SECTIONS: NavSection[] = [ { title: 'SDK Integration', slug: 'reference/integration' }, { title: 'Tools & Hooks', slug: 'reference/tools-and-hooks' }, { title: 'Config', slug: 'reference/config' }, + { title: 'Config Model', slug: 'reference/config-model' }, { title: 'Glossary', slug: 'reference/glossary' }, ], }, diff --git a/test/docs-build.test.ts b/test/docs-build.test.ts index aeb51d790..e47bc906d 100644 --- a/test/docs-build.test.ts +++ b/test/docs-build.test.ts @@ -19,7 +19,7 @@ const EXPECTED_GET_STARTED = ['installation', 'first-session', 'five-minute-star const EXPECTED_GUIDES = ['build-autonomous-agent', 'building-extensions', 'building-resilient-agents', 'contributing', 'contributors', 'extensibility', 'faq', 'github-auth-setup', 'personal-squad', 'sample-prompts', 'shell', 'tips-and-tricks']; -const EXPECTED_REFERENCE = ['cli', 'sdk', 'config', 'api-reference', 'integration', 'tools-and-hooks', 'glossary']; +const EXPECTED_REFERENCE = ['cli', 'sdk', 'config', 'config-model', 'api-reference', 'integration', 'tools-and-hooks', 'glossary']; const EXPECTED_SCENARIOS = [ 'aspire-dashboard', @@ -54,6 +54,7 @@ const EXPECTED_FEATURES = [ 'capability-routing', 'consult-mode', 'copilot-coding-agent', + 'decision-archival', 'directives', 'enterprise-platforms', 'export-import',