arch: file layout manifest — single source of truth for directory structure across versions

[diberry]

Problem

Squad has no single source of truth for file layout across versions. Five separate code paths each make independent assumptions about where files live:

• init.ts — creates directories at init time
• migrations.ts — moves files between locations on upgrade
• upgrade.ts — syncs template files to target directories
• squad.agent.md — tells agents where to read/write (skills, decisions, etc.)
• skill-source.ts, resolver.ts — runtime reads from hardcoded paths

When a file location changes between versions (e.g., skills moving from .squad/skills/ to .copilot/skills/), every path must be updated independently. Issue #77 (skills discovery bug) is a direct result of this gap — the migration moved files, but the runtime, agent instructions, and spawn templates weren't all updated consistently.

Proposed Solution: File Layout Manifest

A single schema/registry that defines:

• Every directory Squad manages and its purpose
• Canonical location per version (e.g., v0.8: .squad/skills/, v0.9+: .copilot/skills/)
• Read priority order (which locations to check, in what order)
• Write target (where new content goes)
• Migration path (what moves where on upgrade)

All five code paths would reference this manifest instead of hardcoding paths independently.

Benefits

• Version-to-version changes tracked in ONE place
• Migrations auto-derived from manifest diffs
• Runtime read logic generated from manifest (no more either/or bugs)
• Agent instructions can reference manifest for correct write paths
• squad doctor can validate layout against manifest

Related

• bug: skill discovery only reads one directory, ignores the other when both exist #77 — skills discovery bug (first symptom of this gap)
• skill-source.ts either/or pattern
• resolver.ts .find() pattern

[diberry]

Addendum: Versioned Manifest

The manifest MUST include a schema version. This is how migrations work:

jsonc { "schemaVersion": "0.9.0", "layout": { "skills": { "purpose": "Reusable patterns and process knowledge", "canonical": ".copilot/skills/", "readOrder": [".copilot/skills/", ".squad/skills/"], "writeTarget": ".copilot/skills/", "deprecated": [".squad/skills/", ".ai-team/skills/"], "since": "0.9.0" } // ... other directories } }

Migration derivation

Given manifest v0.8 and v0.9, the upgrade tool diffs:

• Moved: skills.canonical changed from .squad/skills/ → .copilot/skills/ → generate mv migration step
• Added: new directory entry in v0.9 not in v0.8 → generate mkdir step
• Deprecated: directory moved to deprecated list → generate read-fallback + deprecation warning
• Removed: directory gone entirely → generate cleanup step with user confirmation

Benefits of versioning

• squad doctor validates layout against the manifest for the installed version
• squad upgrade diffs manifests to auto-generate migration steps
• squad.agent.md references manifest for correct write paths (no hardcoding)
• Runtime (skill-source.ts, resolver.ts) reads readOrder from manifest instead of hardcoding

[diberry]

🤖 Skill Design for Manifest Workflow

Flight here. I've designed a set of 5 reusable agent skills to keep the manifest accurate and in sync throughout the Squad version lifecycle. Each skill is triggered at a specific point in the workflow and maintains one piece of the manifest → code consistency contract.

Core Principles

1. Manifest is source of truth — agents read it, never hardcode paths
2. Async consistency — skills can be run independently or chained
3. Single responsibility — each skill does one job well
4. Validation first — before committing changes, skills verify what they touch

---

The 5 Skills

1. Manifest Audit & Sync ✅

Purpose: Validate that all five hardcoded path locations match the manifest, and flag mismatches.

Trigger: On PR creation, squad doctor run, version bump (pre-release)

Inputs: Current manifest, version being worked on, codepaths to audit (init.ts, migrations.ts, upgrade.ts, squad.agent.md, skill-source.ts, resolver.ts)

Outputs: Audit report with pass/fail, specific file:line mismatches, diff suggestions for each file

Key Steps:

• Parse manifest and extract all canonical paths, deprecated aliases, readOrder arrays
• Search each codepath for hardcoded path strings (regex patterns)
• Verify each hardcoded path matches readOrder or is deprecated
• Flag mismatches with file:line and suggested fix
• Generate summary: "X files sync'd, Y mismatches found"

---

2. Manifest Evolution Planner 📋

Purpose: Guide maintainer to update manifest atomically when adding/moving/deprecating directories, then auto-generate migration steps.

Trigger: squad manifest:plan --from=0.8.0 --to=0.9.0 or PR that affects directory layout

Inputs: Current manifest version, target version, list of proposed changes (move/add/deprecate)

Outputs: Updated manifest with new schemaVersion, auto-generated migration.ts entries (mv/mkdir/cleanup), validation checklist

Key Steps:

• Load old manifest, create draft new manifest with bumped schemaVersion
• For each change, prompt maintainer to confirm and update manifest
• Diff old vs new manifests: identify moved, added, deprecated, removed directories
• Generate migration step for each diff (mv, mkdir, cleanup, deprecation warning)
• Generate checklist of ALL places that reference these directories

---

3. Migration Validator & Applier 🔄

Purpose: Given old and new manifest versions, validate migrations will succeed on a real squad instance, then safely apply them.

Trigger: squad upgrade command, pre-merge on upgrade PRs, squad doctor --repair

Inputs: Old manifest version, new manifest version, squad instance root directory, dry-run mode flag

Outputs: Dry-run report (what will move/create/deprecate), actual migration execution, validation that layout matches manifest, rollback instructions if needed

Key Steps:

• Diff manifests to derive steps: moved dirs, new dirs, deprecated, removed
• For each move: verify source exists, target doesn't exist, permissions are correct
• For deprecated: leave in place, add deprecation marker file, log warning
• Execute migrations in dependency order (create parents before children)
• Post-migration, run "Manifest Audit & Sync" to verify everything matches

---

4. Agent Path Resolver 🧭

Purpose: At agent spawn time, resolve correct read/write paths for current Squad version, so agents never hardcode or guess.

Trigger: Every agent spawn, when agent writes to .squad/, when agent reads from .squad/

Inputs: Current Squad version (from package.json), manifest, directory key (e.g., "skills", "decisions"), operation type ("read" or "write")

Outputs: Resolved path(s): write → single canonical path, read → array of paths in readOrder (first existing wins), fallback behavior for deprecated

Key Steps:

• Load manifest and current version (from package.json or SQUAD_VERSION env)
• Look up directory entry in manifest (readOrder, writeTarget)
• For write: return manifest[dir].writeTarget
• For read: iterate readOrder, return first existing path
• If reading from deprecated, log deprecation warning with migration guidance

---

5. Manifest Doctor & Repair 🏥

Purpose: Detect when squad instance's directory layout doesn't match manifest, diagnose why, repair it.

Trigger: squad doctor run, squad init (validate), when agent can't find expected files

Inputs: Current manifest, squad instance root, repair flag (dry-run vs. actual)

Outputs: Diagnosis (layout matches ✅ or list of issues), repair actions (create missing dirs, move orphaned files, remove deprecated)

Key Steps:

• Walk manifest layout, check each directory exists at canonical path
• If missing, check deprecated paths (might be old version)
• If found in deprecated, confirm move to canonical
• Walk filesystem for unexpected directories (orphans)
• Generate repair plan, execute if approved

---

Skill Invocation Flowchart

`
Brady starts version work
↓

1. Manifest Evolution Planner → Brady updates manifest, skill generates migrations
   ↓
2. Manifest Audit & Sync → Find all hardcoded paths, generate sync checklist
   ↓
3. Brady updates init.ts, squad.agent.md, resolver.ts (or agents do it)
   ↓
4. Manifest Audit & Sync → Verify all synced before PR
   ↓
5. On squad upgrade:
   • Migration Validator & Applier → Dry-run first, then execute
   • Manifest Doctor & Repair → Verify layout matches manifest post-upgrade
     ↓
6. Agent Path Resolver → Active on every agent spawn, ensures paths are current
   `

---

Integration Points

• squad.agent.md — references manifest; agents use Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 to resolve paths
• resolver.ts — calls Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 instead of hardcoding readOrder
• skill-source.ts — calls Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 to find skills
• migrations.ts — generated by Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2
• squad doctor — invokes Skills docs: add feature pages for link, nap, and scrub-emails commands #1 and StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 for validation/repair
• CI/GitHub Actions — runs Skill docs: add feature pages for link, nap, and scrub-emails commands #1 on all PRs (gate merges on manifest sync)

---

Implementation Priority

1. Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 first (Agent Path Resolver) — runtime active ingredient; everything depends on it
2. Skill docs: add feature pages for link, nap, and scrub-emails commands #1 (Manifest Audit & Sync) — enables PR validation
3. Create manifest.json in .squad/ with v0.8.0 schema + all current directories
4. Skills feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2, feat(sdk): StorageProvider Phase 1 — interface, FSStorageProvider, security hardening (#481) #3, StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 — higher-level workflows
5. Add CI step — run Skill docs: add feature pages for link, nap, and scrub-emails commands #1 on every PR to prevent drift before merge

This design makes manifest the single source of truth, version work predictable, and gives maintainers tools to keep Squad's directory layout consistent.

[diberry]

[diberry]

🧪 FIDO — Quality Review

Status: ⚠️ NEEDS WORK — Strong concept, critical gaps in testability & edge cases

Well-motivated architecture fix addressing real pain (issue #77). The manifest + 5-skill design reduces hardcoding effectively. However, significant concerns about complexity, testability, and failure modes need resolution before implementation.

---

✅ STRENGTHS

1. Root cause analysis is solid — The 5 independent code paths causing bug: skill discovery only reads one directory, ignores the other when both exist #77 is real. A manifest is the right solution.
2. Versioning strategy is smart — schemaVersion + readOrder + deprecated paths enable backward compatibility without fragile either/or logic.
3. Skill design is modular — Each skill has clear single responsibility and defined inputs/outputs.
4. Migration auto-derivation — Diff-based approach reduces manual error.

---

🔴 CRITICAL GAPS

1. Testability — The 5 Skills Are Untestable As Designed

The skills are agent workflows with manual prompts, not deterministic test cases.

Questions:

• Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2 (Evolution Planner) prompts maintainer interactively. How do you test this? Mock the human? Record expected interactions?
• Skill feat(sdk): StorageProvider Phase 1 — interface, FSStorageProvider, security hardening (#481) #3 (Migration Validator) needs real squad instances at v0.8 and v0.9. Is there a test fixture for this?
• Skill docs: add feature pages for link, nap, and scrub-emails commands #1 (Audit & Sync) searches hardcoded strings via regex. What prevents false positives? Does it catch paths in comments vs actual code?

Proposal: Extract deterministic core of each skill into testable functions, then wrap in agent prompts. Example: validateMigrationStep(oldManifest, newManifest, instanceRoot) → {issues: [], canRun: bool} — test this independently; agents call it.

Deliverable needed: Test plan with concrete test cases:

• Input fixtures (manifest v0.8, v0.9, squad instance directories)
• Expected outputs (audit report, migration plan, validation result)
• At least 1 positive + 1 negative case per skill

2. Edge Cases — Several Failure Modes Unaddressed

Corrupted Manifest:

• What if manifest.json is unparseable (bad JSON)? Does resolver.ts fail silently or throw?
• What if readOrder is empty? What if writeTarget is not in readOrder?
• Proposal: JSON Schema validation in Skill StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 with repair capability.

Missing Manifest:

• If .squad/manifest.json is missing entirely, Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 fails on every agent spawn. Bad for fresh installs.
• Should there be fallback to hardcoded paths?
• Proposal: Define minimum viable manifest. If missing, create from defaults or fail loudly with setup instructions.

Version Mismatch:

• What if package.json version doesn't match any manifest schemaVersion?
• Proposal: Skill feat(sdk): StorageProvider Phase 1 — interface, FSStorageProvider, security hardening (#481) #3 should handle this. Document version matching strategy.

Partial Migrations:

• If migration v0.8 to v0.9 partially fails (e.g., skills move succeeds, decisions move fails), what's the recovery?
• Can users re-run? Do you track migration state?
• Proposal: Add migration audit trail to manifest ("appliedMigrations": ["0.8.0→0.9.0"]). Skill feat(sdk): StorageProvider Phase 1 — interface, FSStorageProvider, security hardening (#481) #3 uses this to resume or reject re-runs.

3. Error Paths — No Fallback Strategy

Current assumption: Manifest always exists, is valid, Skill #4 always succeeds.

Fallback hierarchy needed:

• Read paths: Try readOrder; if all fail, try deprecated; if all fail, error (not silent).
• Write paths: Write canonical; if inaccessible, error (don't fall back).
• Migrations: Dry-run first; require user confirmation; provide rollback instructions.
• Validation: Audit mismatches are CI blockers; doctor mismatches are warnings.

4. Customer Impact — Upgrade Risk Not Addressed

Scenario: Squad v0.7 user upgrades to v0.9 (manifest introduces).

Questions:

• Is upgrade automatic or user-confirmed?
• Is there a dry-run mode (squad upgrade --dry-run)?
• If something fails, is there rollback?
• Does squad doctor catch issues post-upgrade?
• Data loss risk if migration fails silently and old directories left behind?

Proposal:

• Dry-run + user confirmation mandatory.
• Post-migration, run Skill StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 (Doctor) automatically. Abort if issues found.
• Document upgrade path clearly in migration guide.

5. The 5 Skills — Are They All Necessary?

You're proposing design + 5 implementations + orchestration + CI. That's a lot of surface area.

Honest assessment:

• Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 (Agent Path Resolver) — MUST HAVE. Runtime depends on it.
• Skill docs: add feature pages for link, nap, and scrub-emails commands #1 (Audit & Sync) — NEEDED to prevent drift. But is it a PR blocker or just a report?
• Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2 (Evolution Planner) — NICE TO HAVE. Can you just document the process and do it manually first? Automate later.
• Skill feat(sdk): StorageProvider Phase 1 — interface, FSStorageProvider, security hardening (#481) #3 (Migration Validator) — DEPENDS on Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2. If Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2 is manual, does this become simpler?
• Skill StorageProvider: Phase 3 — CLI migration to SquadState APIs #5 (Doctor & Repair) — NEEDED for troubleshooting. Automated or on-demand?

Proposal: Prioritize ruthlessly. Build Skill #4 + #1 first. Prove value reduces hardcoding. Then add #2, #3, #5 if needed.

---

🟡 COMPLEXITY BUDGET — Is This Worth It?

Problem: 5 hardcoded paths cause #77. Manifest centralizes these.

Cost: Schema, 5 skills, CI integration, migration logic, fallback handling, test fixtures for all versions.

Alternatives:

• Alt 1: Fix bug: skill discovery only reads one directory, ignores the other when both exist #77 manually this time. Add linter rule to prevent drift going forward. (Lower cost, doesn't scale.)
• Alt 2: Implement only Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 + docs: add feature pages for link, nap, and scrub-emails commands #1 first. Prove value. Then expand.

Recommendation: Build incrementally. Start Skill #4. If works, do Skill #1. Then evaluate whether #2/3/5 are needed.

---

📋 BEFORE CODING

1. Define test fixtures & cases for each skill with concrete inputs/outputs
2. Document failure modes & fallback behavior
3. Walkthrough real v0.7 → v0.9 upgrade scenario
4. Commit to Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4 → docs: add feature pages for link, nap, and scrub-emails commands #1 first. Don't build all 5 at once.
5. Define when/where Skill docs: add feature pages for link, nap, and scrub-emails commands #1 runs in CI (hard blocker or advisory?)

Direction is supportable. Let's validate testability and edge cases first.

[diberry]

Sub-Issues Created

This issue has been broken down into 3 sequenced sub-issues per the review feedback:

Phase 1: Foundation

• Foundation — Manifest Schema + Path Resolver (Skill #4) #80 — Foundation — Manifest Schema + Path Resolver (Skill StorageProvider: RoutingConfig.moduleOwnership and TeamConfig.projectContext always empty #4)

Phase 2: Code Migration

• Code Migration — Audit Skill (#1) + Runtime Integration #81 — Code Migration — Audit Skill (docs: add feature pages for link, nap, and scrub-emails commands #1) + Runtime Integration

Phase 3: Automation

• Automation — Migration Generator (Skill #2) + Doctor (Skill #5) #82 — Automation — Migration Generator (Skill feat(docs): improve search with Pagefind tuning, section badges, and test coverage #2) + Doctor (Skill StorageProvider: Phase 3 — CLI migration to SquadState APIs #5)

Sequencing Plan

1. Start with Foundation — Manifest Schema + Path Resolver (Skill #4) #80 (foundation)
2. Then Code Migration — Audit Skill (#1) + Runtime Integration #81 (code migration)
3. Then Automation — Migration Generator (Skill #2) + Doctor (Skill #5) #82 (automation)

All review feedback addressed: schema + path resolver, test fixtures, version strategy, backwards compat, fallback strategy, and proof-of-value approach.

[github-actions]

🏗️ Squad Triage — Flight (Lead)

Issue: #79 — arch: file layout manifest — single source of truth for directory structure across versions
Assigned to: FIDO (Quality Owner)
Reason: Issue relates to testing/quality work

@copilot evaluation: No strong capability match — routed to squad member.

Team roster:

• Flight (Lead) → label: squad:flight
• Procedures (Prompt Engineer) → label: squad:procedures
• EECOM (Core Dev) → label: squad:eecom
• FIDO (Quality Owner) → label: squad:fido
• PAO (DevRel) → label: squad:pao
• CAPCOM (SDK Expert) → label: squad:capcom
• CONTROL (TypeScript Engineer) → label: squad:control
• Surgeon (Release Manager) → label: squad:surgeon
• Booster (CI/CD Engineer) → label: squad:booster
• GNC (Node.js Runtime) → label: squad:gnc
• Network (Distribution) → label: squad:network
• RETRO (Security) → label: squad:retro
• INCO (CLI UX & Visual Design) → label: squad:inco
• GUIDO (VS Code Extension) → label: squad:guido
• Telemetry (Aspire & Observability) → label: squad:telemetry
• VOX (REPL & Interactive Shell) → label: squad:vox
• DSKY (TUI Engineer) → label: squad:dsky
• Sims (E2E Test Engineer) → label: squad:sims
• Handbook (SDK Usability) → label: squad:handbook
• Ralph (Work Monitor) → label: squad:ralph
• @copilot (Coding Agent) → label: squad:copilot

To reassign, remove the current squad:* label and add the correct one.