RFC: Centralized follow-ups backlog with drift detection (emerged from Sentinel CHARTER-11 retrospective)

[montfort]

RFC: Centralized follow-ups backlog with drift detection

Author: Claude Opus 4.7 (claude-code-v1.0) on behalf of StrangeDaysTech/sentinel adopter
Source: StrangeDaysTech/sentinel end-of-chain retrospective (CHARTER-08 → 09 → 10 → 11)
Date: 2026-05-06

Hallazgo

DevTrail's §Follow-ups convention works at the per-AILOG write time — when an AILOG is created, the implementer documents what's deferred to subsequent Charters or operational triggers. After 47 AILOGs in StrangeDaysTech/sentinel, the cumulative list of pending follow-ups dispersed across §Follow-ups sections + R<N> (new, not in Charter) entries became large enough that re-discovering them required a multi-agent retrospective scan (two parallel Explore agents + manual classification, ~30 min of agent work to produce a usable triage).

The 4-Charter chain (CHARTER-08 → CHARTER-11) demonstrated the gap empirically:

• Each Charter generated R<N> risks + §Follow-ups entries.
• Without a central registry, the operator could not see "what's pending across the project" without asking the agent to re-classify each chain link's follow-ups in isolation.
• Some follow-ups were closed by subsequent Charters (R5 of CHARTER-09 closed by CHARTER-10; R5 of CHARTER-10 closed by CHARTER-11) but the connection was implicit — no document tracked the "open → closed" transition explicitly.
• Other follow-ups predate the chain by months (~30 entries from etapa1 through PLAN-07); without a registry, they sit in markdown sections that no tool indexes and no review mechanism surfaces.

Two failure modes emerge from this gap:

1. Lost work: a follow-up tagged "do this when X arrives" is forgotten when X arrives because nobody re-reads the AILOG that documented it. Example from Sentinel: CHARTER-09 §Risk R5 (a date-boundary flake) would have flake'd intermittently every May 1-7 / Jun 1-7 / etc. forever because no tooling surfaced it; only the CHARTER-10 retrospective accident triggered the fix.

2. Re-analysis overhead: every operator session that wants to know "what's pending" must scan all AILOGs from scratch. The triage that produced the 47-entry table for the Sentinel adopter took two parallel Explore agents + manual classification (~30 min of agent work). Repeated across many sessions, this is real cost.

The DevTrail convention provides the data (every AILOG follows the same §Follow-ups shape) but no aggregation. The gap is at the framework level — every adopter that runs DevTrail at scale will hit the same wall.

Implementación (Sentinel-side)

Sentinel implemented a local solution under CHARTER-12 to validate the workflow before proposing it upstream. Three components:

1. Central registry — .devtrail/follow-ups-backlog.md

A single markdown file with YAML frontmatter + structured body:

---
last_scan: 2026-05-06
schema_version: v0
fully_extracted_ailogs:
  - AILOG-2026-04-11-001
  - AILOG-2026-04-12-001
  ... (47 total)
---

## Bucket: ready
### FU-001 — <description>
- **Origin**: AILOG-NNNN-NN-NN-NNN <context>
- **Status**: open | in-progress | closed | superseded
- **Trigger**: ready | <calendar date> | when <X> | <other>
- **Destination**: <Charter id or "operations" or "Phase 6+">
- **Cost**: <effort estimate>
- **Notes**: <free-form context>

Five buckets organize the entries by trigger type:

• ready — actionable now, no dependency on external trigger
• time-triggered — calendar-based trigger (audit cycle, periodic review)
• charter-triggered — gated on a future Charter that touches the relevant area
• phase-blocked — gated on a future component (Phase 6+ in Sentinel's case)
• operational — manual operator decision or external system action

The fully_extracted_ailogs list in frontmatter is the load-bearing metadata for the drift detection — see component 2.

2. Drift detection script — scripts/check-followups-drift.sh

A bash script that scans AILOGs and detects ones with §Follow-ups or R<N> (new, not in Charter) entries that are NOT in fully_extracted_ailogs. Three modes:

• Default: scans AILOGs modified in git diff origin/main..HEAD. Warns on drift, exits 1.
• --apply: same scan + auto-appends new entries to backlog under ## Bucket: ready with auto-numbered FU-NNN ids, AND appends the AILOG id to fully_extracted_ailogs. Operator reclassifies bucket later.
• --scan-all: scan every AILOG (periodic full sweep).

Per-AILOG granularity (vs per-bullet) is the load-bearing design choice. Per-bullet matching requires fingerprinting (text hashing or fuzzy comparison), which produces false positives whenever a backlog entry paraphrases the AILOG bullet — and curated backlog entries always paraphrase. Per-AILOG matching is robust: an AILOG is either fully extracted (trust the backlog) or it isn't (extract everything). When a follow-up is added to an already-extracted AILOG post-Charter close, operator can manually remove the AILOG from the list to force re-scan — this is intentional because most AILOGs are write-once after Charter close.

Empirical verification (CHARTER-12 §Tasks #5):

• Clean state → exit 0 ✓
• Brand new AILOG with follow-ups → exit 1 + clear stderr ✓
• --apply on new AILOG → backlog + frontmatter updated atomically ✓
• Post-cleanup → exit 0 ✓
• --scan-all against initial backlog (47 AILOGs all in fully_extracted_ailogs) → exit 0 ✓

3. Agent integration — CLAUDE.md instruction

Two additions to the Sentinel CLAUDE.md:

• Pre-commit checklist gains: "Created or modified any AILOG with ## Follow-ups or R<N> (new, not in Charter) entries? → Run scripts/check-followups-drift.sh --apply to extend .devtrail/follow-ups-backlog.md in the same commit."
• Session-start awareness: "At session start, glance at .devtrail/follow-ups-backlog.md to know what's pending across the project."

This makes the agent the primary maintainer of the registry; the script is the verification + automation layer; the operator is the periodic reviewer (re-bucketing entries, marking closed, etc.).

Sugerencia (upstream framework)

The local implementation works for Sentinel but has limitations that only the upstream framework can address. Three concrete additions to the devtrail CLI would generalize the pattern for all adopters, mirroring the existing devtrail charter trio:

devtrail followups list [--bucket=<name>] [--status=<state>]

Enumerate open follow-ups across the central registry. Filterable by bucket and status. Output formats: text (default, human-readable) and json (machine-readable for CI / dashboards).

devtrail followups close <FU-NNN> [--reason=<text>]

Mark a follow-up as closed (with optional reason text). Updates the registry's status field; does NOT delete the entry (keeps the history visible). Optionally records the closing PR / Charter via --by-charter=CHARTER-NN or --by-pr=<url>.

devtrail followups drift [--apply] [--scan-all]

Detect AILOGs with un-extracted follow-ups, mirroring the local script but with framework-level conveniences:

• AILOG-aware: reads originating_ailogs from Charter frontmatter to know which AILOGs belong to which Charter, optionally suppressing drift on AILOGs already covered by a closed Charter.
• Bucket-classification heuristic: instead of dumping new entries to "Bucket: ready" for manual reclassification, use the AILOG's frontmatter tags + Charter's frontmatter effort_estimate to suggest a bucket automatically. Operator can override.
• Schema validation: the registry follows a JSON Schema (proposed .devtrail/schemas/follow-ups-backlog.schema.v0.json); CLI validates on every write.

Convention alignment with existing trio

The existing devtrail charter CLI exposes new / status / drift / close for Charter lifecycle. The proposed devtrail followups would mirror this: list (≈ status), drift, close, plus the implicit add (via auto-extraction during Charter close). This keeps the surface area predictable for adopters already familiar with Charter commands.

Integration with audit cycle

When an external auditor reports findings via devtrail charter audit --merge-reports, real-debt findings that were NOT remediated atomically pre-close should auto-create follow-up entries in the central registry (with Origin: <auditor-id> via CHARTER-NN audit cycle). Today these findings live only in the audit review.md; surfacing them in the central registry closes a known gap from the v1 audit cycle (CHARTER-07 → CHARTER-08).

Adoption strategy

This RFC proposes a phased adoption that lets adopters opt in incrementally:

• Phase 1 (no framework change): adopters can implement locally exactly as Sentinel did under CHARTER-12. The .devtrail/follow-ups-backlog.md shape + bash script + CLAUDE.md instruction is reproducible. Sentinel's CHARTER-12 PR is the reference implementation.
• Phase 2 (framework helper): framework ships a devtrail followups subcommand that reads the same .devtrail/follow-ups-backlog.md shape and provides the list / close / drift operations. Adopters who already implemented Phase 1 migrate by deleting their local scripts/check-followups-drift.sh and switching the CLAUDE.md instruction to invoke devtrail followups drift --apply instead.
• Phase 3 (audit cycle integration): framework auto-extracts unresolved auditor findings into the central registry as part of devtrail charter audit --merge-reports. Closes the gap between audit findings and the central backlog.

Each phase is independently valuable. Sentinel adopter is at Phase 1 today; willing to migrate to Phase 2 if framework adopts the pattern.

References

• Sentinel StrangeDaysTech/sentinel CHARTER-12 PRs: https://github.com/StrangeDaysTech/sentinel/pull/53 (implementation, merged 2026-05-06) + https://github.com/StrangeDaysTech/sentinel/pull/54 (close-out, merged 2026-05-06)
• Sentinel CHARTER-08 → CHARTER-11 chain (the empirical evidence for the gap):
  • https://github.com/StrangeDaysTech/sentinel/pull/45 (CHARTER-08)
  • https://github.com/StrangeDaysTech/sentinel/pull/47 (CHARTER-09 — first chain link)
  • https://github.com/StrangeDaysTech/sentinel/pull/49 (CHARTER-10 — second link)
  • https://github.com/StrangeDaysTech/sentinel/pull/51 (CHARTER-11 — third link, chain stabilized)
• Prior Sentinel-driven RFCs in StrangeDaysTech/devtrail:
  • issue RFC: Canonical approval workflow for review_required documents (AIDEC, ETH, MCARD, ADR) #67 — multi-approval pattern (precedent for retroactive approval workflow)
  • issue Discussion: O3 — should drift print 'INFO: N paths suppressed' always-on, --verbose-only, or --no-ailog-suppress-only? #91 — audit cycle v1 (precedent for tool-use auditors)
  • issue audit-skills v0: substantive-depth limitations in primary-adopter cycle (Sentinel CHARTER-07) + prompt resolver HTML-comment bug #102 — cli tool-use (precedent for shell-access auditors)

This RFC follows the same pattern: empirical observation in Sentinel → local implementation as proof of concept → upstream proposal with adoption path.

[montfort]

Resolved via option A — pattern documented in governance docs (fw-4.10.0)

The follow-ups backlog pattern is now upstream as a reproducible convention. Shipped in Framework 4.10.0 (PR #112, tag fw-4.10.0).

What landed:

• New governance pattern document dist/.devtrail/00-governance/FOLLOW-UPS-BACKLOG-PATTERN.md (EN + ES + zh-CN) covering: registry shape (.devtrail/follow-ups-backlog.md with fully_extracted_ailogs frontmatter), 5 buckets (ready / time-triggered / charter-triggered / phase-blocked / operational), per-entry schema, drift detection script (3 modes), per-AILOG vs per-bullet rationale, agent integration block for CLAUDE.md / AGENT.md, adoption walkthrough, reference implementation pointer to Sentinel CHARTER-12 (PRs sentinel#53 / fix(cli): allocate table columns by water-fill (3.2.5) #54), and v0 open questions.
• Pointer entries in AGENT-RULES.md (new "Patterns" section) and QUICK-REFERENCE.md (new "Patterns" table) — adopters discover the pattern naturally when reading governance.
• CHANGELOG entry under ## Framework 4.10.0.

What we deliberately did NOT do (and why):

• No devtrail followups subcommand — the pattern is v0, validated in N=1 domain (Sentinel). Cristalization to CLI is deferred until a second adopter exercises the pattern. Honoring the principle that we don't crystallize prematurely. The bash script + markdown registry shape is fully portable; a future cristalization migrates adopters by deleting their local script and switching the agent instruction.
• No relationship analysis with the deferred mapa-cognitivo proposal — flagged as a needed conversation but deferred until more empirical experience accumulates across both pattern and mapa.
• No CLI version bump — CLI stays at 3.10.0; this is a docs-only release.

Adopter impact:

Adopters at ~20+ AILOGs can now run devtrail update-framework to pull the new doc + the pointers from AGENT-RULES.md / QUICK-REFERENCE.md. Sentinel is at Phase 1 (per the RFC's adoption strategy) — when Phase 2 (framework helper) eventually ships, the migration is a delete-and-update, not a rewrite.

Closing this RFC as resolved by option A. If/when a second adopter accumulates a comparable backlog and exercises the pattern, we revisit Phase 2 (CLI helper) with two-domain empirical evidence.

Thanks for the well-grounded RFC and the reference implementation under sentinel#53/#54 — this PR was almost entirely a faithful documentation pass over your shipped work.