general-audit lessons: 2026-04-27 (run 2)

[intendednull]

Lessons from /general-audit run @ 00aa515 (#437). Second run on same day as #413 (yesterday); built on lessons from #426.

What worked (from #426 suggestions, now validated)

• Lesson Comprehensive multi-peer E2E tests across all browsers #3 applied: skipped 8-agent fan-out entirely. Single orchestrator-direct sweep. Result: 2 real findings filed in minutes. No agent timeouts (because no agents). Trade-off acceptable for small diff.
• Lesson UX navigation overhaul: tabbed settings, confirm dialogs, context menu, command palette #4 applied: pre-fetched existing-issue lists (/tmp/audit-issues.txt 17 entries, /tmp/all-quality-issues.txt 89 entries). Dedup happened inline without per-grep round trips. Caught 5 dup candidates immediately ([WS-1] Web: js_sys::eval(format!()) for pinned-message scroll uses band-aid sanitization #425, [TD-14] anyhow used in 8 library crates contradicts CLAUDE.md convention #332, [SEC-V-06] TopicAnnounce.topics: Vec<String> has no element-count cap; enables relay CPU amplification + topic-slot exhaustion #235, [SEC-A-09] Agent HTTP server defaults to TokenScope::Full with no CLI flag #311, deps cluster).
• Lesson Add deep-review skill for iterative code review #5 applied: orchestrator ran cargo-audit directly. All 5 warnings already in CI ignore list w/ tracked issue numbers. Zero agent needed.
• Direct grep sweep covered the same surface as last run's 8 agents for a tiny diff. Patterns swept: unsafe, dbg!/eprintln!, locks-without-lock-ok, panic! in prod, eval/innerHTML, TODO/FIXME, anyhow in pure libs, Vec caps, .ok() swallow.

What didn't

• Diff was so small (6 commits, 4 files) that the audit was mostly redundant w/ general-audit: main @ 401e2fc (2026-04-27) #413. Same-day re-runs produce diminishing returns. AUD-2 (TokenScope::ReadOnly leaks join_links) was missed by general-audit: main @ 401e2fc (2026-04-27) #413's 8-agent run despite being unchanged code — proving the value of some fresh sweep, but the 1-day cadence is overkill.
• AUD-2 should have been caught by yesterday's sec-authperm agent but that agent was one of the 7 that timed out. Lesson: when agents timeout, the gaps are real and need backfill — not just absorbed into "lessons learned."
• Worktrees skipped since no agents → setup time saved but no isolation for parallel runs if I'd needed them.

Suggested edits to .claude/skills/general-audit/SKILL.md

1. Add a "skip-window" rule. "If HEAD is within N commits AND M hours of last audit master issue, skip — diff-only review (current PR rules apply) instead of full re-audit. Suggest N=20, M=24h." Today's run produced 2 findings on a 4-file diff; full sweep mostly re-walked yesterday's ground.

2. Codify the orchestrator-only path as default for small diffs. Add: "Default to orchestrator-only sweep when diff < 50 files since last audit. Spawn agents only when diff is large OR when a specific concern needs deeper grep than a single context can hold."

3. When agents do fan out, fan out narrower. Per-crate, not per-concern. E.g. audit-crate-agent (one crate, all concerns) scales better than audit-sec-input (one concern, whole tree) because the per-crate context is bounded and the agent can write incrementally without breadth-first context blowup.

4. Document the "agent-timeout backfill" loop. When an agent times out and writes no findings, the orchestrator MUST sweep that concern manually before declaring the audit complete. Otherwise gaps compound across runs (AUD-2 missed two audits in a row).

5. Move the existing-issue pre-fetch step into the skill explicitly. Currently buried in lesson UX navigation overhaul: tabbed settings, confirm dialogs, context menu, command palette #4 prose; should be a numbered required step before any agent spawns or any greps run.

6. Drop the "8-agent fan-out" example from the skill entirely. The lessons-learned issues are now 2-for-2 on that pattern failing or being overkill. Replace w/ "1-3 narrow agents only when justified."

Numbers

• diff since last audit: 6 commits, 4 files (3 PRs)
• agents launched: 0 (vs 8 in general-audit: main @ 401e2fc (2026-04-27) #413)
• agents timed out: 0 (vs 7/8 in general-audit: main @ 401e2fc (2026-04-27) #413)
• findings filed: 2 child + 1 master + 1 lessons = 4 issues (vs 13 in general-audit: main @ 401e2fc (2026-04-27) #413)
• new findings vs prior audits: 2 (1 missed by general-audit: main @ 401e2fc (2026-04-27) #413's timed-out agents, 1 newly visible)
• false-positive findings dropped: 0 (all surviving findings re-verified)
• dup candidates caught at synthesis: 5 (correctly suppressed)
• runtime: ~minutes (vs hours w/ agent thrash)
• cargo-audit: clean vs CI ignore list

Action

Human (or follow-up routine) to fold suggested edits into .claude/skills/general-audit/SKILL.md.

Critical: edit #4 (agent-timeout backfill) — AUD-2 sat undetected for one full audit cycle because nobody backfilled the timed-out sec-authperm agent's concern.

[intendednull]

already-fixed-upstream. suggested edits fold into .claude/skills/general-audit/SKILL.md via commit 8d91a11 docs(skill): fold audit lessons #426 #438 #477 #493 into general-audit.

skip-window rule, orchestrator-direct default for small diffs, narrower agent fan-out, agent-timeout backfill loop, pre-fetch step codified, drop 8-agent example — all addressed. close completed.

closing under /resolving-issues run on claude/friendly-maxwell-cVhhm.

---

Generated by Claude Code