reflector: port karpathy-loop's bounded reflection pattern#2
Merged
Conversation
added 2 commits
April 19, 2026 11:36
Borrow symbiont-karpathy-loop's post-phase reflector idea into the pen
test pipeline. Between phases the controller now invokes a bounded
reflector agent that reads the phase's findings and writes
subject-predicate-object lessons to a new `knowledge` table. The next
phase's agent recalls those lessons before planning so learning flows
forward across the engagement without any agent mutating another's
tools or policy.
Boundary is enforced the same way as in karpathy-loop: Cedar's
`reflector.cedar` uses `forbid ... unless` defensive negation to keep
the reflector's surface bounded to `store_knowledge`,
`recall_knowledge`, and `query_findings`. Every scan/enum/exploit/post-
exploit action is rejected at the gate even if someone later widens a
permit elsewhere. The reflector's DSL capabilities list is the second
layer: it declares nothing it could use to act on targets.
Changes:
- db/schema.sql, src/db.rs: `knowledge` table + typed insert/recall
- src/knowledge_tools.rs: `store_knowledge` + `recall_knowledge` MCP
tools, gated on `PenTest::KnowledgeStore`
- agents/reflector.dsl + policies/reflector.cedar: bounded agent
- agents/{recon,enum,vuln-assess,exploit,post-exploit,reporter}.dsl:
add `recall_knowledge` capability + phase-entry recall
- agents/engagement-controller.dsl: invoke reflector after each phase
- policies/tool-authorization.cedar: authorize `recall_knowledge`
globally (LOW risk, read-only)
- policies/evidence.cedar: exempt knowledge tools from the pending-
tool-runs backlog check
- tests: db round-trip, reflector policy shape assertions, knowledge
tool registration checks
- AGENTS.md: updated agent hierarchy, key files, and tool roster
- Agent count 7 -> 8; hierarchy shows reflector between phases and reporter, with a short paragraph on what it does and how it's bounded. - MCP tool count 31 -> 35 in the architecture diagram. - Policy table adds reflector.cedar with its defensive forbid-unless note; count 7 -> 8. - Data layer section introduces the knowledge store next to findings and evidence, with one concrete triple example. - Repository structure shows reflector.dsl and knowledge_tools.rs. - Key design decisions gains a "bounded reflector" entry. - Capability comparison table gains a cross-phase-learning row.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Ports the reflection/knowledge-store pattern from
symbiont-karpathy-loopinto this pipeline. Between phases, a bounded reflector agent distils a phase's findings into subject-predicate-object lessons. The next phase's agent recalls those lessons before planning, so learning flows forward across an engagement without any agent mutating another's tools or policy.What's added
knowledgeSQLite table (db/schema.sql,src/db.rs) with subject-predicate-object triples,confidencein [0.0, 1.0], engagement + phase scoping.src/knowledge_tools.rs, Cedar resourcePenTest::KnowledgeStore):store_knowledge— reflector-only, gated byreflector.cedarrecall_knowledge— read-only, permitted for every phase agentreflectoragent (agents/reflector.dsl) withreflect()that reads phase findings, dedups against prior triples, writes new ones.policies/reflector.cedar) usesforbid ... unlessdefensive negation. Even if someone later widens a permit elsewhere, the global forbid keeps the reflector's surface bounded to{store_knowledge, recall_knowledge, query_findings}.recon/enum/vuln-assess/exploit/post-exploit/reporter) — each gains arecall_knowledgecapability and calls it at phase entry.agents/engagement-controller.dsl) now invokes the reflector after each phase.tool-authorization.cedarauthorizesrecall_knowledgeat LOW risk globally;evidence.cedarexempts the knowledge tools from the pending-tool-runs backlog check.README.mdandAGENTS.mdagent hierarchy, key files table, tool roster (now 35 tools across 8 modules).Why this pattern
The pen test pipeline already has retest comparisons (
compare_engagements) and a finding audit trail, but nothing carries procedural lessons across phases or engagements. The karpathy-loop repo demonstrates that a bounded reflector — strictly separated from the actor it teaches — can produce concrete, indexable lessons without becoming a recursive self-improver. That's a good fit here: later phases benefit from what earlier phases learned, with zero increase in any phase agent's capabilities.Defense in depth
The reflector is bounded at three layers:
reflector.dsldeclares only{store_knowledge, recall_knowledge, query_findings}. It has no way to name a scan/exploit tool.permit— narrowly scoped to those three tools.forbid ... unless— global deny onexecute_toolwith a small whitelist. Any future tool added outside the whitelist is rejected automatically.Plus explicit
forbidclauses forscan,exploit, andpost_exploitactions that any reader can skim.Test plan
cargo test --test db_tests knowledge_— insert/recall round-trip, per-engagement isolation, limit capcargo test --test tool_registration_tests knowledge_— tool count, Cedar resource, actions, no human gatecargo test --test reflector_policy_test— policy shape assertions (principal, forbid-unless, tool whitelist, action forbids)cargo test --test db_tests init_db_creates_tables—knowledgetable appears in schemacargo test --test tool_registration_tests total_tools_is_35— total count guardsymbi policy evaluate policies/reflector.cedaragainst a reflector principal requestingnmap_scan— should denysymbi policy evaluate policies/reflector.cedaragainst a reflector principal requestingstore_knowledge— should permitask(reflector, phase=X)is invoked after every phase and subsequent phase agents see non-emptyrecall_knowledgeoutputNotes for reviewers
cargo checkcouldn't be exercised locally becausesymbi-channel-adapter's relative path inCargo.tomlresolves to a location that doesn't exist on this machine — same for the main branch. All Rust additions mirror the existingevidence_tools.rs/db.rspatterns exactly, so the shape should hold. CI on the canonical repo layout should validate.