Skip to content

feat(ce-review): add compact returns to reduce orchestrator context during merge#535

Merged
tmchow merged 9 commits intomainfrom
tmchow/review-mapreduce-eval
Apr 8, 2026
Merged

feat(ce-review): add compact returns to reduce orchestrator context during merge#535
tmchow merged 9 commits intomainfrom
tmchow/review-mapreduce-eval

Conversation

@tmchow
Copy link
Copy Markdown
Collaborator

@tmchow tmchow commented Apr 8, 2026

Summary

Sub-agents now produce two outputs: a full-detail JSON artifact file on disk and a compact merge-tier-only return to the orchestrator. This reduces orchestrator context during Stage 5 merge while preserving full detail for headless output enrichment and debugging.

Inspired by the work originally by @Drewx-Design in #157.

Sub-agent execution:
  1. Analyze diff (unchanged)
  2. Write full JSON (all fields) -> .context/ce-review/{run_id}/{reviewer}.json
  3. Return compact JSON (merge-tier only) -> orchestrator

Orchestrator:
  Stage 5: Merge on compact returns (title, severity, file, line,
           confidence, routing fields, suggested_fix)
  Stage 6: For headless mode, enrich surviving findings with
           why_it_matters/evidence from disk

Changes

File Change
subagent-template.md Two-output contract: artifact file (when run_id present) + compact return (always). Read-only exception for .context/ write. {run_id} and {reviewer_name} variables added.
findings-schema.json _meta.return_tiers documenting merge-tier vs detail-tier field split. Schema required array unchanged (enforces full fields for artifact file).
SKILL.md Run-id generation before dispatch. Stage 5 validates merge-tier fields only (not full schema). Headless detail enrichment reads from per-agent artifact files using file + line lookup. Fixes pre-existing duplicate step numbering (5,5,6 -> 5,6,7).
persona-catalog.md Adversarial reviewer trigger now file-type aware -- counts only executable code lines toward the >=50 line threshold, not prose/instruction Markdown or JSON schemas.

Design decisions

  • suggested_fix stays in compact returns -- it's small (often null), directly supports the orchestrator's auto-apply decision in interactive mode, and doesn't meaningfully impact context savings. Without it, the orchestrator lacks fix context and hesitates on safe_auto routing. why_it_matters and evidence (the bulky fields) remain detail-tier only.
  • Merge stays in the orchestrator -- not delegated to a separate judge agent. A merge agent adds latency and fragility for marginal benefit.
  • Graceful degradation -- if artifact file write fails, the compact return still provides everything merge needs. Headless output loses Why/Evidence lines for that finding and notes the gap in Coverage.
  • file + line lookup for enrichment (not title) -- Stage 5 dedup applies normalize(title) which could alter the title string, breaking a title-based lookup.
  • File-type aware adversarial selection -- adversarial techniques (race conditions, cascade failures) target runtime code. Pure prose/instruction diffs don't benefit from this lens. High-risk topics (auth, payments) still trigger regardless of file type.

Test plan

  • Self-review using the modified skill found and fixed: suggested_fix tier misplacement causing auto-apply hesitation, adversarial over-triggering on prose diffs
  • Verified graceful degradation when artifact file write is denied -- agents fell back to compact-return-only
  • Full end-to-end validation of artifact file writes requires a session with pre-approved Write permissions for .context/

Compound Engineering
Claude Code

…uring merge

Sub-agents now write full-detail JSON (including why_it_matters, evidence,
suggested_fix) to artifact files on disk and return only merge-tier fields
to the orchestrator. This reduces orchestrator context by ~24-37k tokens
during Stage 5 merge while preserving full detail for headless output
enrichment and debugging.

Key changes:
- Subagent template: two-output contract (artifact file + compact return),
  conditional on run_id presence for report-only compatibility
- Findings schema: return_tiers metadata documenting the merge/detail split
- SKILL.md: run-id generation before dispatch, merge-tier-only validation
  in Stage 5, headless detail enrichment from per-agent artifact files
  with file+line lookup for merged multi-reviewer findings

Also fixes pre-existing duplicate step numbering in Stage 5 (was 5,5,6
now correctly 5,6,7).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0d539b066b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
…are reviewer selection

suggested_fix is small and optional but directly supports the orchestrator's
auto-apply decision in interactive mode. Without it in the compact return, the
orchestrator lacks fix context and hesitates on safe_auto routing. Move it from
detail-tier to merge-tier across schema, template, and SKILL.md.

Also add file-type awareness to adversarial reviewer selection — prose/instruction
Markdown and JSON schemas don't benefit from runtime-focused techniques (race
conditions, cascade failures). Skip adversarial for pure prose diffs unless they
describe high-risk behavior like auth or payments.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f21cc461a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
tmchow and others added 2 commits April 8, 2026 00:03
…ookup (#535)

Stage 5 compact return validation now checks value constraints (severity
enum, autofix_class enum, owner enum, confidence range, type checks) in
addition to field presence. Previously only presence was checked, allowing
malformed values to survive into routing.

Detail enrichment now scopes artifact lookups to contributing reviewers
and disambiguates same-line findings by normalized title, preventing
cross-wiring when multiple findings share a file+line.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 38eedeb42d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
…r suggested_fix (#535)

Headless enrichment text still called suggested_fix a detail-tier field
after it was moved to merge-tier. Now correctly uses suggested_fix from
compact returns and only loads why_it_matters/evidence from artifact
files. Fallback path preserves suggested_fix even when artifacts fail.

Also adds a clarifying note in the subagent template explaining that
the embedded schema describes the artifact file format, not the compact
return -- resolving the contradiction between required fields in the
schema and the "Do NOT include" instruction.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a1cae0414d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
…el validation (#535)

Line 65 headless mode rules still listed suggested_fix as detail-tier
loaded from artifacts -- leftover from before the merge-tier move. Now
correctly lists it alongside other merge-tier fields.

Stage 5 validation now checks top-level compact return fields (reviewer,
residual_risks, testing_gaps) before processing findings, catching
malformed returns that would silently degrade agreement attribution.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f044cdca27

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
tmchow and others added 2 commits April 8, 2026 00:31
…tion (#535)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Step 2 relied on the reader carrying "include resolved threads" from
the gate-fires paragraph two steps earlier. In practice the orchestrator
grouped only new threads, missing clusters that span new + resolved
items. Restating the input set at the grouping step prevents this.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a36f9f4c1e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread plugins/compound-engineering/skills/ce-review/SKILL.md Outdated
…mat as sub-bullets (#535)

- Detail enrichment matching now uses file + line_bucket(line, +/-3)
  instead of exact file + line, matching Stage 5 dedup tolerance
- Reformatted detail enrichment section from prose to labeled sub-bullets
  for better agent adherence (Field tiers, Artifact matching, Reviewer
  order, No-match fallback)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@tmchow
Copy link
Copy Markdown
Collaborator Author

tmchow commented Apr 8, 2026

@codex review

@chatgpt-codex-connector
Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@tmchow tmchow merged commit a5ce094 into main Apr 8, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Apr 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant