From 30c3547503b194cb64b58dad0860cf4ea982c99f Mon Sep 17 00:00:00 2001 From: Xiwei Pan Date: Fri, 13 Mar 2026 21:06:28 +0800 Subject: [PATCH] Improve fix-issue and fix-issue-batch skills - fix-issue: Add changelog comment step (8b), require Warn results to be fixed (not just Fail), simplify re-check to inline instead of subagent - fix-issue-batch: Dispatch each issue as a subagent for fresh context, present all questions/options as text for human to answer at once Addresses #636 (batch agent cutting corners). Co-Authored-By: Claude Opus 4.6 --- .claude/skills/fix-issue-batch/SKILL.md | 242 +++++++++++++++++++++ .claude/skills/fix-issue/SKILL.md | 275 ++++++++++++++++++++++++ 2 files changed, 517 insertions(+) create mode 100644 .claude/skills/fix-issue-batch/SKILL.md create mode 100644 .claude/skills/fix-issue/SKILL.md diff --git a/.claude/skills/fix-issue-batch/SKILL.md b/.claude/skills/fix-issue-batch/SKILL.md new file mode 100644 index 000000000..3474130a1 --- /dev/null +++ b/.claude/skills/fix-issue-batch/SKILL.md @@ -0,0 +1,242 @@ +--- +name: fix-issue-batch +description: Batch-fix [Model] or [Rule] issues from the GitHub Project board Backlog column — calls /fix-issue on each one sequentially +--- + +# Fix Issue Batch + +Iterate through `[Model]` or `[Rule]` issues **from the Backlog column of the GitHub Project board** and call `/fix-issue` on each one sequentially. + +## Invocation + +``` +/fix-issue-batch +``` + +## Constants + +GitHub Project board IDs (same as fix-issue / project-pipeline): + +| Constant | Value | +|----------|-------| +| `PROJECT_NUMBER` | `8` | +| `PROJECT_OWNER` | `CodingThrust` | +| `PROJECT_ID` | `PVT_kwDOBrtarc4BRNVy` | +| `STATUS_FIELD_ID` | `PVTSSF_lADOBrtarc4BRNVyzg_GmQc` | +| `STATUS_READY` | `61e4505c` | + +## Process + +```dot +digraph fix_batch { + rankdir=TB; + "Parse argument" [shape=box]; + "Fetch project board Backlog" [shape=box]; + "Filter & sort" [shape=box]; + "Print queue" [shape=box]; + "Pick next issue" [shape=diamond]; + "Call /fix-issue N" [shape=box]; + "Done" [shape=doublecircle]; + + "Parse argument" -> "Fetch project board Backlog"; + "Fetch project board Backlog" -> "Filter & sort"; + "Filter & sort" -> "Print queue"; + "Print queue" -> "Pick next issue"; + "Pick next issue" -> "Call /fix-issue N" [label="remaining"]; + "Pick next issue" -> "Done" [label="none left"]; + "Call /fix-issue N" -> "Pick next issue"; +} +``` + +--- + +## Step 1: Parse Argument + +Accept one argument: `model` or `rule` (case-insensitive). + +- `model` → filter for issues with `[Model]` in the title +- `rule` → filter for issues with `[Rule]` in the title + +If no argument or invalid argument → stop with: "Usage: `/fix-issue-batch `" + +--- + +## Step 2: Fetch Issues from Project Board Backlog + +Fetch all items from the GitHub Project board and filter to the **Backlog** column: + +```bash +gh project item-list 8 --owner CodingThrust --format json --limit 500 +``` + +From the JSON result: +1. Filter items where `status == "Backlog"` +2. Keep only items whose `content.title` starts with `[Model]` or `[Rule]` (matching the argument) +3. For each item, extract: issue number (`content.number`), title (`content.title`), and project item ID (`id`) + +Then fetch labels for each matching issue (needed for categorization): + +```bash +gh issue view --repo CodingThrust/problem-reductions --json labels +``` + +Or batch-fetch with a single search query to avoid N+1: + +```bash +gh issue list --repo CodingThrust/problem-reductions \ + --state open --search "[Model] in:title" \ + --json number,title,labels --limit 200 +``` + +Cross-reference the board items with the label data to build the final list. + +--- + +## Step 3: Filter and Sort + +From the Backlog issues: + +1. **Categorize by current label status:** + - `already-good` — has `Good` label + - `has-failures` — has at least one failure label (`PoorWritten`, `Wrong`, `Trivial`, `Useless`) + - `needs-check` — no check-issue comment yet (no `Good`/`PoorWritten`/`Wrong`/`Trivial`/`Useless` label) +2. **Sort by priority then issue number:** `already-good` first, then `has-failures`, then `needs-check`. Within each group, sort by issue number ascending (oldest first). This prioritizes issues that already have a check report and are closest to being ready. + +--- + +## Step 4: Print Queue + +Show the user what will be processed: + +``` +## Fix queue: [Model] issues (N total) + +| # | Issue | Title | Status | +|---|-------|-------|--------| +| 1 | #235 | [Model] SteinerTree | already-good | +| 2 | #233 | [Model] StrongConnectivityAugmentation | has-failures (PoorWritten) | +| 3 | #234 | [Model] FeedbackVertexSet | needs-check | +| ... | ... | ... | ... | + +Priority: already-good → has-failures → needs-check. +Processing N issues total. +``` + +Then ask the user to confirm before starting: + +> Ready to start? I'll process each issue with `/fix-issue`, one at a time. +> +> 1. **Start** — begin processing from the first issue +> 2. **Start from #N** — skip to a specific issue number +> 3. **Cancel** + +--- + +## Step 5: Process Each Issue + +**CRITICAL:** Each issue MUST be dispatched as a **subagent** for analysis, giving it a fresh context window to prevent cutting corners. The subagent does NOT interact with the human — it returns a structured report, then the main agent presents it for human decisions. + +For each issue in the queue (priority order: `already-good` → `has-failures` → `needs-check`): + +### 5a: Check prerequisite + +If no comment starting with `## Issue Quality Check` exists, run `/check-issue ` first. + +### 5b: Dispatch analysis subagent + +``` +Agent tool: + subagent_type: "general-purpose" + description: "Analyze issue #" + prompt: | + Analyze GitHub issue # for fix-issue. + 1. Fetch the issue: gh issue view --json title,body,labels,comments + 2. Find the most recent "## Issue Quality Check" comment + 3. Parse all Fail and Warn results (warnings are NOT ignorable) + 4. For each issue, classify as mechanical or substantive + 5. For mechanical issues: apply the fix to a draft body + 6. For substantive issues: prepare 2-3 concrete options with your recommendation + + Return EXACTLY this format: + + ## Analysis for #: + + ### Auto-fixes applied + | # | Section | Issue | Fix | + |---|---------|-------|-----| + | 1 | ... | ... | ... | + + ### Questions for human + **Q1: <topic>** + <description of the problem> + - (a) <option 1> ← recommended + - (b) <option 2> + - (c) <option 3> + + **Q2: ...** + + ### Draft body + <full updated issue body with mechanical fixes applied, substantive issues marked as `[PENDING Q1]`> +``` + +### 5c: Present to human + +Print the subagent's report and ask the human to answer all questions at once: + +``` +## Issue #<NUMBER> (<current>/<total>): <title> + +<auto-fixes table from subagent> + +<questions from subagent> + +Please answer the questions above (e.g. "Q1: a, Q2: b"), or type "skip" to skip this issue. +``` + +### 5d: Apply answers and finalize + +After human responds: +- If **"skip"**: move to next issue +- Otherwise: apply the human's choices to the draft body, re-check (run 4 quality checks inline), then finalize on GitHub (edit body, post changelog comment, update labels, move to Ready — see fix-issue Steps 6–8) + +### 5e: Continue + +Print progress and ask whether to continue: + +``` +Done #<NUMBER>. (<current>/<total> complete, <remaining> remaining) +Next: #<NEXT_NUMBER> <next_title> +``` + +> 1. **Continue** — process the next issue +> 2. **Skip next** — skip the next issue and continue to the one after +> 3. **Stop** — end the batch here and print summary + +--- + +## Step 6: Summary + +After all issues are processed, print a summary: + +``` +## Batch fix complete + +| Result | Count | Issues | +|--------|-------|--------| +| Fixed & moved to Ready | 7 | #233, #234, #235, #236, #237, #238, #240 | +| Skipped (by user) | 1 | #239 | +| Total | 8 | | +``` + +--- + +## Common Mistakes + +| Mistake | Fix | +|---------|-----| +| Fetching all open issues instead of board | Only process issues from the **Backlog** column of GitHub Project #8 | +| Skipping already-good issues | Process ALL Backlog issues; `already-good` are processed first (highest priority) | +| Not running check-issue first | If no check report exists, run `/check-issue` before `/fix-issue` | +| Processing in random order | Always sort by issue number ascending within priority groups | +| Continuing after user cancels | Respect cancel/skip requests immediately | +| Missing project scopes | Run `gh auth refresh -s read:project,project` if board access fails | diff --git a/.claude/skills/fix-issue/SKILL.md b/.claude/skills/fix-issue/SKILL.md new file mode 100644 index 000000000..1091c0a0d --- /dev/null +++ b/.claude/skills/fix-issue/SKILL.md @@ -0,0 +1,275 @@ +--- +name: fix-issue +description: Fix quality issues found by check-issue — auto-fixes mechanical problems, brainstorms substantive issues with human, then re-checks and moves to Ready +--- + +# Fix Issue + +Fix errors and warnings from a `check-issue` report. Auto-fixes mechanical issues, brainstorms substantive ones with the human, edits the issue body, re-checks once, then asks the human to approve or iterate. + +## Invocation + +``` +/fix-issue <issue-number> +``` + +## Constants + +GitHub Project board IDs: + +| Constant | Value | +|----------|-------| +| `PROJECT_ID` | `PVT_kwDOBrtarc4BRNVy` | +| `STATUS_FIELD_ID` | `PVTSSF_lADOBrtarc4BRNVyzg_GmQc` | +| `STATUS_READY` | `61e4505c` | + +## Process + +```dot +digraph fix_issue { + rankdir=TB; + "Fetch issue + check comment" [shape=box]; + "Parse failures & warnings" [shape=box]; + "Auto-fix mechanical issues" [shape=box]; + "Present auto-fixes to human" [shape=box]; + "Brainstorm substantive issues" [shape=box]; + "Re-check locally" [shape=box]; + "Ask human" [shape=diamond]; + "Edit GitHub + update labels + move to Ready" [shape=box, style=filled, fillcolor="#ccffcc"]; + "Ask what to change (free-form)" [shape=box]; + "Apply changes + re-check locally" [shape=box]; + + "Fetch issue + check comment" -> "Parse failures & warnings"; + "Parse failures & warnings" -> "Auto-fix mechanical issues"; + "Auto-fix mechanical issues" -> "Present auto-fixes to human"; + "Present auto-fixes to human" -> "Brainstorm substantive issues"; + "Brainstorm substantive issues" -> "Re-check locally"; + "Re-check locally" -> "Ask human"; + "Ask human" -> "Edit GitHub + update labels + move to Ready" [label="1: looks good"]; + "Ask human" -> "Ask what to change (free-form)" [label="2: modify again"]; + "Ask what to change (free-form)" -> "Apply changes + re-check locally"; + "Apply changes + re-check locally" -> "Ask human"; +} +``` + +--- + +## Step 1: Fetch Issue and Check Comment + +```bash +gh issue view <NUMBER> --json title,body,labels,comments +``` + +- Detect issue type from title: `[Rule]` or `[Model]` +- Find the **most recent** comment that starts with `## Issue Quality Check` — this is the check-issue report +- If no check comment found, STOP with message: "No check-issue report found. Run `/check-issue <NUMBER>` first." + +--- + +## Step 2: Parse Failures and Warnings + +Extract from the check comment's summary table: + +| Field | How to extract | +|-------|---------------| +| Check name | First column (Usefulness, Non-trivial, Correctness, Well-written) | +| Result | Second column (Pass / Fail / Warn) | +| Details | Third column (one-line summary) | + +Then parse the detailed sections below the table for specifics: +- Each `### <Check Name>` section contains the full explanation +- The `#### Recommendations` section (if present) contains suggestions + +Build a structured list of **all issues to fix** — include both `Fail` **and** `Warn` results. Warnings are not ignorable; they must be resolved before moving to Ready. + +Tag each issue as: +- `mechanical` — can be auto-fixed without human input +- `substantive` — requires human brainstorming + +### Classification Rules + +**Mechanical** (auto-fixable): + +| Issue pattern | Fix strategy | +|--------------|-------------| +| Undefined symbol in overhead/algorithm | Add definition derived from context (e.g., "let n = \|V\|") | +| Inconsistent notation across sections | Standardize to the most common usage in the issue | +| Missing/wrong code metric names | Look up correct names via `pred show <target> --json` → `size_fields` | +| Formatting issues (broken tables, missing headers) | Reformat to match issue template | +| Incomplete `(TBD)` in fields derivable from other sections | Fill from context | +| Incorrect DOI format | Reformat to `https://doi.org/...` | + +**Substantive** (brainstorm with human): + +| Issue pattern | Why human input needed | +|--------------|----------------------| +| Naming decisions (optimization prefix, CamelCase choice, too long name) | Codebase convention judgment call | +| Missing or incorrect complexity bounds | Requires literature verification | +| Missing type dependencies | Architectural decision about codebase | +| Incorrect mathematical claims | Domain expertise needed | +| Incomplete reduction algorithm | Core technical content | +| Incomplete or trivial example | Needs meaningful design, provide 3 options for the human to choose from | +| Decision vs optimization framing | Check associated `[Rule]` issues first — if a rule targets the decision version, implement that; if it targets optimization, implement that; if both exist, split into two separate model issues. Problem modeling choice | +| Ambiguous overhead expressions | Requires understanding the reduction | + +--- + +## Step 3: Auto-Fix Mechanical Issues + +For each `mechanical` issue: + +1. Identify the exact section in the issue body that needs editing +2. Apply the fix +3. Record what was changed (for presenting to human in Step 4) + +Use `pred show <problem> --json` to look up: +- Valid problem names and aliases +- `size_fields` for correct metric names +- Existing variants and fields + +**Do NOT edit the issue on GitHub yet** — collect all fixes (mechanical + substantive) first. + +--- + +## Step 4: Present Auto-Fixes to Human + +Print a summary of all mechanical fixes applied: + +``` +## Auto-fixes applied + +| # | Section | Issue | Fix | +|---|---------|-------|-----| +| 1 | Size Overhead | Symbol `m` undefined | Added ... | +``` + +Then present the substantive issues that need discussion: + +``` +## Issues requiring your input + +1. **Decision vs optimization:** ... +``` + +--- + +## Step 5: Brainstorm Substantive Issues + +For each substantive issue, present it to the human **one at a time**: + +1. State the problem clearly +2. Offer 2-3 concrete options when possible (with your recommendation) +3. Wait for the human's response +4. Apply the chosen fix to the draft issue body + +Use web search if needed to help resolve issues: +- Literature search for correct complexity bounds +- Verify algorithm claims +- Find better references + +After all substantive issues are resolved, show the human the complete updated issue body (or a diff summary if the body is long). + +--- + +## Step 6: Re-Check Locally + +Re-run the 4 quality checks (Usefulness, Non-trivial, Correctness, Well-written) from `check-issue` against the **draft issue body** (not yet pushed to GitHub). Use `pred show`, `pred path`, web search as needed. Do NOT post a GitHub comment. + +Print results to the human as a summary table (Check / Result / Details). + +--- + +## Step 7: Ask Human for Decision + +Use `AskUserQuestion` to present the options: + +> The issue has been re-checked locally. What would you like to do? +> +> 1. **Looks good** — I'll push the edits to GitHub, update labels, and move it to Ready +> 2. **Modify again** — tell me what else you'd like to change + +### If human picks 2: Modify Again + +Ask the human (free-form) what they want to change: + +> What would you like to modify? Describe the changes you want. + +Apply the requested changes to the draft issue body, re-check locally (Step 6), then ask again (Step 7). Repeat until the human picks "Looks good". + +--- + +## Step 8: Finalize (on "Looks good") + +Only reached when the human approves. Now push everything to GitHub. + +### 8a: Edit the issue body + +```bash +# Write updated body to temp file +cat > /tmp/fix_issue_body.md <<'BODYEOF' +$UPDATED_BODY +BODYEOF + +gh issue edit <NUMBER> --body-file /tmp/fix_issue_body.md +``` + +### 8b: Comment on the issue with a changelog + +Post a comment summarizing what was changed, so reviewers can see the diff at a glance: + +```bash +gh issue comment <NUMBER> --body "$(cat <<'EOF' +## Fix-issue changelog + +- <bullet for each change made, e.g. "Fixed undefined symbol `m` in Size Overhead section"> +- ... + +Applied by `/fix-issue`. +EOF +)" +``` + +### 8c: Update labels + +```bash +gh issue edit <NUMBER> --remove-label "Useless,Trivial,Wrong,PoorWritten" 2>/dev/null +gh issue edit <NUMBER> --add-label "Good" +``` + +### 8d: Move to Ready on project board + +```bash +# Find the project item ID for this issue +ITEM_ID=$(gh project item-list 8 --owner CodingThrust --format json | \ + jq -r '.items[] | select(.content.number == <NUMBER>) | .id') + +gh project item-edit \ + --id "$ITEM_ID" \ + --project-id PVT_kwDOBrtarc4BRNVy \ + --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ + --single-select-option-id 61e4505c +``` + +### 8e: Confirm + +```text +Done! Issue #<NUMBER>: + - Body updated on GitHub + - Labels: removed failure labels, added "Good" + - Board: moved to Ready +``` + +--- + +## Common Mistakes + +| Mistake | Fix | +|---------|-----| +| Pushing to GitHub before human approves | All edits stay local until human picks "Looks good" | +| Hallucinating paper content for complexity bounds | Use web search; if not found, say so and ask human | +| Using `pred show` on a problem that doesn't exist yet | Check existence first; for new problems, skip metric lookup | +| Overwriting human's original content | Preserve original text; only modify the specific sections flagged | +| Not preserving `<!-- Unverified -->` markers | Keep existing provenance markers; add new ones for AI-filled content | +| Running check-issue more than once per iteration | Re-check exactly once after edits, then ask human | +| Closing the issue | Never close. Labels and board status only | +| Force-pushing or modifying git | This skill only edits GitHub issues via `gh`. No git operations |