diff --git a/.claude/skills/final-review/SKILL.md b/.claude/skills/final-review/SKILL.md index 271bbe122..4e7d96987 100644 --- a/.claude/skills/final-review/SKILL.md +++ b/.claude/skills/final-review/SKILL.md @@ -28,51 +28,73 @@ GitHub Project board IDs (for `gh project item-edit`): ## Workflow -### Step 0: Discover "Final review" PRs +### Step 0: Generate the Final-Review Report -If a specific PR number was given, use it directly. Otherwise: +Step 0 should be a single report-generation step. Do not manually unpack board selection, PR metadata, merge prep, or deterministic checks with shell snippets. -1. Fetch all project board items: - ```bash - gh project item-list 8 --owner CodingThrust --limit 500 --format json - ``` -2. Filter items where `Status == "Final review"`. Items may be Issues (with linked PRs) or PRs directly. -3. If none found, report "No items in the Final review column" and stop. -4. Pick the first one. If the item is an Issue, find the linked PR by searching open PRs for `Fix #` in the title. Print title, PR number, issue number, and URL. +```bash +REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner) +STATE_FILE=/tmp/problemreductions-final-review-selection.json +set -- python3 scripts/pipeline_skill_context.py final-review --repo "$REPO" --state-file "$STATE_FILE" --format text +if [ -n "${PR:-}" ]; then + set -- "$@" --pr "$PR" +fi +REPORT=$("$@") +printf '%s\n' "$REPORT" +``` + +The report is the Step 0 packet. It already includes **all** mechanical context: +- Selection: board item, PR number, linked issue, title, URL +- Recommendation Seed: suggested mode and deterministic blockers +- Subject +- Comment Summary (with linked issue context) +- Merge Prep +- Deterministic Checks +- Changed Files +- Diff Stat +- Full Diff + +Branch from the report: +- `Bundle status: empty` => stop with `No items in the Final review column` +- `Bundle status: ready` => continue normally (check warnings — a self-review warning means the reviewer is the PR author; flag it but do not block) +- `Bundle status: ready-with-warnings` => continue only with the narrow warning fallback described in the report + +When you need to take actions later, use the identifiers already printed in the report (`Board item`, `PR`, URL). If you absolutely need raw structured data for a corner case, rerun the same command with `--format json`, but do not rebuild Step 0 manually. + +### Step 1: Push the Merge with Main + +The context script already merged `origin/main` into the PR branch in the worktree. Read the report's `Merge Prep` section: -### Step 1: Gather PR context +- **Merge status: clean** — push the merge commit from the worktree: + ```bash + cd + git push + ``` +- **Merge status: conflicted** — note the conflicts. You can still continue with the review steps below and decide whether to resolve or hold in Step 6. +- **Merge prep failed** — skip this step; the warning fallback applies. -Collect all information needed for the review: +### Step 1a: Use the Bundled Review Context -1a. **PR metadata**: `gh pr view --json title,body,labels,files,additions,deletions,commits,headRefName,baseRefName,url,state` +**Trust the report.** The Step 0 report already contains all mechanical context — selection, comments, linked issue, merge prep, deterministic checks, changed files, diff stat, full diff, and `pred list` output. Do NOT re-fetch any of this data with separate tool calls (e.g., `gh api` for comments, `gh pr diff`, `gh pr view`, `pred list`). Extract everything directly from the report text. -1b. **PR diff**: `gh pr diff ` — read the full diff to understand all changes. +If the report is in the warning fallback path, keep the fallback narrow. Prefer hold/manual follow-up over reconstructing the whole pipeline inside the skill. -1c. **Linked issue**: Extract the linked issue number from PR body (look for `Fixes #N`, `Closes #N`, or `#N` references). Fetch issue body: `gh issue view --json title,body,labels` +### Step 1b: Comment Audit (REQUIRED) -1d. **Determine PR type**: From labels and title, classify as `[Model]` or `[Rule]`. - - For `[Model]`: identify the problem name being added - - For `[Rule]`: identify the source and target problem names +Final review must check the comment history before recommending merge. -1e. **Existing problems**: Run `pred list` (CLI tool, not MCP) to show all currently registered problems and reductions. This provides context for evaluating usefulness. +Use the report's `Comment Summary` and `Linked Issue Context` sections as the starting point. If you need to inspect the underlying comment threads in detail, do that only after reading the report. -1f. **Check for conflicts with main**: Run `gh pr view --json mergeable`. If there are merge conflicts, launch a subagent to merge `origin/main` into the PR branch (in a worktree) and push the merge commit. +Build a list of every actionable comment and classify each as: +- `addressed` +- `superseded / no longer applicable` +- `still open` -1g. **PR / issue comment audit (REQUIRED)**: Final review must check the comment history before recommending merge. - - Set `REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner)` - - Fetch and read: - - PR conversation comments: `gh api repos/$REPO/issues//comments` - - PR inline review comments: `gh api repos/$REPO/pulls//comments` - - PR review bodies: `gh api repos/$REPO/pulls//reviews` - - linked issue comments, if an issue exists - - Build a list of every actionable comment and classify each as: - - `addressed` - - `superseded / no longer applicable` - - `still open` - - Pay special attention to the `## Review Pipeline Report` comment. If it contains a `Remaining issues for final review` section, those items must be reviewed explicitly here. - - Do **not** recommend merge until every actionable comment has been dispositioned. +Pay special attention to the `## Review Pipeline Report` comment. If it contains a `Remaining issues for final review` section, those items must be reviewed explicitly here. -1h. **Comment status summary**: Prepare a short summary for later steps: +Do **not** recommend merge until every actionable comment has been dispositioned. + +Prepare a short summary for later steps: > **Comment Audit** > @@ -130,31 +152,11 @@ Use `AskUserQuestion` to confirm: ### Step 3b: File whitelist check -Check that the PR only touches files expected for its type. Any file outside the whitelist is flagged for review — it may be a legacy pattern or an unrelated change. - -**Whitelist for [Model] PRs:** -- `src/models//.rs` — model implementation -- `src/unit_tests/models//.rs` — unit tests -- `src/example_db/model_builders.rs` — canonical example registration -- `src/example_db/rule_builders.rs` — only if updating nonempty-style assertions -- `docs/paper/reductions.typ` — paper entry -- `docs/src/reductions/problem_schemas.json` — schema export -- `docs/src/reductions/reduction_graph.json` — graph export -- `tests/suites/trait_consistency.rs` — trait consistency entry -- `problemreductions-cli/tests/cli_tests.rs` — CLI integration tests for `pred create` - -**Whitelist for [Rule] PRs:** -- `src/rules/_.rs` — reduction implementation -- `src/rules/mod.rs` — module registration -- `src/unit_tests/rules/_.rs` — unit tests -- `src/example_db/rule_builders.rs` — canonical example registration -- `src/models//.rs` — only if adding getters needed for overhead expressions -- `docs/paper/reductions.typ` — paper entry -- `docs/src/reductions/reduction_graph.json` — graph export -- `docs/src/reductions/problem_schemas.json` — only if updating field descriptions -- `problemreductions-cli/tests/cli_tests.rs` — CLI integration tests if adding CLI support - -If any file falls outside these whitelists, flag it: +Use the report's `Deterministic Checks` section directly in the common path. + +If the report says whitelist is unavailable because of the warning fallback, call that out explicitly and keep the fallback narrow: either fix the prep problem first or hold the PR instead of rebuilding the deterministic pipeline manually inside the skill. + +If the report says files fall outside the whitelist, flag it: > **File Whitelist Check** > @@ -167,24 +169,29 @@ If all files are whitelisted, report "All files within expected whitelist" and c ### Step 4: Completeness check +Use the report's `Deterministic Checks` section as the baseline checklist for files, paper entries, examples, variants/overhead forms, and trait-consistency coverage. Then apply maintainer judgment on anything the script cannot prove. + +Read the review subject from the report's `Subject` section to understand whether the PR is being reviewed as a model, rule, or generic change. If the deterministic checks are unavailable because of the warning fallback, that should usually push you toward hold/manual follow-up rather than a full merge recommendation. + Verify the PR includes all required components. Check: **For [Model] PRs:** - [ ] Model implementation (`src/models/...`) - [ ] Unit tests (`src/unit_tests/models/...`) - [ ] `declare_variants!` macro with explicit `opt`/`sat` solver-kind markers and intended default variant -- [ ] CLI `pred create` support / help text as needed -- [ ] Canonical model example in `src/example_db/model_builders.rs` +- [ ] Schema / registry entry for CLI-facing model creation (`ProblemSchemaEntry`) +- [ ] Canonical model example function in the model file - [ ] Paper section in `docs/paper/reductions.typ` (`problem-def` entry) - [ ] `display-name` entry in paper -- [ ] `trait_consistency.rs` entry in `test_all_problems_implement_trait_correctly` (+ `test_direction` for optimization) +- [ ] `trait_consistency.rs` entry in `src/unit_tests/trait_consistency.rs` (`test_all_problems_implement_trait_correctly`, plus `test_direction` for optimization) **For [Rule] PRs:** - [ ] Reduction implementation (`src/rules/...`) +- [ ] `src/rules/mod.rs` registration - [ ] Unit tests (`src/unit_tests/rules/...`) - [ ] `#[reduction(overhead = {...})]` with correct expressions -- [ ] Uses only the `overhead` form of `#[reduction]` and does not duplicate a primitive exact endpoint registration -- [ ] Canonical rule example in `src/example_db/rule_builders.rs` +- [ ] Uses only the `overhead` form of `#[reduction]` +- [ ] Canonical rule example function in the rule file - [ ] Paper section in `docs/paper/reductions.typ` (`reduction-rule` entry) **Paper-example consistency check (both Model and Rule PRs):** @@ -194,6 +201,22 @@ The paper example must use data from the generated JSON (`docs/paper/examples/ge 2. For **[Rule] PRs**: the paper's `reduction-rule` entry must call `load-example(source, target)` (defined in `reductions.typ`) to load the canonical example from `rules.json`, and derive all concrete values from the loaded data using Typst array operations — no hand-written instance data. 3. For **[Model] PRs**: read the problem's entry in `models.json` and compare its `instance` field against the paper's `problem-def` example. The paper example must use the same instance (allowing 0-indexed JSON vs 1-indexed math notation). If they differ, flag: "Paper example does not match `example_db` canonical instance in `models.json`." +**Issue–test round-trip consistency check (both Model and Rule PRs):** + +The unit test's example instance and expected solution must match the issue's example. Compare using the report's `Linked Issue Context` and `Full Diff`: + +1. **Instance match**: The unit test's `example_instance()` (or equivalent setup) must construct the same graph/weights/parameters as described in the issue's "Example Instance" section. Check vertex count, edge list, weights, and any problem-specific fields (e.g., terminals, clauses). +2. **Solution match**: The expected optimal value in the test (e.g., `SolutionSize::Valid(6)`) must equal the issue's stated optimal. For rules, the closed-loop test must verify that reducing and solving the target gives the same optimum as solving the source directly. +3. **Brute-force verification**: A brute-force test must exist that independently confirms the expected optimum, not just assert a hardcoded value. + +If any mismatch is found, flag it: + +> **Issue–Test Consistency** +> +> Mismatch: [describe what differs — e.g., "Issue says optimal cost = 6 but test asserts 7"] + +If all match, report "Issue example and unit tests are consistent." + Report missing items: > **Completeness Check** @@ -256,42 +279,68 @@ Present a summary table: Then present all numbered issues from Step 5 as a multi-select `AskUserQuestion`: -> **Which issues should be fixed before merging?** (select all that apply, or "Merge as-is") -> - "Merge as-is" — no fixes needed -> - "Fix 1: [short description]" — [one-line summary] -> - "Fix 2: [short description]" — [one-line summary] +> **How should we proceed?** (select all that apply) +> - "Approve & Merge" — approve the PR, then squash-merge and move to Done +> - "Record 1: [short description]" — record for follow-up fix (does not block merge) +> - "Record 2: [short description]" — record for follow-up fix +> - ... +> - "Quick fix 1: [short description]" — fix now before merging +> - "Quick fix 2: [short description]" — fix now before merging > - ... > - "OnHold" — move to OnHold column with a reason -This lets the reviewer cherry-pick exactly which issues to fix. If the reviewer selects fixes, proceed to Step 7 Quick fix. If "Merge as-is", proceed to Step 7 Merge. +"Record" items are non-blocking — they get posted as a follow-up comment on the PR/issue but do not prevent merging. "Quick fix" items are applied immediately before merging. -If any actionable PR / issue comment from Step 1g is still open, `Merge as-is` must **not** be your recommendation. Recommend either **Quick fix** or **OnHold** instead. +If any actionable PR / issue comment from Step 1b is still open, `Approve & Merge` must **not** be your recommendation. Recommend either **Quick fix** or **OnHold** instead. ### Step 7: Execute decision -**If Merge:** -1. Print the PR URL prominently: `https://github.com/CodingThrust/problem-reductions/pull/` -2. Say: "Please merge this PR in your browser. After merging, I'll move the linked issue to Done." -3. Wait for user confirmation, then move the project board item to `Done` (`6aca54fa`). +**If Approve & Merge:** +1. If any "Record" items were selected, post them as a follow-up comment: + ```bash + COMMENT_FILE=$(mktemp) + cat > "$COMMENT_FILE" <<'EOF' + **Follow-up items** (recorded during final review): + - [item 1] + - [item 2] + EOF + python3 scripts/pipeline_pr.py comment --repo "$REPO" --pr "" --body-file "$COMMENT_FILE" + rm -f "$COMMENT_FILE" + ``` +2. Approve and merge the PR (approve may fail if you are the PR author — that's OK, continue to merge): + ```bash + gh pr review --approve || true + gh pr merge --squash --delete-branch + ``` +3. Move the project board item to `Done`: + ```bash + python3 scripts/pipeline_board.py move done + ``` **If OnHold:** 1. Ask the reviewer for the reason (use `AskUserQuestion` with free text). 2. Post a comment on the PR (or linked issue) with the reason: ```bash - gh pr comment --body "**On Hold**: " + COMMENT_FILE=$(mktemp) + printf '**On Hold**: %s\n' "" > "$COMMENT_FILE" + python3 scripts/pipeline_pr.py comment --repo "$REPO" --pr "" --body-file "$COMMENT_FILE" + rm -f "$COMMENT_FILE" ``` -3. Move the project board item to `OnHold` (`48dfe446`): +3. Move the project board item to `OnHold`: ```bash - gh project item-edit --project-id PVT_kwDOBrtarc4BRNVy --id --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc --single-select-option-id 48dfe446 + python3 scripts/pipeline_board.py move on-hold ``` **If Quick fix:** 1. Apply only the fixes the reviewer selected in Step 6. -2. Checkout the PR branch in a worktree, apply fixes, commit, push. +2. Work in the worktree from Step 0, apply fixes, commit, push. 3. After push, go back to Step 6 to re-confirm the decision. **If Reject:** 1. Ask the reviewer for the reason. 2. Post a comment explaining the rejection. 3. Close the PR: `gh pr close --comment ""` -4. Move the project board item to `OnHold` (`48dfe446`). +4. Move the project board item to `OnHold`: + ```bash + python3 scripts/pipeline_board.py move on-hold + ``` diff --git a/.claude/skills/fix-pr/SKILL.md b/.claude/skills/fix-pr/SKILL.md index 61149c887..07473fca8 100644 --- a/.claude/skills/fix-pr/SKILL.md +++ b/.claude/skills/fix-pr/SKILL.md @@ -9,80 +9,50 @@ Resolve PR review comments, fix CI failures, and address codecov coverage gaps f ## Step 1: Gather PR State -**IMPORTANT:** Do NOT use `gh api --jq` for extracting data — it uses a built-in jq that -chokes on response bodies containing backslashes (common in Copilot code suggestions). -Always pipe to `python3 -c` instead. (`gh pr view --jq` is fine — only `gh api --jq` is affected.) +Step 1 should be a single report-generation step. Use the shared scripted helper to produce one skill-readable PR context packet. Do not rebuild this logic inline with `gh api | python3 -c` unless you are debugging the helper itself. ```bash -# Get repo identifiers -REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner) # e.g., "owner/repo" +REPORT=$(python3 scripts/pipeline_pr.py context --current --format text) +printf '%s\n' "$REPORT" +``` -# Get PR number -PR=$(gh pr view --json number --jq .number) +The report should already include: +- repo, PR number, title, URL, head SHA +- comment counts +- CI summary +- Codecov summary +- linked issue context -# Get PR head SHA (on remote) -HEAD_SHA=$(gh api repos/$REPO/pulls/$PR | python3 -c "import sys,json; print(json.load(sys.stdin)['head']['sha'])") -``` +Use the values printed in that report for the rest of this skill. If you absolutely need raw structured data for a corner case, rerun the same command with `--format json`, but do not rebuild Step 1 manually. ### 1a. Fetch Review Comments **Check ALL four sources.** User inline comments are the most commonly missed — do not skip any. -```bash -# 1. Inline review comments on code lines (from ALL reviewers: users AND Copilot) -gh api repos/$REPO/pulls/$PR/comments | python3 -c " -import sys,json -comments = json.load(sys.stdin) -print(f'=== Inline comments: {len(comments)} ===') -for c in comments: - line = c.get('line') or c.get('original_line') or '?' - print(f'[{c[\"user\"][\"login\"]}] {c[\"path\"]}:{line} — {c[\"body\"][:200]}') -" - -# 2. Review-level comments (top-level review body from formal reviews) -gh api repos/$REPO/pulls/$PR/reviews | python3 -c " -import sys,json -reviews = json.load(sys.stdin) -print(f'=== Reviews: {len(reviews)} ===') -for r in reviews: - if r.get('body'): - print(f'[{r[\"user\"][\"login\"]}] {r[\"state\"]}: {r[\"body\"][:200]}') -" +Start from the report's `Comment Summary`. It should tell you whether any source is non-empty before you inspect raw threads. -# 3. Issue-level comments (general discussion) -gh api repos/$REPO/issues/$PR/comments | python3 -c " -import sys,json -comments = [c for c in json.load(sys.stdin) if 'codecov' not in c['user']['login']] -print(f'=== Issue comments: {len(comments)} ===') -for c in comments: - print(f'[{c[\"user\"][\"login\"]}] {c[\"body\"][:200]}') -" -``` - -**Verify counts:** If any source returns 0, confirm it's genuinely empty — don't assume no feedback exists. +If you need the raw comment arrays for detailed triage, rerun `python3 scripts/pipeline_pr.py context --current --format json` and inspect: +- `comments["inline_comments"]` +- `comments["reviews"]` +- `comments["human_issue_comments"]` +- `comments["human_linked_issue_comments"]` +- `comments["codecov_comments"]` ### 1b. Check CI Status -```bash -# All check runs on the PR head -gh api repos/$REPO/commits/$HEAD_SHA/check-runs | python3 -c " -import sys,json -for cr in json.load(sys.stdin)['check_runs']: - print(f'{cr[\"name\"]}: {cr.get(\"conclusion\") or cr[\"status\"]}') -" -``` +Read the report's `CI Summary`. The structured JSON fallback includes: +- `state` — `pending`, `failure`, or `success` +- `runs` — normalized check-run details +- `pending` / `failing` / `succeeding` counts ### 1c. Check Codecov Report -```bash -# Codecov bot comment with coverage diff -gh api repos/$REPO/issues/$PR/comments | python3 -c " -import sys,json -for c in json.load(sys.stdin): - if c['user']['login'] == 'codecov[bot]': - print(c['body']) -" -``` +Read the report's `Codecov` section. The structured JSON fallback includes: +- `found` — whether a Codecov comment is present +- `patch_coverage` +- `project_coverage` +- `filepaths` — deduplicated paths referenced by Codecov links +- `body` — the raw latest Codecov comment body ## Step 2: Triage and Prioritize @@ -130,23 +100,10 @@ Copilot suggestions with `suggestion` blocks contain exact code. Evaluate each: ### 5a. Identify Uncovered Lines -From the codecov bot comment (fetched in Step 1c), extract: +From the `CODECOV` JSON (fetched in Step 1c), extract: - Files with missing coverage - Patch coverage percentage -- Specific uncovered lines (linked in the report) - -For detailed line-by-line coverage, use the Codecov API: - -```bash -# Get file-level coverage for the PR -gh api repos/$REPO/issues/$PR/comments | python3 -c " -import sys,json,re -for c in json.load(sys.stdin): - if c['user']['login'] == 'codecov[bot]': - for m in re.findall(r'filepath=([^&\"]+)', c['body']): - print(m) -" -``` +- Specific uncovered files referenced in `filepaths` Then read the source files and identify which new/changed lines lack test coverage. diff --git a/.claude/skills/issue-to-pr/SKILL.md b/.claude/skills/issue-to-pr/SKILL.md index 2651ae289..e5bba683b 100644 --- a/.claude/skills/issue-to-pr/SKILL.md +++ b/.claude/skills/issue-to-pr/SKILL.md @@ -18,10 +18,10 @@ For Codex, open this `SKILL.md` directly and treat the slash-command forms above ``` Receive issue number [+ --execute flag] - -> Fetch issue with gh - -> Verify Good label (from check-issue) - -> If not Good: STOP - -> If Good: research references, write plan, create PR + -> Fetch structured issue preflight report + -> Verify Good label and rule-model guards + -> If guards fail: STOP + -> If guards pass: research references, write plan, create or resume PR -> If --execute: run plan via subagent-driven-development, then review-implementation ``` @@ -29,44 +29,48 @@ Receive issue number [+ --execute flag] ### 1. Parse Input -Extract issue number and flags from arguments: +Extract issue number, repo, and flags from arguments: - `123` -> issue #123, plan only - `123 --execute` -> issue #123, plan + execute - `https://github.com/owner/repo/issues/123` -> issue #123 - `owner/repo#123` -> issue #123 in owner/repo -### 2. Fetch Issue +Normalize to: +- `ISSUE=` +- `REPO=` (default `CodingThrust/problem-reductions`) +- `EXECUTE=true|false` + +### 2. Fetch Issue + Preflight Guards ```bash -gh issue view --json title,body,labels,assignees,comments +ISSUE_JSON=$(python3 scripts/pipeline_checks.py issue-context \ + --repo "$REPO" \ + --issue "$ISSUE" \ + --format json) ``` -Present issue summary to user. **Also review all comments** — contributors and maintainers may have posted clarifications, corrections, additional context, or design decisions that refine or override parts of the original issue body. Incorporate relevant comment content when writing the plan. +Treat `ISSUE_JSON` as the source of truth for the deterministic preflight data: +- `title`, `body`, `labels`, and `comments` provide the issue summary and comment thread +- `kind`, `source_problem`, and `target_problem` provide parsed issue metadata +- `checks.good_label`, `checks.source_model`, and `checks.target_model` provide guard outcomes +- `existing_prs`, `resume_pr`, and `action` tell you whether to resume an open PR instead of creating a new one + +Present the issue summary to the user. **Also review all comments** — contributors and maintainers may have posted clarifications, corrections, additional context, or design decisions that refine or override parts of the original issue body. Incorporate relevant comment content when writing the plan. ### 3. Verify Issue Has Passed check-issue The issue must have already passed the `check-issue` quality gate (Stage 1 validation). Do NOT re-validate the issue here. -**Gate condition:** The issue must have the `Good` label (added by `check-issue` when all checks pass). - -```bash -LABELS=$(gh issue view --json labels --jq '[.labels[].name] | join(",")') -``` - -- If `Good` is NOT in the labels → **STOP**: "Issue #N has not passed check-issue. Please run `/check-issue ` first." -- If `Good` is present → continue to step 4. +Use `ISSUE_JSON.checks.good_label`: +- If it is `fail` → **STOP**: "Issue #N has not passed check-issue. Please run `/check-issue ` first." +- If it is `pass` → continue. ### 3.5. Model-Existence Guard (for `[Rule]` issues only) -For `[Rule]` issues, parse the source and target problem names from the title (e.g., `[Rule] BinPacking to ILP` → source=BinPacking, target=ILP). Verify that **both** models already exist in the codebase on `main`: +For `[Rule]` issues, `ISSUE_JSON` already includes `source_problem`, `target_problem`, and the deterministic model-existence checks. -```bash -grep -r "struct SourceName" src/models/ -grep -r "struct TargetName" src/models/ -``` - -- If **both** models exist → continue to step 4. -- If either model is missing → **STOP**. Comment on the issue: "Blocked: model `` does not exist in main yet. Please implement it first (or file a `[Model]` issue)." +- If both `checks.source_model` and `checks.target_model` are `pass` → continue to step 4. +- If either is `fail` → **STOP**. Comment on the issue: "Blocked: model `` does not exist in main yet. Please implement it first (or file a `[Model]` issue)." **One item per PR:** Do NOT implement a missing model as part of a `[Rule]` PR. Each PR should contain exactly one model or one rule, never both. This avoids bloated PRs and repeated implementation when the model is needed by multiple rules. @@ -106,26 +110,23 @@ Include the concrete details from the issue (problem definition, reduction algor ### 6. Create PR (or Resume Existing) -**Check for existing PR first:** -```bash -EXISTING_PR=$(gh pr list --search "Fixes #" --state open --json number,headRefName --jq '.[0].number // empty') -``` +Use the `ISSUE_JSON.action` and `ISSUE_JSON.resume_pr` fields from Step 2. -**If a PR already exists** (`EXISTING_PR` is non-empty): -- Switch to its branch: `git checkout ` -- Capture `PR=$EXISTING_PR` +**If an open PR already exists** (`action == "resume-pr"`): +- Switch to its branch: `git checkout ` +- Capture `PR=` - Skip plan creation — jump directly to Step 7 (execute) -**If no existing PR** — create one with only the plan file: - -**Pre-flight checks** (before creating the branch): -1. Verify clean working tree: `git status --porcelain` must be empty. If not, STOP and ask user to stash or commit. -2. Check if branch already exists: `git rev-parse --verify issue-- 2>/dev/null`. If it exists, switch to it with `git checkout` (no `-b`) instead of creating a new one. +**If no open PR exists** (`action == "create-pr"`) — create one with only the plan file: ```bash -# Create branch (from main) -git checkout main -git rev-parse --verify issue-- 2>/dev/null && git checkout issue-- || git checkout -b issue-- +# Prepare or reuse the issue branch (this enforces a clean working tree) +BRANCH_JSON=$(python3 scripts/pipeline_worktree.py prepare-issue-branch \ + --issue \ + --slug \ + --base main \ + --format json) +BRANCH=$(printf '%s\n' "$BRANCH_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin)['branch'])") # Stage the plan file git add docs/plans/.md @@ -134,17 +135,27 @@ git add docs/plans/.md git commit -m "Add plan for #: " # Push -git push -u origin issue-<number>-<slug> +git push -u origin "$BRANCH" -# Create PR -gh pr create --title "Fix #<number>: <title>" --body " +# Create PR body +PR_BODY_FILE=$(mktemp) +cat > "$PR_BODY_FILE" <<'EOF' ## Summary <Brief description> -Fixes #<number>" +Fixes #<number> +EOF -# Capture PR number -PR=$(gh pr view --json number --jq .number) +# Create PR and capture the created PR number +PR_JSON=$(python3 scripts/pipeline_pr.py create \ + --repo "$REPO" \ + --title "Fix #<number>: <title>" \ + --body-file "$PR_BODY_FILE" \ + --base main \ + --head "$BRANCH" \ + --format json) +PR=$(printf '%s\n' "$PR_JSON" | python3 -c "import sys,json; print(json.load(sys.stdin)['pr_number'])") +rm -f "$PR_BODY_FILE" ``` ### 7. Execute Plan (only with `--execute`) @@ -194,7 +205,8 @@ Post an implementation summary comment on the PR **before** pushing. This commen - Note any open questions or trade-offs made ```bash -gh pr comment $PR --body "$(cat <<'EOF' +COMMENT_FILE=$(mktemp) +cat > "$COMMENT_FILE" <<'EOF' ## Implementation Summary ### Changes @@ -206,7 +218,8 @@ gh pr comment $PR --body "$(cat <<'EOF' ### Open Questions - [any trade-offs or items needing review — or "None"] EOF -)" +python3 scripts/pipeline_pr.py comment --repo "$REPO" --pr "$PR" --body-file "$COMMENT_FILE" +rm -f "$COMMENT_FILE" git push make copilot-review @@ -261,7 +274,7 @@ Run /review-pipeline to process Copilot comments, fix CI, and run agentic tests. | Generic plan | Use specifics from the issue, mapped to add-model/add-rule steps | | Skipping CLI registration in plan | add-model still requires alias/create/example-db planning, but not manual CLI dispatch-table edits | | Not verifying facts from issue | Use WebSearch/WebFetch to cross-check claims | -| Branch already exists on retry | Check with `git rev-parse --verify` before `git checkout -b` | -| Dirty working tree | Verify `git status --porcelain` is empty before branching | +| Branch already exists on retry | Use `pipeline_worktree.py prepare-issue-branch` — it will reuse the existing branch instead of failing on `git checkout -b` | +| Dirty working tree | Use `pipeline_worktree.py prepare-issue-branch` — it stops before branching if the worktree is dirty | | Bundling model + rule in one PR | Each PR must contain exactly one model or one rule — STOP and block if model is missing (Step 3.5) | | Plan files left in PR | Delete plan files before final push (Step 7c) | diff --git a/.claude/skills/project-pipeline/SKILL.md b/.claude/skills/project-pipeline/SKILL.md index f02504652..e73926639 100644 --- a/.claude/skills/project-pipeline/SKILL.md +++ b/.claude/skills/project-pipeline/SKILL.md @@ -5,7 +5,7 @@ description: Pick a Ready issue from the GitHub Project board, move it through I # Project Pipeline -Pick a "Ready" issue from the [GitHub Project board](https://github.com/orgs/CodingThrust/projects/8/views/1), move it to "In Progress", run `issue-to-pr --execute`, then move it to "Review pool". The separate `review-pipeline` handles Copilot comments, CI fixes, and agentic testing. +Pick a "Ready" issue from the [GitHub Project board](https://github.com/orgs/CodingThrust/projects/8/views/1), claim it into "In Progress", run `issue-to-pr --execute`, then move it to "Review pool". The separate `review-pipeline` handles Copilot comments, CI fixes, and agentic testing. ## Invocation @@ -36,41 +36,49 @@ This skill runs **fully autonomously** — no confirmation prompts, no user ques ## Steps -### 0. Discover and Rank Ready Issues +### 0. Generate the Project-Pipeline Report -#### 0a. Fetch Ready Issues +Step 0 should be a single report-generation step. Do not manually list Ready items, list In-progress items, grep model declarations, or re-derive blocked rules with separate shell commands. ```bash -gh project item-list 8 --owner CodingThrust --format json --limit 500 -``` - -Filter items where `status == "Ready"`. Partition into `[Model]` and `[Rule]` buckets. - -#### 0b. Gather Context for Ranking +set -- python3 scripts/pipeline_skill_context.py project-pipeline --repo CodingThrust/problem-reductions --repo-root . --format text -1. **Existing problems:** Grep for problem struct definitions in the codebase: `grep -r "^pub struct" src/models/ | sed 's/.*pub struct \([A-Za-z]*\).*/\1/'` to get all problem names currently implemented on `main`. -2. **Pending rules:** From the full project board JSON, collect all `[Rule]` issues that are in "Ready" or "In Progress" status. Parse their source/target problem names (e.g., `[Rule] BinPacking to ILP` → source=BinPacking, target=ILP). +# If a specific issue number was provided, validate it through the same bundle: +# set -- "$@" --issue <number> -#### 0c. Check Eligibility - -**Rule issues require both source and target models to exist on `main`.** For each `[Rule]` issue, parse the source and target problem names (e.g., `[Rule] BinPacking to ILP` → source=BinPacking, target=ILP). Check that both appear in the existing problems list (from Step 0b grep). +REPORT=$("$@") +printf '%s\n' "$REPORT" +``` -- If both models exist in the codebase → **eligible** -- If either model is missing from the codebase → **ineligible**, mark it `[blocked]` with reason (e.g., "model X not yet implemented on main") +The report is the Step 0 packet. It should already include: +- Queue Summary +- Eligible Ready Issues +- Blocked Ready Issues +- In Progress Issues +- Requested Issue validation when a specific issue was supplied -Do NOT consider pending `[Model]` issues as satisfying the dependency — only models already merged to `main` count. This prevents bundling model + rule in the same PR. +Branch from the report: +- `Bundle status: empty` => STOP with `No Ready issues are currently available.` +- `Bundle status: no-eligible-issues` => STOP with `Ready issues exist, but all current rule candidates are blocked by missing models on main.` +- `Bundle status: requested-missing` => STOP with `Issue #N is not currently in the Ready column.` +- `Bundle status: requested-blocked` => STOP with the blocking reason from the report +- `Bundle status: ready` => continue -All `[Model]` issues are always eligible (no dependency check needed). +The report already handled the deterministic setup: +- it loaded the Ready and In-progress issue sets +- it scanned existing problems on main +- it marked blocked `[Rule]` issues whose source or target model is still missing +- it computed the pending-rule unblock counts used for C3 -#### 0d. Score Eligible Issues +#### 0a. Score Eligible Issues Score only **eligible** issues on three criteria. For `[Model]` issues, extract the problem name. For `[Rule]` issues, extract both source and target problem names. | Criterion | Weight | How to Assess | |-----------|--------|---------------| -| **C1: Industrial/Theoretical Importance** | 3 | Read the issue body. Score 0-2: **2** = widely used in industry or foundational in complexity theory (e.g., ILP, SAT, MaxFlow, TSP, GraphColoring); **1** = moderately important or well-studied (e.g., SubsetSum, SetCover, Knapsack); **0** = niche or primarily academic | -| **C2: Related to Existing Problems** | 2 | Check if the problem connects to problems already in the reduction graph (via `list_problems`). Score 0-2: **2** = directly related (shares input structure or has known reductions to/from ≥2 existing problems, but is NOT a trivial variant of an existing one); **1** = loosely related (same domain, connects to 1 existing problem); **0** = isolated or is essentially a variant/renaming of an existing problem | -| **C3: Unblocks Pending Rules** | 2 | Check if this issue is a dependency for pending `[Rule]` issues. Score 0-2: **2** = unblocks ≥2 pending rules (a `[Model]` issue whose problem appears as source or target in ≥2 pending rules); **1** = unblocks 1 pending rule; **0** = does not unblock any pending rule | +| **C1: Industrial/Theoretical Importance** | 3 | Read the report's issue summary for each eligible issue. Score 0-2: **2** = widely used in industry or foundational in complexity theory (e.g., ILP, SAT, MaxFlow, TSP, GraphColoring); **1** = moderately important or well-studied (e.g., SubsetSum, SetCover, Knapsack); **0** = niche or primarily academic | +| **C2: Related to Existing Problems** | 2 | Use the report's Ready/In-progress context plus `pred list` if needed. Score 0-2: **2** = directly related (shares input structure or has known reductions to/from ≥2 existing problems, but is NOT a trivial variant of an existing one); **1** = loosely related (same domain, connects to 1 existing problem); **0** = isolated or is essentially a variant/renaming of an existing problem | +| **C3: Unblocks Pending Rules** | 2 | Read the `Pending rules unblocked` count already printed in the report for each eligible issue. Score 0-2: **2** = unblocks ≥2 pending rules; **1** = unblocks 1 pending rule; **0** = does not unblock any pending rule | **Final score** = C1 × 3 + C2 × 2 + C3 × 2 (max = 12) @@ -78,7 +86,7 @@ Score only **eligible** issues on three criteria. For `[Model]` issues, extract **Important for C2:** A problem that is merely a weighted/unweighted variant or a graph-subtype specialization of an existing problem scores **0** on C2, not 2. The goal is to add genuinely new problem types that expand the graph's reach. -#### 0e. Print Ranked List +#### 0b. Print Ranked List Print all Ready issues with their scores for visibility (no confirmation needed). Blocked rules appear at the bottom with their reason: @@ -96,48 +104,59 @@ Ready issues (ranked): 3 #130 [Rule] MultivariateQuadratic to ILP -- model "MultivariateQuadratic" not yet implemented ``` -#### 0f. Pick Issues +#### 0c. Pick Issues + +**If a specific issue number was provided:** validate and claim it through the scripted bundle: + +```bash +STATE_FILE=/tmp/problemreductions-ready-selection.json +CLAIM=$(python3 scripts/pipeline_board.py claim-next ready "$STATE_FILE" --number <number> --format json) +``` + +The report should already have stopped you before this point if the requested issue was missing or blocked. -**If a specific issue number was provided:** verify it is in the Ready column. If it is blocked, STOP with a message explaining which model is missing. +After successful validation, extract `ITEM_ID`, `ISSUE`, and `TITLE` from `CLAIM` using the same commands shown below. -**If `--all`:** proceed with all eligible issues in ranked order (highest score first). Models before Rules at same score. Blocked rules are skipped. After each issue is processed, re-check eligibility for remaining rules (a just-merged Model may unblock them). +**If `--all`:** proceed with all eligible issues in ranked order (highest score first). Models before Rules at same score. Blocked rules are skipped. After each issue is processed, regenerate the report before the next claim, because a just-merged Model may unblock pending rules. -**Otherwise (no args):** pick the highest-scored eligible (non-blocked) issue and proceed immediately (no confirmation). +**Otherwise (no args):** score the eligible issues from the report, pick the highest-scored one, and proceed immediately (no confirmation). After picking the issue number, claim it through the scripted bundle: + +```bash +STATE_FILE=/tmp/problemreductions-ready-selection.json +CLAIM=$(python3 scripts/pipeline_board.py claim-next ready "$STATE_FILE" --number <chosen-issue-number> --format json) +``` + +Extract the board item metadata from `CLAIM`: + +```bash +ITEM_ID=$(printf '%s\n' "$CLAIM" | python3 -c "import sys,json; print(json.load(sys.stdin)['item_id'])") +ISSUE=$(printf '%s\n' "$CLAIM" | python3 -c "import sys,json; data=json.load(sys.stdin); print(data['issue_number'] or data['number'])") +TITLE=$(printf '%s\n' "$CLAIM" | python3 -c "import sys,json; print(json.load(sys.stdin)['title'])") +``` ### 1. Create Worktree Create an isolated git worktree for this issue so the main working directory stays clean: ```bash -REPO_ROOT=$(git rev-parse --show-toplevel) -git fetch origin main -BRANCH="issue-<number>-<slug>" -WORKTREE_DIR=".worktrees/$BRANCH" -mkdir -p .worktrees -git worktree add "$WORKTREE_DIR" -b "$BRANCH" origin/main +WORKTREE=$(python3 scripts/pipeline_worktree.py create-issue --issue "$ISSUE" --slug <slug> --base origin/main --format json) +BRANCH=$(printf '%s\n' "$WORKTREE" | python3 -c "import sys,json; print(json.load(sys.stdin)['branch'])") +WORKTREE_DIR=$(printf '%s\n' "$WORKTREE" | python3 -c "import sys,json; print(json.load(sys.stdin)['worktree_dir'])") cd "$WORKTREE_DIR" ``` All subsequent steps run inside the worktree. This ensures the user's main checkout is never modified. -### 2. Move to "In Progress" - -Extract the project item ID for the chosen issue from the JSON output (the `id` field of the matching item). +### 2. Claim Result -```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id a12cfc9c -``` +`claim-next ready` has already moved the selected issue from `Ready` to `In progress`. Keep using `ITEM_ID` from the `CLAIM` JSON payload for later board transitions. ### 3. Run issue-to-pr --execute Invoke the `issue-to-pr` skill with `--execute` (working directory is the worktree): ``` -/issue-to-pr <number> --execute +/issue-to-pr "$ISSUE" --execute ``` This handles the full pipeline: fetch issue, verify Good label, research, write plan, create PR, implement, review, fix CI. @@ -149,31 +168,19 @@ This handles the full pipeline: fetch issue, verify Good label, research, write After `issue-to-pr` succeeds, move the issue to the `Review pool` column for the second-stage review pipeline: ```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id 7082ed60 +python3 scripts/pipeline_board.py move <ITEM_ID> review-pool ``` **If `issue-to-pr` failed after creating a PR:** move the issue to `Final review` instead so a human can take over: ```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id 51a3d8bb +python3 scripts/pipeline_board.py move <ITEM_ID> final-review ``` **If no PR was created** (issue-to-pr failed before creating a PR): move the issue back to "Ready" instead: ```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id f37d0d80 +python3 scripts/pipeline_board.py move <ITEM_ID> ready ``` ### 5. Clean Up Worktree diff --git a/.claude/skills/review-implementation/SKILL.md b/.claude/skills/review-implementation/SKILL.md index 3e61fbd96..470bab5eb 100644 --- a/.claude/skills/review-implementation/SKILL.md +++ b/.claude/skills/review-implementation/SKILL.md @@ -16,61 +16,42 @@ Dispatches two parallel review subagents with fresh context (no implementation h - `/review-implementation rule mis_qubo` -- review a specific rule - `/review-implementation generic` -- code quality only (no structural checklist) -## Step 1: Detect What Changed +## Step 1: Generate the Review-Implementation Report -Determine whether new model/rule files were added: +Step 1 should be a single report-generation step. Do not manually rebuild git range detection, `review-context`, current PR lookup, or linked-issue loading with separate shell snippets. ```bash -# Check for NEW files across the entire branch -git diff --name-only --diff-filter=A main..HEAD -``` - -Detection rules: -- New file in `src/models/` (not `mod.rs`) -> **model review** (structural + quality) -- New file in `src/rules/` (not `mod.rs`, `traits.rs`, `cost.rs`, `graph.rs`, `registry.rs`) -> **rule review** (structural + quality) -- Only modified files (no new model/rule) -> **quality review only** -- Both new model and rule files -> dispatch structural for both + quality -- Explicit argument overrides auto-detection - -Extract the problem name(s) and rule source/target from the file paths. +set -- python3 scripts/pipeline_skill_context.py review-implementation --repo-root . --format text -## Step 2: Prepare Subagent Context - -Get the git SHAs for the review range: +# Explicit subject overrides still go through the same bundle. Examples: +# set -- "$@" --kind model --name MaximumClique +# set -- "$@" --kind rule --name mis_qubo --source MaximumIndependentSet --target QUBO +# set -- "$@" --kind generic -```bash -BASE_SHA=$(git merge-base main HEAD) # or HEAD~N for batch reviews -HEAD_SHA=$(git rev-parse HEAD) +REPORT=$("$@") +printf '%s\n' "$REPORT" ``` -Get the diff summary and changed file list: +The report is the Step 1 packet. It should already include: +- Review Range: base SHA, head SHA, repo root +- Scope: review type, subject, added model/rule files +- Deterministic Checks: whitelist + completeness status +- Changed Files +- Diff Stat +- Current PR +- Linked Issue Context -```bash -git diff --stat $BASE_SHA..$HEAD_SHA -git diff --name-only $BASE_SHA..$HEAD_SHA -``` - -### Detect Linked Issue +Use the report as the default source of truth for the rest of this skill. If you need structured data for a corner case, rerun the same command with `--format json`, but do not rebuild Step 1 manually. -Check if the current branch has a PR linked to an issue: +## Step 2: Prepare Subagent Context -```bash -# Get PR number for current branch -PR_NUM=$(gh pr view --json number -q .number 2>/dev/null) - -# If PR exists, extract the linked issue number from the body -if [ -n "$PR_NUM" ]; then - ISSUE_NUM=$(gh pr view $PR_NUM --json body -q .body | grep -oE '#[0-9]+' | head -1 | tr -d '#') -fi - -# Fetch the issue body and comments if found -if [ -n "$ISSUE_NUM" ]; then - ISSUE_BODY=$(gh issue view $ISSUE_NUM --json title,body -q '"# " + .title + "\n\n" + .body') - ISSUE_COMMENTS=$(gh issue view $ISSUE_NUM --json comments -q '.comments[] | "**" + .author.login + "** (" + .createdAt + "):\n" + .body + "\n"') -fi -``` +Read the packet directly: +- `Review Range` for `{BASE_SHA}` and `{HEAD_SHA}` +- `Scope` for `{REVIEW_TYPE}` and the concrete model/rule metadata +- `Changed Files` and `Diff Stat` for the quality-reviewer prompt +- `Linked Issue Context` for `{ISSUE_CONTEXT}` -If an issue is found, pass `{ISSUE_CONTEXT}` (title + body + comments) to both subagents. If not, set `{ISSUE_CONTEXT}` to "No linked issue found." Comments often contain clarifications, corrections, or additional requirements from maintainers. +If the report says `Current PR` is absent, set `{ISSUE_CONTEXT}` to `No linked issue found.` Comments often contain clarifications, corrections, or additional requirements from maintainers, so prefer the report text over re-fetching issue state. ## Step 3: Dispatch Subagents in Parallel @@ -80,11 +61,11 @@ Dispatch using `Agent` tool with `subagent_type="superpowers:code-reviewer"`: - Read `structural-reviewer-prompt.md` from this skill directory - Fill placeholders: - - `{REVIEW_TYPE}` -> "model", "rule", or "model + rule" - - `{REVIEW_PARAMS}` -> summary of what's being reviewed + - `{REVIEW_TYPE}` -> from the report's `Scope` section (`model`, `rule`, `model + rule`, or `generic`) + - `{REVIEW_PARAMS}` -> summary of what's being reviewed from the report - `{PROBLEM_NAME}`, `{CATEGORY}`, `{FILE_STEM}` -> for model reviews - `{SOURCE}`, `{TARGET}`, `{RULE_STEM}`, `{EXAMPLE_STEM}` -> for rule reviews - - `{ISSUE_CONTEXT}` -> full issue title + body + comments (or "No linked issue found.") + - `{ISSUE_CONTEXT}` -> the report's `Linked Issue Context` section (or "No linked issue found.") - Prompt = filled template ### Quality Reviewer (always) @@ -93,11 +74,11 @@ Dispatch using `Agent` tool with `subagent_type="superpowers:code-reviewer"`: - Read `quality-reviewer-prompt.md` from this skill directory - Fill placeholders: - - `{DIFF_SUMMARY}` -> output of `git diff --stat` - - `{CHANGED_FILES}` -> list of changed files + - `{DIFF_SUMMARY}` -> the report's `Diff Stat` + - `{CHANGED_FILES}` -> the report's `Changed Files` - `{PLAN_STEP}` -> description of what was implemented (or "standalone review") - - `{BASE_SHA}`, `{HEAD_SHA}` -> git range - - `{ISSUE_CONTEXT}` -> full issue title + body + comments (or "No linked issue found.") + - `{BASE_SHA}`, `{HEAD_SHA}` -> the report's `Review Range` + - `{ISSUE_CONTEXT}` -> the report's `Linked Issue Context` section (or "No linked issue found.") - Prompt = filled template **Both subagents must be dispatched in parallel** (single message with two Agent tool calls — use `run_in_background: true` on one, foreground on the other, then read the background result with `TaskOutput`). diff --git a/.claude/skills/review-pipeline/SKILL.md b/.claude/skills/review-pipeline/SKILL.md index e5c85f558..00c326881 100644 --- a/.claude/skills/review-pipeline/SKILL.md +++ b/.claude/skills/review-pipeline/SKILL.md @@ -34,75 +34,53 @@ GitHub Project board IDs (for `gh project item-edit`): ## Autonomous Mode -This skill runs **fully autonomously** -- no confirmation prompts, no user questions. +This skill runs **fully autonomously** except for one case: if the scripted `review-pipeline` context bundle returns `status == "needs-user-choice"`, STOP and ask the user which PR is the intended target. ## Steps -### 0. Discover Review pool Items +### 0. Generate the Review-Pipeline Report -```bash -gh project item-list 8 --owner CodingThrust --format json --limit 500 -``` - -Filter items where `status == "Review pool"`. Each item should have an associated PR. Extract the PR number from the item title or linked issue. - -#### 0a. Check Copilot Review Status - -For each candidate PR, check whether Copilot has already submitted a review: +Step 0 should be a single report-generation step. Do not manually unpack board selection, worktree prep, or PR context with shell snippets. ```bash REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner) -gh api repos/$REPO/pulls/$PR/reviews --jq '[.[] | select(.user.login == "copilot-pull-request-reviewer[bot]")] | length' -``` - -A PR is **eligible** only if the count is ≥ 1 (Copilot has submitted at least one review). PRs without a Copilot review yet are marked `[waiting for Copilot]` and skipped. - -#### 0b. Print the List - -Print all Review pool items with their Copilot status: - -``` -Review pool PRs: - #570 Fix #117: [Model] GraphPartitioning [copilot reviewed] - #571 Fix #97: [Rule] BinPacking to ILP [waiting for Copilot] +STATE_FILE=/tmp/problemreductions-review-selection.json +set -- python3 scripts/pipeline_skill_context.py review-pipeline --repo "$REPO" --state-file "$STATE_FILE" --format text +if [ -n "${PR:-}" ]; then + set -- "$@" --pr "$PR" +fi +REPORT=$("$@") +printf '%s\n' "$REPORT" ``` -**If a specific PR number was provided:** verify it is in the Review pool column. If it is waiting for Copilot, STOP with a message: `PR #N is waiting for Copilot review. Re-run after Copilot has reviewed.` - -**If `--all`:** process only eligible (Copilot-reviewed) items in order (lowest PR number first). Skip waiting items. - -**Otherwise:** pick the first eligible item. If no items are eligible, STOP with: `No Review pool PRs have been reviewed by Copilot yet.` - -### 0g. Claim: Move to "Under review" - -**Immediately** move the chosen PR to the `Under review` column to signal that an agent is actively working on it. This prevents other agents from picking the same PR: - -```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id f04790ca +The report is the Step 0 packet. It should already include: +- Selection: board item, PR number, linked issue, title, URL +- Recommendation Seed: suggested mode and deterministic blockers +- Comment Summary +- CI / Coverage +- Merge Prep +- Linked Issue Context + +Branch from the report: +- `Bundle status: empty` => STOP with `No Review pool PRs are currently eligible for review-pipeline.` +- `Bundle status: needs-user-choice` => STOP and ask the user which PR is intended +- `Bundle status: ready` => continue with the already-claimed item and prepared worktree + +For ambiguous cards, the report should print short options and the recommendation. Format the prompt like: + +```text +Review pool card links multiple repo PRs: +1. PR #170 — CLOSED — Superseded LCS model +2. PR #173 — OPEN — Fix #109: Add LCS reduction (Recommended) ``` -In `--all` mode, claim each PR right before processing it (not all at once). - -### 1. Create Worktree and Checkout PR Branch +The bundle already handled the mechanical claim step: +- normal eligible PRs are claimed through the review queue +- explicit `--pr` matches on ambiguous cards are treated as deterministic disambiguation and claimed automatically -Create an isolated git worktree so the main working directory stays clean: +When you need to take actions later, use the identifiers already printed in the report (`Board item`, `PR`, worktree path). If you absolutely need raw structured data for a corner case, rerun the same command with `--format json`, but do not rebuild Step 0 manually. -```bash -REPO_ROOT=$(git rev-parse --show-toplevel) -REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner) -BRANCH=$(gh pr view $PR --json headRefName --jq .headRefName) -WORKTREE_DIR=".worktrees/review-$BRANCH" -mkdir -p .worktrees -git fetch origin $BRANCH -git worktree add "$WORKTREE_DIR" $BRANCH -cd "$WORKTREE_DIR" -``` - -All subsequent steps run inside the worktree. +All subsequent steps run inside the prepared worktree and should read facts from the report instead of re-fetching them by default. ### 1a. Resolve Conflicts with Main @@ -112,41 +90,31 @@ All subsequent steps run inside the worktree. 2. If they changed, read the current versions on main (`git show origin/main:.claude/skills/add-model/SKILL.md` and `git show origin/main:.claude/skills/add-rule/SKILL.md`) to understand what's different. 3. When resolving conflicts in model/rule implementation files, prefer the patterns from main's current skills — the PR's implementation may be based on outdated skill instructions. -Check if the branch has merge conflicts with main: - -```bash -git fetch origin main -git merge origin/main --no-edit -``` +Read the merge result from the report's `Merge Prep` section. -- If the merge succeeds cleanly: push the merge commit and continue. +- If the report says the merge status is `clean`: push the merge commit and continue. - If there are conflicts: - 1. Inspect the conflicting files with `git diff --name-only --diff-filter=U`. + 1. Inspect the conflicting files listed in the report. 2. Compare the current skill versions on main vs the PR branch to understand which patterns are current. 3. Resolve conflicts (prefer main's patterns for skill-generated code, the PR branch for problem-specific logic, main for regenerated artifacts like JSON). 4. Stage resolved files, commit, and push. -- If conflicts are too complex to resolve automatically (e.g., overlapping logic changes in the same function): - 1. Abort the merge: `git merge --abort` +- If the report says the merge status is `conflicted` and the overlap is otherwise too complex to resolve automatically: + 1. Abort the merge: `git merge --abort` if a merge is still in progress 2. Move the project item back to `Review pool`: ```bash - gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id 7082ed60 + python3 scripts/pipeline_board.py move <ITEM_ID> review-pool ``` 3. Report: `PR #N has complex merge conflicts with main — needs manual resolution.` 4. STOP processing this PR. ### 2. Fix Copilot Review Comments -Copilot review is guaranteed to exist (verified in Step 0). Fetch the comments: +Use the report as the primary mechanical context: +- `Comment Summary` +- `CI / Coverage` +- `Linked Issue Context` -```bash -COMMENTS=$(gh api repos/$REPO/pulls/$PR/comments --jq '[.[] | select(.user.login == "copilot-pull-request-reviewer[bot]")]') -``` - -If there are actionable comments: invoke `/fix-pr` to address them, then push: +Inspect the report's Copilot comment count and linked issue context. If there are actionable comments: invoke `/fix-pr` to address them, then push: ```bash git push @@ -156,55 +124,7 @@ If Copilot approved with no actionable comments: skip to next step. ### 2a. Check Issue Comments and Human PR Reviews -Extract the linked issue number from the PR title (pattern: `Fix #N:`): - -```bash -ISSUE=$(gh pr view $PR --json title --jq .title | grep -oP '(?<=Fix #)\d+') -``` - -Fetch all comment sources: - -```bash -# 1. Linked issue comments (from contributors, excluding bots) -if [ -n "$ISSUE" ]; then - gh api repos/$REPO/issues/$ISSUE/comments | python3 -c " -import sys,json -comments = [c for c in json.load(sys.stdin) if not c['user']['login'].endswith('[bot]')] -print(f'=== Issue #{sys.argv[1]} comments: {len(comments)} ===') -for c in comments: - print(f'[{c[\"user\"][\"login\"]}] {c[\"body\"][:300]}') - print('---') -" "$ISSUE" -fi - -# 2. Human PR review comments (inline, excluding Copilot) -gh api repos/$REPO/pulls/$PR/comments | python3 -c " -import sys,json -comments = [c for c in json.load(sys.stdin) if not c['user']['login'].endswith('[bot]')] -print(f'=== Human PR inline comments: {len(comments)} ===') -for c in comments: - line = c.get('line') or c.get('original_line') or '?' - print(f'[{c[\"user\"][\"login\"]}] {c[\"path\"]}:{line} — {c[\"body\"][:300]}') -" - -# 3. Human PR conversation comments (general discussion, excluding bots) -gh api repos/$REPO/issues/$PR/comments | python3 -c " -import sys,json -comments = [c for c in json.load(sys.stdin) if not c['user']['login'].endswith('[bot]')] -print(f'=== Human PR conversation comments: {len(comments)} ===') -for c in comments: - print(f'[{c[\"user\"][\"login\"]}] {c[\"body\"][:300]}') -" - -# 4. Human review-level comments (top-level review body) -gh api repos/$REPO/pulls/$PR/reviews | python3 -c " -import sys,json -reviews = [r for r in json.load(sys.stdin) if not r['user']['login'].endswith('[bot]') and r.get('body')] -print(f'=== Human reviews: {len(reviews)} ===') -for r in reviews: - print(f'[{r[\"user\"][\"login\"]}] {r[\"state\"]}: {r[\"body\"][:300]}') -" -``` +Reuse the report's `Comment Summary` and `Linked Issue Context` sections. If you need the raw structured comment objects for a corner case, rerun the bundle with `--format json`. For each actionable comment found: @@ -267,26 +187,13 @@ For each retry: 1. **Wait for CI to complete** (poll every 30s, up to 15 minutes): ```bash - for i in $(seq 1 30); do - sleep 30 - HEAD_SHA=$(gh api repos/$REPO/pulls/$PR | python3 -c "import sys,json; print(json.load(sys.stdin)['head']['sha'])") - STATUS=$(gh api repos/$REPO/commits/$HEAD_SHA/check-runs | python3 -c " - import sys,json - runs = json.load(sys.stdin)['check_runs'] - if not runs: - print('PENDING') - else: - failed = [r['name'] for r in runs if r.get('conclusion') not in ('success', 'skipped', None)] - pending = [r['name'] for r in runs if r.get('conclusion') is None and r['status'] != 'completed'] - if pending: - print('PENDING') - elif failed: - print('FAILED') - else: - print('GREEN') - ") - if [ "$STATUS" != "PENDING" ]; then break; fi - done + CI=$(python3 scripts/pipeline_pr.py wait-ci --repo "$REPO" --pr "$PR" --timeout 900 --interval 30 --format json) + STATUS=$(printf '%s\n' "$CI" | python3 -c " +import sys,json +state = json.load(sys.stdin)['state'] +mapping = {'success': 'GREEN', 'failure': 'FAILED', 'timeout': 'FAILED', 'pending': 'PENDING'} +print(mapping.get(state, 'FAILED')) +") ``` - If `GREEN` on the **first** iteration (before any fix-pr): skip the fix loop, done. @@ -315,11 +222,7 @@ git worktree remove "$WORKTREE_DIR" --force ### 6. Move to "Final review" ```bash -gh project item-edit \ - --id <ITEM_ID> \ - --project-id PVT_kwDOBrtarc4BRNVy \ - --field-id PVTSSF_lADOBrtarc4BRNVyzg_GmQc \ - --single-select-option-id 51a3d8bb +python3 scripts/pipeline_board.py move <ITEM_ID> final-review ``` ### 7. Report @@ -327,7 +230,8 @@ gh project item-edit \ Post the review summary as a PR comment so it's visible to human reviewers: ```bash -gh pr comment $PR --body "$(cat <<'EOF' +COMMENT_FILE=$(mktemp) +cat > "$COMMENT_FILE" <<'EOF' ## Review Pipeline Report | Check | Result | @@ -346,7 +250,8 @@ gh pr comment $PR --body "$(cat <<'EOF' 🤖 Generated by review-pipeline EOF -)" +python3 scripts/pipeline_pr.py comment --repo "$REPO" --pr "$PR" --body-file "$COMMENT_FILE" +rm -f "$COMMENT_FILE" ``` Adapt the table values to match the actual results for the PR. If CI failed after 3 retries, report `failed (3 retries)` instead of `green`. @@ -381,6 +286,8 @@ Completed: 2/2 | All moved to Final review | Mistake | Fix | |---------|-----| | PR not in Review pool column | Verify status before processing; STOP if not Review pool | +| Processing a closed PR from a stale issue card | Require PR state `OPEN`; skip stale closed PRs | +| Guessing on an issue card with multiple linked repo PRs | Stop, show options to the user, and recommend the most likely correct OPEN PR | | Picking a PR before Copilot has reviewed | Check `pulls/$PR/reviews` for copilot-pull-request-reviewer[bot]; skip if absent | | Missing project scopes | Run `gh auth refresh -s read:project,project` | | Skipping review-implementation | Always run structural completeness check in Step 2b — it catches gaps Copilot misses (paper entries, CLI registration, trait_consistency) | diff --git a/Makefile b/Makefile index d7011b9ea..53917dd42 100644 --- a/Makefile +++ b/Makefile @@ -1,6 +1,6 @@ # Makefile for problemreductions -.PHONY: help build test mcp-test fmt clippy doc mdbook paper examples clean coverage rust-export compare qubo-testdata export-schemas release run-plan run-issue run-pipeline run-pipeline-forever run-review run-review-forever diagrams jl-testdata cli cli-demo copilot-review +.PHONY: help build test mcp-test fmt clippy doc mdbook paper examples clean coverage rust-export compare qubo-testdata export-schemas release run-plan run-issue run-pipeline run-pipeline-forever run-review run-review-forever board-next board-claim board-ack board-move issue-context issue-guards pr-context pr-wait-ci worktree-issue worktree-pr diagrams jl-testdata cli cli-demo copilot-review RUNNER ?= codex CLAUDE_MODEL ?= opus @@ -37,6 +37,16 @@ help: @echo " run-pipeline-forever - Loop: poll Ready column for new issues, run-pipeline when new ones appear" @echo " run-review [N=<number>] - Pick PR from Review pool, fix comments/CI, run agentic tests" @echo " run-review-forever - Loop: poll Review pool for Copilot-reviewed PRs, run-review when new ones appear" + @echo " board-next MODE=<ready|review|final-review> [NUMBER=<n>] [FORMAT=text|json] - Get the next eligible queued project item" + @echo " board-claim MODE=<ready|review> [NUMBER=<n>] [FORMAT=text|json] - Claim and move the next eligible queued project item" + @echo " board-ack MODE=<ready|review|final-review> ITEM=<id> - Acknowledge a queued project item" + @echo " board-move ITEM=<id> STATUS=<status> - Move a project item to a named status" + @echo " issue-context ISSUE=<number> [REPO=<owner/repo>] - Fetch structured issue preflight JSON" + @echo " issue-guards ISSUE=<number> [REPO=<owner/repo>] - Backward-compatible alias for issue-context" + @echo " pr-context PR=<number> [REPO=<owner/repo>] - Fetch structured PR snapshot JSON" + @echo " pr-wait-ci PR=<number> [REPO=<owner/repo>] - Poll CI until terminal state and print JSON" + @echo " worktree-issue ISSUE=<number> SLUG=<slug> - Create an issue worktree from origin/main" + @echo " worktree-pr PR=<number> [REPO=<owner/repo>] - Checkout a PR into an isolated worktree" @echo " copilot-review - Request Copilot code review on current PR" @echo "" @echo " Set RUNNER=claude to use Claude instead of Codex (default: codex)" @@ -374,11 +384,21 @@ cli-demo: cli # make run-pipeline N=97 (processes specific issue) run-pipeline: @. scripts/make_helpers.sh; \ + state_file=$${STATE_FILE:-/tmp/problemreductions-ready-state.json}; \ if [ -n "$(N)" ]; then \ - PROMPT=$$(skill_prompt project-pipeline "/project-pipeline $(N)" "process GitHub issue $(N)"); \ + issue="$(N)"; \ else \ - PROMPT=$$(skill_prompt project-pipeline "/project-pipeline" "pick and process the next Ready issue"); \ + status=0; \ + selection=$$(board_next_json ready "" "" "$$state_file") || status=$$?; \ + if [ "$$status" -eq 1 ]; then \ + echo "No Ready issues are currently eligible."; \ + exit 1; \ + elif [ "$$status" -ne 0 ]; then \ + exit "$$status"; \ + fi; \ + issue=$$(printf '%s\n' "$$selection" | python3 -c "import sys,json; data=json.load(sys.stdin); print(data['issue_number'] or data['number'])"); \ fi; \ + PROMPT=$$(skill_prompt_with_context project-pipeline "/project-pipeline $$issue" "process GitHub issue $$issue" "Selected queue item" "$$selection"); \ run_agent "pipeline-output.log" "$$PROMPT" # Poll Ready column for new issues and run-pipeline when new ones appear @@ -387,16 +407,174 @@ run-pipeline-forever: @. scripts/make_helpers.sh; \ MAKE=$(MAKE) watch_and_dispatch ready run-pipeline "Ready issues" +# Get the next eligible board item from the scripted queue logic +# Usage: make board-next MODE=ready +# make board-next MODE=review REPO=CodingThrust/problem-reductions +# make board-next MODE=final-review REPO=CodingThrust/problem-reductions +# make board-next MODE=review REPO=CodingThrust/problem-reductions NUMBER=570 FORMAT=json +# STATE_FILE=/tmp/custom.json make board-next MODE=ready +board-next: + @if [ -z "$(MODE)" ]; then \ + echo "MODE=ready|review|final-review is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + state_file=$${STATE_FILE:-/tmp/problemreductions-$(MODE)-state.json}; \ + case "$(MODE)" in \ + review|final-review) \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + poll_project_items "$(MODE)" "$$state_file" "$$repo" "$(NUMBER)" "$(if $(FORMAT),$(FORMAT),text)"; \ + ;; \ + *) \ + poll_project_items "$(MODE)" "$$state_file" "" "$(NUMBER)" "$(if $(FORMAT),$(FORMAT),text)"; \ + ;; \ + esac + +# Claim and move the next eligible board item through the scripted queue logic +# Usage: make board-claim MODE=ready +# make board-claim MODE=review REPO=CodingThrust/problem-reductions +# make board-claim MODE=review REPO=CodingThrust/problem-reductions NUMBER=570 FORMAT=json +# STATE_FILE=/tmp/custom.json make board-claim MODE=ready +board-claim: + @if [ -z "$(MODE)" ]; then \ + echo "MODE=ready|review is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + state_file=$${STATE_FILE:-/tmp/problemreductions-$(MODE)-state.json}; \ + case "$(MODE)" in \ + review) \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + claim_project_items "$(MODE)" "$$state_file" "$$repo" "$(NUMBER)" "$(if $(FORMAT),$(FORMAT),json)"; \ + ;; \ + ready) \ + claim_project_items "$(MODE)" "$$state_file" "" "$(NUMBER)" "$(if $(FORMAT),$(FORMAT),json)"; \ + ;; \ + *) \ + echo "MODE=ready|review is required"; \ + exit 2; \ + ;; \ + esac + +# Advance a scripted board queue after an item is processed +# Usage: make board-ack MODE=ready ITEM=PVTI_xxx +# STATE_FILE=/tmp/custom.json make board-ack MODE=review ITEM=PVTI_xxx +# STATE_FILE=/tmp/custom.json make board-ack MODE=final-review ITEM=PVTI_xxx +board-ack: + @if [ -z "$(MODE)" ] || [ -z "$(ITEM)" ]; then \ + echo "MODE=ready|review|final-review and ITEM=<project-item-id> are required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + state_file=$${STATE_FILE:-/tmp/problemreductions-$(MODE)-state.json}; \ + ack_polled_item "$$state_file" "$(ITEM)" + +# Move a project board item to a named status through the shared board script +# Usage: make board-move ITEM=PVTI_xxx STATUS=under-review +board-move: + @if [ -z "$(ITEM)" ] || [ -z "$(STATUS)" ]; then \ + echo "ITEM=<project-item-id> and STATUS=<backlog|ready|in-progress|review-pool|under-review|final-review|on-hold|done> are required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + move_board_item "$(ITEM)" "$(STATUS)" + +# Fetch deterministic issue preflight JSON for issue-to-pr +# Usage: make issue-context ISSUE=117 +# make issue-context ISSUE=117 REPO=CodingThrust/problem-reductions +issue-context: + @if [ -z "$(ISSUE)" ]; then \ + echo "ISSUE=<number> is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + repo=$${REPO:-CodingThrust/problem-reductions}; \ + issue_context "$$repo" "$(ISSUE)" + +# Fetch deterministic issue preflight JSON for issue-to-pr +# Usage: make issue-guards ISSUE=117 +# make issue-guards ISSUE=117 REPO=CodingThrust/problem-reductions +issue-guards: + @if [ -z "$(ISSUE)" ]; then \ + echo "ISSUE=<number> is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + repo=$${REPO:-CodingThrust/problem-reductions}; \ + issue_guards "$$repo" "$(ISSUE)" + +# Fetch structured PR snapshot JSON from the shared helper +# Usage: make pr-context PR=570 +# make pr-context PR=570 REPO=CodingThrust/problem-reductions +pr-context: + @if [ -z "$(PR)" ]; then \ + echo "PR=<number> is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + pr_snapshot "$$repo" "$(PR)" + +# Poll CI for a PR until it reaches a terminal state +# Usage: make pr-wait-ci PR=570 +# make pr-wait-ci PR=570 TIMEOUT=1200 INTERVAL=15 +pr-wait-ci: + @if [ -z "$(PR)" ]; then \ + echo "PR=<number> is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + timeout=$${TIMEOUT:-900}; \ + interval=$${INTERVAL:-30}; \ + pr_wait_ci "$$repo" "$(PR)" "$$timeout" "$$interval" + +# Create an issue worktree from origin/main +# Usage: make worktree-issue ISSUE=117 SLUG=graph-partitioning +worktree-issue: + @if [ -z "$(ISSUE)" ] || [ -z "$(SLUG)" ]; then \ + echo "ISSUE=<number> and SLUG=<slug> are required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + base=$${BASE:-origin/main}; \ + create_issue_worktree "$(ISSUE)" "$(SLUG)" "$$base" + +# Checkout a PR into an isolated worktree +# Usage: make worktree-pr PR=570 +# make worktree-pr PR=570 REPO=CodingThrust/problem-reductions +worktree-pr: + @if [ -z "$(PR)" ]; then \ + echo "PR=<number> is required"; \ + exit 2; \ + fi + @. scripts/make_helpers.sh; \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + checkout_pr_worktree "$$repo" "$(PR)" + # Usage: make run-review (picks next Review pool PR automatically) # make run-review N=570 (processes specific PR) # RUNNER=claude make run-review (use Claude instead of Codex) run-review: @. scripts/make_helpers.sh; \ - if [ -n "$(N)" ]; then \ - PROMPT=$$(skill_prompt review-pipeline "/review-pipeline $(N)" "process PR #$(N)"); \ + repo=$${REPO:-$$(gh repo view --json nameWithOwner --jq .nameWithOwner)}; \ + state_file=$${STATE_FILE:-/tmp/problemreductions-review-state.json}; \ + pr="$(N)"; \ + selection=$$(review_pipeline_context "$$repo" "$$pr" "$$state_file"); \ + status_name=$$(printf '%s\n' "$$selection" | python3 -c "import sys,json; print(json.load(sys.stdin)['status'])"); \ + if [ "$$status_name" = "empty" ]; then \ + echo "No Review pool PRs are currently eligible."; \ + exit 1; \ + fi; \ + if [ "$$status_name" = "ready" ]; then \ + pr=$$(printf '%s\n' "$$selection" | python3 -c "import sys,json; print(json.load(sys.stdin)['selection']['pr_number'])"); \ + slash_cmd="/review-pipeline $$pr"; \ + codex_desc="process PR #$$pr"; \ else \ - PROMPT=$$(skill_prompt review-pipeline "/review-pipeline" "pick and process the next Review pool PR"); \ + slash_cmd="/review-pipeline"; \ + codex_desc="inspect the review pipeline bundle and resolve the next action"; \ fi; \ + PROMPT=$$(skill_prompt_with_context review-pipeline "$$slash_cmd" "$$codex_desc" "Review pipeline context" "$$selection"); \ run_agent "review-output.log" "$$PROMPT" # Poll Review pool column for Copilot-reviewed PRs and run-review when new ones appear diff --git a/docs/plans/2026-02-08-qubo-reductions-plan.md b/docs/plans/2026-02-08-qubo-reductions-plan.md deleted file mode 100644 index 7c1a53c02..000000000 --- a/docs/plans/2026-02-08-qubo-reductions-plan.md +++ /dev/null @@ -1,414 +0,0 @@ -# Problem-to-QUBO Reductions Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Implement 7 reductions from NP-hard problems to QUBO, with tests, examples, and paper documentation (Issue #18). - -**Architecture:** Each reduction creates a `QUBO<f64>` matrix encoding the source problem's objective + constraints as penalty terms. All reductions follow the existing pattern in `src/rules/spinglass_qubo.rs`: a result struct implementing `ReductionResult`, a `ReduceTo` impl with `#[reduction]` macro, unit tests via `#[path]`, and integration tests in `tests/suites/reductions.rs`. - -**Tech Stack:** Rust, `#[reduction]` proc macro, `inventory` for registration, `BruteForce` solver for tests. Ground truth JSON in `tests/data/qubo/` (already generated via PR #29). - -**Branch:** `issue-18-qubo-reductions` (already exists, PR #29) - ---- - -### Task 1: IndependentSet → QUBO - -Maximize weighted IS = minimize `-Σ w_i·x_i + P·Σ_{(i,j)∈E} x_i·x_j` where `P > Σ w_i`. - -**Files:** -- Create: `src/rules/independentset_qubo.rs` -- Create: `src/unit_tests/rules/independentset_qubo.rs` -- Modify: `src/rules/mod.rs` — add `mod independentset_qubo;` + `pub use` - -**Step 1: Write unit test** - -File: `src/unit_tests/rules/independentset_qubo.rs` - -```rust -use super::*; -use crate::solvers::{BruteForce, Solver}; - -#[test] -fn test_independentset_to_qubo_closed_loop() { - // Path graph: 0-1-2-3 (4 vertices, 3 edges) - // Maximum IS = {0, 2} or {1, 3} (size 2) - let is = IndependentSet::<SimpleGraph, Unweighted>::new(4, vec![(0, 1), (1, 2), (2, 3)]); - let reduction = ReduceTo::<QUBO<f64>>::reduce_to(&is); - let qubo = reduction.target_problem(); - - let solver = BruteForce::new(); - let qubo_solutions = solver.find_best(qubo); - - for sol in &qubo_solutions { - let extracted = reduction.extract_solution(sol); - assert!(is.solution_size(&extracted).is_valid); - // IS of size 2 - assert_eq!(extracted.iter().filter(|&&x| x == 1).count(), 2); - } -} - -#[test] -fn test_independentset_to_qubo_triangle() { - // Triangle: 0-1-2 (complete graph K3) - // Maximum IS = any single vertex (size 1) - let is = IndependentSet::<SimpleGraph, Unweighted>::new(3, vec![(0, 1), (1, 2), (0, 2)]); - let reduction = ReduceTo::<QUBO<f64>>::reduce_to(&is); - let qubo = reduction.target_problem(); - - let solver = BruteForce::new(); - let qubo_solutions = solver.find_best(qubo); - - for sol in &qubo_solutions { - let extracted = reduction.extract_solution(sol); - assert!(is.solution_size(&extracted).is_valid); - assert_eq!(extracted.iter().filter(|&&x| x == 1).count(), 1); - } -} -``` - -**Step 2: Run test, verify it fails** - -Run: `cargo test --all-features test_independentset_to_qubo` -Expected: compilation error (module not found) - -**Step 3: Write reduction implementation** - -File: `src/rules/independentset_qubo.rs` - -```rust -//! Reduction from IndependentSet to QUBO. -//! -//! Maximize Σ w_i·x_i s.t. x_i·x_j = 0 for (i,j) ∈ E -//! = Minimize -Σ w_i·x_i + P·Σ_{(i,j)∈E} x_i·x_j -//! -//! Q[i][i] = -w_i, Q[i][j] = P for edges. P = 1 + Σ w_i. - -use crate::models::graph::IndependentSet; -use crate::models::optimization::QUBO; -use crate::poly; -use crate::reduction; -use crate::rules::registry::ReductionOverhead; -use crate::rules::traits::{ReduceTo, ReductionResult}; -use crate::topology::SimpleGraph; -use crate::traits::Problem; -use crate::types::{ProblemSize, Unweighted}; - -#[derive(Debug, Clone)] -pub struct ReductionISToQUBO { - target: QUBO<f64>, - source_size: ProblemSize, -} - -impl ReductionResult for ReductionISToQUBO { - type Source = IndependentSet<SimpleGraph, Unweighted>; - type Target = QUBO<f64>; - - fn target_problem(&self) -> &Self::Target { &self.target } - fn extract_solution(&self, target_solution: &[usize]) -> Vec<usize> { - target_solution.to_vec() - } - fn source_size(&self) -> ProblemSize { self.source_size.clone() } - fn target_size(&self) -> ProblemSize { self.target.problem_size() } -} - -#[reduction( - source_graph = "SimpleGraph", - overhead = { ReductionOverhead::new(vec![("num_vars", poly!(num_vertices))]) } -)] -impl ReduceTo<QUBO<f64>> for IndependentSet<SimpleGraph, Unweighted> { - type Result = ReductionISToQUBO; - - fn reduce_to(&self) -> Self::Result { - let n = self.num_vertices(); - let edges = self.edges(); - let penalty = 1.0 + n as f64; // P > sum of unit weights - - let mut matrix = vec![vec![0.0; n]; n]; - for i in 0..n { - matrix[i][i] = -1.0; // -w_i (unit weight) - } - for (u, v) in &edges { - let (i, j) = if u < v { (*u, *v) } else { (*v, *u) }; - matrix[i][j] += penalty; - } - - ReductionISToQUBO { - target: QUBO::from_matrix(matrix), - source_size: self.problem_size(), - } - } -} - -#[cfg(test)] -#[path = "../unit_tests/rules/independentset_qubo.rs"] -mod tests; -``` - -**Step 4: Register in `src/rules/mod.rs`** - -Add after `mod spinglass_qubo;`: -```rust -mod independentset_qubo; -``` - -Add after `pub use spinglass_qubo::...`: -```rust -pub use independentset_qubo::ReductionISToQUBO; -``` - -**Step 5: Run tests** - -Run: `cargo test --all-features test_independentset_to_qubo` -Expected: PASS - -**Step 6: Run clippy + full test suite** - -Run: `make test clippy` -Expected: all pass, no warnings - -**Step 7: Commit** - -```bash -git add src/rules/independentset_qubo.rs src/unit_tests/rules/independentset_qubo.rs src/rules/mod.rs -git commit -m "feat: add IndependentSet → QUBO reduction" -``` - ---- - -### Task 2: VertexCovering → QUBO - -Minimize `Σ w_i·x_i + P·Σ_{(i,j)∈E} (1-x_i)(1-x_j)`. Expanding: `Q[i][i] = w_i - P·deg(i)`, `Q[i][j] = P`. - -**Files:** -- Create: `src/rules/vertexcovering_qubo.rs` -- Create: `src/unit_tests/rules/vertexcovering_qubo.rs` -- Modify: `src/rules/mod.rs` - -Same pattern as Task 1. Key differences: -- VC minimizes (same as QUBO), so no sign flip on objective -- Penalty enforces: every edge has at least one endpoint selected -- `Q[i][i] = 1.0 - penalty * degree(i)`, `Q[i][j] = penalty` for edges -- Penalty `P = 1 + n` (unit weights) -- Test: cycle graph C4 (4 vertices, 4 edges) → min VC = 2 vertices - -**Step 1: Write test** (same structure as Task 1) -**Step 2: Verify fails** -**Step 3: Implement** — struct `ReductionVCToQUBO`, same boilerplate -**Step 4: Register in mod.rs** -**Step 5-6: Test + clippy** -**Step 7: Commit** `"feat: add VertexCovering → QUBO reduction"` - ---- - -### Task 3: MaxCut → QUBO - -Maximize cut = Σ_{(i,j)∈E} w_ij·(x_i⊕x_j). Minimize negative: `Q[i][i] = -Σ_j w_ij`, `Q[i][j] = 2·w_ij` (upper triangular). - -Note: MaxCut edges carry weights. Use `self.edges()` which returns `Vec<(usize, usize, W)>`. - -**Files:** -- Create: `src/rules/maxcut_qubo.rs` -- Create: `src/unit_tests/rules/maxcut_qubo.rs` -- Modify: `src/rules/mod.rs` - -Key: MaxCut is `MaxCut<SimpleGraph, W>` with edge weights. For unweighted, use `MaxCut::unweighted(n, edges)`. - -- `Q[i][j] = 2·w_ij` for i < j (upper triangular; the `w_ij(x_i + x_j - 2x_ix_j)` formula) -- `Q[i][i] = -Σ_{j:(i,j)∈E} w_ij` -- Test: cycle C4 → max cut = 4 (all edges cut by bipartition) -- No penalty needed — MaxCut is unconstrained - -**Step 1-7:** Same flow. Commit: `"feat: add MaxCut → QUBO reduction"` - ---- - -### Task 4: Coloring (KColoring) → QUBO - -One-hot encoding: `x_{v,c} = 1` iff vertex v gets color c. QUBO index: `v*K + c`. - -- One-hot penalty: `P₁·Σ_v (1 - Σ_c x_{v,c})²` -- Edge penalty: `P₂·Σ_{(u,v)∈E} Σ_c x_{u,c}·x_{v,c}` -- QUBO has `n·K` variables - -**Special:** `KColoring<const K: usize, G, W>` uses const generic. For the reduction, we implement for a specific K (e.g., `K=3`). Or better: implement for generic K using the existing pattern. - -Actually, looking at `coloring_ilp.rs`, there are two reductions: -- `ReductionColoringToILP` for `Coloring<SimpleGraph, W>` (deprecated Coloring type?) -- `ReductionKColoringToILP<const K: usize, W>` for `KColoring<K, SimpleGraph, W>` - -We should implement for `KColoring<K, SimpleGraph, Unweighted>`. The `extract_solution` decodes one-hot: for each vertex, find which color bit is 1. - -The struct needs to store `num_vertices` and `K` for extraction. - -**Files:** -- Create: `src/rules/coloring_qubo.rs` -- Create: `src/unit_tests/rules/coloring_qubo.rs` -- Modify: `src/rules/mod.rs` - -**Test:** Triangle K3, 3 colors → exactly 6 valid colorings (3! permutations). - -**Step 1-7:** Same flow. Commit: `"feat: add KColoring → QUBO reduction"` - ---- - -### Task 5: SetPacking → QUBO - -Same structure as IS on intersection graph: `Q[i][i] = -w_i`, `Q[i][j] = P` for overlapping pairs. - -Use `self.overlapping_pairs()` to get conflicting set pairs. - -**Files:** -- Create: `src/rules/setpacking_qubo.rs` -- Create: `src/unit_tests/rules/setpacking_qubo.rs` -- Modify: `src/rules/mod.rs` - -**Test:** 3 sets with some overlaps → verify max packing found. - -**Step 1-7:** Same flow. Commit: `"feat: add SetPacking → QUBO reduction"` - ---- - -### Task 6: KSatisfiability (K=2) → QUBO - -Max-2-SAT penalty formulation. Each clause contributes to Q based on literal signs. - -For clause `(l₁ ∨ l₂)` where `l = x` or `l = ¬x`: -- `(x_i ∨ x_j)`: penalty `(1-x_i)(1-x_j)` = `1 - x_i - x_j + x_ix_j` -- `(¬x_i ∨ x_j)`: penalty `x_i(1-x_j)` = `x_i - x_ix_j` -- `(x_i ∨ ¬x_j)`: penalty `(1-x_i)x_j` = `x_j - x_ix_j` -- `(¬x_i ∨ ¬x_j)`: penalty `x_ix_j` - -CNFClause uses 1-indexed signed integers: positive = variable, negative = negated. E.g., `[1, -2]` = `(x₁ ∨ ¬x₂)`. - -**Files:** -- Create: `src/rules/ksatisfiability_qubo.rs` -- Create: `src/unit_tests/rules/ksatisfiability_qubo.rs` -- Modify: `src/rules/mod.rs` - -**Test:** 3 vars, 4 clauses → verify all clauses satisfied by extracted solution. - -**Step 1-7:** Same flow. Commit: `"feat: add KSatisfiability(K=2) → QUBO reduction"` - ---- - -### Task 7: ILP (binary) → QUBO - -Binary ILP: `min c^T x s.t. Ax ≤ b`. Feature-gated behind `ilp`. - -Formulation: `Q[i][i] += c_i` (objective) + `P·Σ_k (Σ_j a_{kj}·x_j - b_k)²` (constraint penalties). - -Expanding the quadratic penalty for constraint k: -- `Q[i][j] += P·a_{ki}·a_{kj}` for i ≤ j -- `Q[i][i] += P·a_{ki}·(a_{ki} - 2·b_k)` (diagonal adjustment) - -ILP fields are public: `self.constraints`, `self.objective`, `self.sense`, `self.bounds`, `self.num_vars`. - -Only valid for binary ILP (all bounds = [0,1]). Should assert this. - -For Maximize objectives, negate the objective coefficients (QUBO minimizes). - -**Files:** -- Create: `src/rules/ilp_qubo.rs` (with `#[cfg(feature = "ilp")]`) -- Create: `src/unit_tests/rules/ilp_qubo.rs` -- Modify: `src/rules/mod.rs` - -**Test:** Binary ILP with 3 vars, 2 constraints → verify feasible optimal found. - -**Step 1-7:** Same flow. Commit: `"feat: add ILP (binary) → QUBO reduction"` - ---- - -### Task 8: Integration Tests - -Add integration tests in `tests/suites/reductions.rs` that load JSON ground truth from `tests/data/qubo/` and compare against Rust reductions. - -**Files:** -- Modify: `tests/suites/reductions.rs` - -For each reduction, add a module like: -```rust -mod is_qubo_reductions { - use super::*; - - #[test] - fn test_is_to_qubo_ground_truth() { - // Load JSON, create source problem, reduce, verify QUBO matrix and optimal match - } -} -``` - -**Commit:** `"test: add integration tests for QUBO reductions against ground truth"` - ---- - -### Task 9: Example Program - -Create `examples/qubo_reductions.rs` demonstrating all 7 reductions with practical stories. - -**File:** Create `examples/qubo_reductions.rs` - -Each demo: -1. Create a small practical instance (e.g., "Find the largest non-conflicting set of wireless towers") -2. Reduce to QUBO -3. Solve with BruteForce -4. Extract and explain the solution - -Run: `cargo run --example qubo_reductions --features ilp` - -**Commit:** `"docs: add QUBO reductions example program"` - ---- - -### Task 10: Paper Documentation - -Update `docs/paper/reductions.typ` with 7 new theorems. - -**File:** Modify `docs/paper/reductions.typ` - -For each reduction: -1. Add theorem in Section 3.1 (trivial reductions — these are standard penalty formulations) -2. Add proof with the QUBO formulation -3. Add Rust code example (from `examples/qubo_reductions.rs`) -4. Update summary table with overhead and reference - -Also update `@def:qubo` to list new "Reduces from" links. - -Run: `make export-graph && make paper` - -**Commit:** `"docs: add QUBO reduction theorems and examples to paper"` - ---- - -### Task 11: Final Verification - -```bash -make test # All tests pass -make clippy # No warnings -make export-graph # Reduction graph updated -make paper # Paper compiles -make coverage # >95% for new code -``` - -**Commit:** any final fixups - ---- - -## Key Reference Files - -| Purpose | Path | -|---------|------| -| Model pattern | `src/rules/spinglass_qubo.rs` | -| Test pattern | `src/unit_tests/rules/spinglass_qubo.rs` | -| Module registry | `src/rules/mod.rs` | -| Integration tests | `tests/suites/reductions.rs` | -| ILP feature gate | `src/rules/mod.rs:28-45` (example) | -| Ground truth JSON | `tests/data/qubo/*.json` | -| Paper | `docs/paper/reductions.typ` | -| IS model | `src/models/graph/independent_set.rs` | -| VC model | `src/models/graph/vertex_covering.rs` | -| MaxCut model | `src/models/graph/max_cut.rs` | -| KColoring model | `src/models/graph/kcoloring.rs` | -| SetPacking model | `src/models/set/set_packing.rs` | -| KSat model | `src/models/satisfiability/ksat.rs` | -| ILP model | `src/models/optimization/ilp.rs` | diff --git a/docs/plans/2026-02-10-dry-mdbook-docs.md b/docs/plans/2026-02-10-dry-mdbook-docs.md deleted file mode 100644 index 5682c6a0b..000000000 --- a/docs/plans/2026-02-10-dry-mdbook-docs.md +++ /dev/null @@ -1,372 +0,0 @@ -# DRY mdBook Documentation Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Remove hardcoded problem/reduction docs from mdBook, make the interactive graph the primary navigation hub linking to rustdoc, and deploy CLAUDE.md in the book. - -**Architecture:** The interactive reduction graph on the introduction page becomes the main entry point for browsing problems and reductions — nodes link to rustdoc struct pages, edges link to rustdoc reduction module pages. All category-specific problem pages and reduction detail pages are removed. Topology is explained as part of variant design in the introduction. CLAUDE.md is auto-copied into the book at build time. - -**Tech Stack:** Rust (registry metadata), JavaScript (Cytoscape.js graph), mdBook, Makefile - ---- - -### Task 1: Add `doc_path` to `NodeJson` and `EdgeJson` - -**Files:** -- Modify: `src/rules/graph.rs:35-62` (NodeJson, EdgeJson structs and `to_json()`) -- Modify: `src/rules/registry.rs:37-49` (ReductionEntry struct) - -**Step 1: Add `doc_module` field to `ReductionEntry`** - -Add a `doc_module: &'static str` field to `ReductionEntry` in `src/rules/registry.rs`. This holds the module name for rustdoc linking (e.g., `"vertexcovering_independentset"`). - -**Step 2: Update the `#[reduction]` macro to emit `doc_module`** - -In the proc macro crate (`problemreductions-macros`), update the `#[reduction]` attribute to automatically set `doc_module` from the source file stem. The macro already generates `inventory::submit!` calls — add `doc_module: env!("CARGO_PKG_NAME")` or derive from the impl block's module context. Alternatively, use `module_path!()` at the submit site and strip the crate prefix at export time. - -**Step 3: Add `doc_path` to `NodeJson` and `EdgeJson`** - -```rust -pub struct NodeJson { - pub name: String, - pub variant: BTreeMap<String, String>, - pub category: String, - pub doc_path: String, // e.g., "models/graph/independent_set" -} - -pub struct EdgeJson { - pub source: VariantRef, - pub target: VariantRef, - pub bidirectional: bool, - pub doc_module: String, // e.g., "vertexcovering_independentset" -} -``` - -**Step 4: Compute `doc_path` in `to_json()`** - -In `ReductionGraph::to_json()`, compute node doc paths from name + category using CamelCase→snake_case conversion: -- `"IndependentSet"` + `"graph"` → `"models/graph/independent_set"` -- `"CircuitSAT"` + `"specialized"` → `"models/specialized/circuit_sat"` - -For edges, use the `doc_module` from `ReductionEntry`. - -**Step 5: Verify JSON output** - -Run: `cargo run --example export_graph` - -Check that `docs/paper/reduction_graph.json` contains `doc_path` on nodes and `doc_module` on edges. - -**Step 6: Run tests** - -Run: `make test` -Expected: All pass (JSON format change doesn't break existing tests since tests don't check JSON fields exhaustively). - -**Step 7: Commit** - -```bash -git add src/rules/graph.rs src/rules/registry.rs -git commit -m "feat(registry): add doc_path to reduction graph JSON nodes and edges" -``` - ---- - -### Task 2: Update interactive graph to link to rustdoc - -**Files:** -- Modify: `docs/src/introduction.md` (Cytoscape.js code) - -**Step 1: Update double-click handler for nodes** - -Change the existing `dbltap` handler to construct rustdoc URLs from `doc_path`: - -```javascript -cy.on('dbltap', 'node', function(evt) { - var d = evt.target.data(); - if (d.doc_path) { - window.location.href = '../' + d.doc_path + '/index.html'; - } -}); -``` - -Note: the `../` prefix is needed because the introduction page is at `book/introduction.html` and rustdoc is at `book/api/problemreductions/...`. - -**Step 2: Pass `doc_path` through to Cytoscape node data** - -Update the node creation to include `doc_path` from the JSON: - -```javascript -baseNodes.forEach(function(n) { - elements.push({ data: { - id: n.name, label: n.name, - category: n.category || 'other', - doc_path: n.doc_path || '' - }}); -}); -``` - -**Step 3: Add edge click handler** - -Add a tap handler for edges that opens the reduction module docs: - -```javascript -cy.on('tap', 'edge', function(evt) { - var d = evt.target.data(); - if (d.doc_module) { - window.open('../api/problemreductions/rules/' + d.doc_module + '/index.html', '_blank'); - } -}); -``` - -**Step 4: Pass `doc_module` through to Cytoscape edge data** - -Update edge creation to include `doc_module` from the JSON. - -**Step 5: Update tooltip to show link hints** - -Update the node tooltip to say "Double-click to view API docs" and add edge hover tooltip showing source→target with "Click to view reduction docs". - -**Step 6: Update instruction text** - -Change the hint text to mention all three interactions: path finding (click two nodes), API docs (double-click node), reduction docs (click edge). - -**Step 7: Test locally** - -Run: `make doc` then open `docs/book/introduction.html` - -Verify: -- Double-click a node → navigates to rustdoc page for that problem -- Click an edge → opens reduction module docs in new tab -- Path finding still works (click node → click another node) - -**Step 8: Commit** - -```bash -git add docs/src/introduction.md -git commit -m "feat(docs): link graph nodes to rustdoc, edges to reduction modules" -``` - ---- - -### Task 3: Add variant design and topology to introduction - -**Files:** -- Modify: `docs/src/introduction.md` - -**Step 1: Add paper download link** - -After the library description, add: -```markdown -For theoretical background and correctness proofs, see the [PDF manual](https://codingthrust.github.io/problem-reductions/reductions.pdf). -``` - -**Step 2: Add variant design section** - -After the reduction graph, add a "## Problem Variants" section explaining: -- Problems are parameterized by graph type `G` and weight type `W` -- Base variant: `IndependentSet` (SimpleGraph, unweighted) -- Graph variants: `IndependentSet/GridGraph`, `IndependentSet/UnitDiskGraph` -- Weighted variants: `IndependentSet/Weighted` -- How variants appear as separate nodes in the reduction graph - -Keep this concise — 2-3 paragraphs max. Link to rustdoc for full API details. - -**Step 3: Add topology overview** - -Within the variant design section, briefly describe the 4 graph types: -- **SimpleGraph**: Standard adjacency-based graph (petgraph) -- **GridGraph**: Regular grid layout -- **UnitDiskGraph**: Geometric graph with distance threshold (for quantum hardware mapping) -- **HyperGraph**: Edges can connect any number of vertices - -One sentence each, with links to their rustdoc pages. - -**Step 4: Remove problem categories table** - -The existing table duplicating problem names is no longer needed — the interactive graph serves this purpose. Remove the `## Problem Categories` section. - -**Step 5: Commit** - -```bash -git add docs/src/introduction.md -git commit -m "docs: add variant design, topology overview, and paper link to introduction" -``` - ---- - -### Task 4: Merge reductions content into getting-started - -**Files:** -- Modify: `docs/src/getting-started.md` -- Delete: `docs/src/reductions/using.md` - -**Step 1: Add reduction chaining section** - -Add the "Chaining Reductions" example from `reductions/using.md` to `getting-started.md`, after the existing "Applying Reductions" section. - -**Step 2: Add type safety section** - -Add the "Type Safety" compile_fail example from `reductions/using.md`. - -**Step 3: Update "Next Steps" links** - -Replace the old links to problems/reductions with: -- Link to the interactive reduction graph on the introduction page -- Link to the API reference -- Link to solvers - -**Step 4: Commit** - -```bash -git add docs/src/getting-started.md -git commit -m "docs: merge reduction usage into getting-started" -``` - ---- - -### Task 5: Remove obsolete pages and update SUMMARY.md - -**Files:** -- Delete: `docs/src/problems/index.md` -- Delete: `docs/src/problems/graph.md` -- Delete: `docs/src/problems/satisfiability.md` -- Delete: `docs/src/problems/optimization.md` -- Delete: `docs/src/problems/set.md` -- Delete: `docs/src/problems/specialized.md` -- Delete: `docs/src/reductions/index.md` -- Delete: `docs/src/reductions/using.md` -- Delete: `docs/src/reductions/available.md` -- Delete: `docs/src/reductions/graph.md` -- Delete: `docs/src/topology.md` -- Modify: `docs/src/SUMMARY.md` - -**Step 1: Update SUMMARY.md** - -```markdown -[Introduction](./introduction.md) - -# User Guide - -- [Getting Started](./getting-started.md) -- [Solvers](./solvers.md) -- [File I/O](./io.md) - -# Developer Guide - -- [API Reference](./api.md) -- [Contributing](./contributing.md) -- [CLAUDE.md](./claude.md) -``` - -**Step 2: Delete obsolete files** - -Remove all files listed above. Keep `docs/src/reductions/reduction_graph.json` (still needed by the interactive graph). - -**Step 3: Verify mdBook builds** - -Run: `make doc` -Expected: No broken internal links (mdBook warns on dead links). - -**Step 4: Commit** - -```bash -git add -A docs/src/ -git commit -m "refactor(docs): remove hardcoded problem/reduction pages" -``` - ---- - -### Task 6: Deploy CLAUDE.md in mdBook and update contributing - -**Files:** -- Modify: `Makefile` (doc and mdbook targets) -- Modify: `docs/src/contributing.md` -- Create (build step): `docs/src/claude.md` (auto-copied from `.claude/CLAUDE.md`) - -**Step 1: Update Makefile to copy CLAUDE.md** - -In both `doc:` and `mdbook:` targets, add before `mdbook build`: -```makefile -cp .claude/CLAUDE.md docs/src/claude.md -``` - -**Step 2: Rewrite contributing.md** - -Replace the current content with a human-oriented guide that does NOT duplicate CLAUDE.md: -- Welcome message and link to [CLAUDE.md](./claude.md) for commands and architecture -- How to find issues to work on (link to GitHub issues) -- PR workflow: branch → implement → test → PR -- Authorship recognition (from README) -- Link to the AI-assisted workflow (`/issue-to-pr` skill) -- Code style summary (fmt, clippy, doc comments — one sentence each) - -**Step 3: Add CLAUDE.md to .gitignore for docs/src/** - -Add `docs/src/claude.md` to `.gitignore` so the auto-copied file isn't committed. - -**Step 4: Verify build** - -Run: `make doc` -Expected: CLAUDE.md page renders in the book, contributing.md links to it correctly. - -**Step 5: Commit** - -```bash -git add Makefile docs/src/contributing.md .gitignore -git commit -m "docs: deploy CLAUDE.md in mdBook, rewrite contributing page" -``` - ---- - -### Task 7: Update README contributing section - -**Files:** -- Modify: `README.md` - -**Step 1: Expand contributing section** - -Keep the existing authorship recognition and step-by-step workflow. Add/update: -- Link to the deployed CLAUDE.md page in the mdBook for commands and architecture -- Keep the `make help` reference -- Ensure no overlap with CLAUDE.md content (don't list individual make commands or architecture details) - -**Step 2: Commit** - -```bash -git add README.md -git commit -m "docs: update README contributing section with CLAUDE.md link" -``` - ---- - -### Task 8: Final verification - -**Step 1: Run full checks** - -```bash -make test clippy -make doc -make paper -``` - -Expected: All pass, no warnings. - -**Step 2: Verify in browser** - -Open `docs/book/index.html` and check: -- Interactive graph renders with all nodes -- Double-click node → rustdoc struct page loads -- Click edge → reduction module docs open -- Path finding works -- Variant design section is present -- Paper download link works -- Getting-started includes reduction chaining examples -- Contributing links to CLAUDE.md page -- CLAUDE.md page renders correctly -- No dead internal links - -**Step 3: Commit any final fixes** - -```bash -git add -A -git commit -m "fix: final verification fixes for DRY mdBook docs" -``` diff --git a/docs/plans/2026-02-10-improve-example-instances-impl.md b/docs/plans/2026-02-10-improve-example-instances-impl.md deleted file mode 100644 index 96ec62916..000000000 --- a/docs/plans/2026-02-10-improve-example-instances-impl.md +++ /dev/null @@ -1,708 +0,0 @@ -# Improve Example Instances — Implementation Plan (v2) - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Replace trivial instances (P4, K3, 2-3 variable SAT) in all 30 reduction examples with non-trivial instances (Petersen graph, 5-variable SAT, etc.) to produce better data for the paper. - -**Architecture:** Each example file is independent. We swap the instance construction code, update doc comments and print statements, then re-run to regenerate JSON. The code structure (reduce → solve → extract → export) stays identical. We use the existing `petersen()` and `octahedral()` helpers from `src/topology/small_graphs.rs`. - -**Tech Stack:** Rust, `problemreductions` crate, `small_graphs` module, BruteForce/ILPSolver - ---- - -## Shared Constants - -These concrete instances are referenced across multiple tasks. - -### Petersen Graph -```rust -use problemreductions::topology::small_graphs::petersen; -let (num_vertices, edges) = petersen(); -// 10 vertices, 15 edges, 3-regular, MIS=4, VC=6, Matching=5, DomSet=3, χ=3 -``` - -### Octahedron -```rust -use problemreductions::topology::small_graphs::octahedral; -let (num_vertices, edges) = octahedral(); -// 6 vertices, 12 edges, K_{2,2,2}, Clique=3 -``` - -### 3-SAT Instance (5 variables, 7 clauses) -```rust -let sat = Satisfiability::<i32>::new( - 5, - vec![ - CNFClause::new(vec![1, 2, -3]), // x1 ∨ x2 ∨ ¬x3 - CNFClause::new(vec![-1, 3, 4]), // ¬x1 ∨ x3 ∨ x4 - CNFClause::new(vec![2, -4, 5]), // x2 ∨ ¬x4 ∨ x5 - CNFClause::new(vec![-2, 3, -5]), // ¬x2 ∨ x3 ∨ ¬x5 - CNFClause::new(vec![1, -3, 5]), // x1 ∨ ¬x3 ∨ x5 - CNFClause::new(vec![-1, -2, 4]), // ¬x1 ∨ ¬x2 ∨ x4 - CNFClause::new(vec![3, -4, -5]), // x3 ∨ ¬x4 ∨ ¬x5 - ], -); -``` -**Note:** After implementing, verify this has 2-8 satisfying assignments (not 0 and not all 32). If it has 0 solutions, flip one literal sign. If too many, add a clause. - -### Petersen SpinGlass (frustrated, ±1 couplings) -```rust -use problemreductions::topology::small_graphs::petersen; -let (n, edges) = petersen(); -// Alternating ±1 couplings → frustration on odd cycles -let couplings: Vec<((usize, usize), f64)> = edges.iter().enumerate() - .map(|(i, &(u, v))| ((u, v), if i % 2 == 0 { 1.0 } else { -1.0 })) - .collect(); -let sg = SpinGlass::<SimpleGraph, f64>::new(n, couplings, vec![0.0; n]); -``` - ---- - -## Task 1: MIS → QUBO, ILP, MVC, MSP (4 files) - -**Files:** -- Modify: `examples/reduction_maximumindependentset_to_qubo.rs` -- Modify: `examples/reduction_maximumindependentset_to_ilp.rs` -- Modify: `examples/reduction_maximumindependentset_to_minimumvertexcover.rs` -- Modify: `examples/reduction_maximumindependentset_to_maximumsetpacking.rs` - -**Step 1: Update all 4 files** - -In each file, replace the graph construction with: -```rust -use problemreductions::topology::small_graphs::petersen; - -// Petersen graph: 10 vertices, 15 edges, 3-regular -let (num_vertices, edges) = petersen(); -let is = MaximumIndependentSet::<SimpleGraph, i32>::new(num_vertices, edges.clone()); -``` - -Replace the old: -```rust -let edges = vec![(0, 1), (1, 2), (2, 3)]; -let is = MaximumIndependentSet::<SimpleGraph, i32>::new(4, edges.clone()); -``` - -Update doc comments: -- `//! - Instance: Petersen graph with 10 vertices and 15 edges` -- `//! - Source: MaximumIndependentSet with maximum size 4` -- `//! - QUBO variables: 10` (for the QUBO example) - -Update print statements to say "Petersen graph" instead of "path P4". - -**Step 2: Run all 4 examples to verify** - -```bash -cargo run --all-features --example reduction_maximumindependentset_to_qubo -cargo run --all-features --example reduction_maximumindependentset_to_ilp -cargo run --all-features --example reduction_maximumindependentset_to_minimumvertexcover -cargo run --all-features --example reduction_maximumindependentset_to_maximumsetpacking -``` - -Expected: Each prints solutions with MIS size 4. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_maximumindependentset_*.rs -git commit -m "examples: use Petersen graph for MIS reductions" -``` - ---- - -## Task 2: MVC → ILP, QUBO, MIS, MSC (4 files) - -**Files:** -- Modify: `examples/reduction_minimumvertexcover_to_ilp.rs` -- Modify: `examples/reduction_minimumvertexcover_to_qubo.rs` -- Modify: `examples/reduction_minimumvertexcover_to_maximumindependentset.rs` -- Modify: `examples/reduction_minimumvertexcover_to_minimumsetcovering.rs` - -**Step 1: Update all 4 files** - -Replace graph construction with: -```rust -use problemreductions::topology::small_graphs::petersen; - -let (num_vertices, edges) = petersen(); -let vc = MinimumVertexCover::<SimpleGraph, i32>::new(num_vertices, edges.clone()); -``` - -Replace old C4 `vec![(0, 1), (1, 2), (2, 3), (0, 3)]` or K3 `vec![(0, 1), (1, 2), (0, 2)]`. - -Update doc comments to reference Petersen, VC=6. - -**Step 2: Run all 4 examples** - -```bash -cargo run --all-features --example reduction_minimumvertexcover_to_ilp -cargo run --all-features --example reduction_minimumvertexcover_to_qubo -cargo run --all-features --example reduction_minimumvertexcover_to_maximumindependentset -cargo run --all-features --example reduction_minimumvertexcover_to_minimumsetcovering -``` - -Expected: VC size 6. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_minimumvertexcover_*.rs -git commit -m "examples: use Petersen graph for MVC reductions" -``` - ---- - -## Task 3: Matching + DomSet → ILP, MSP (3 files) - -**Files:** -- Modify: `examples/reduction_maximummatching_to_ilp.rs` -- Modify: `examples/reduction_maximummatching_to_maximumsetpacking.rs` -- Modify: `examples/reduction_minimumdominatingset_to_ilp.rs` - -**Step 1: Update all 3 files** - -For Matching (uses `unweighted` constructor): -```rust -use problemreductions::topology::small_graphs::petersen; - -let (num_vertices, edges) = petersen(); -let matching = MaximumMatching::<SimpleGraph, i32>::unweighted(num_vertices, edges.clone()); -``` - -For DominatingSet: -```rust -use problemreductions::topology::small_graphs::petersen; - -let (num_vertices, edges) = petersen(); -let ds = MinimumDominatingSet::<SimpleGraph, i32>::new(num_vertices, edges.clone()); -``` - -Update doc comments: Matching=5 (perfect), DomSet=3. - -**Step 2: Run all 3 examples** - -```bash -cargo run --all-features --example reduction_maximummatching_to_ilp -cargo run --all-features --example reduction_maximummatching_to_maximumsetpacking -cargo run --all-features --example reduction_minimumdominatingset_to_ilp -``` - -Expected: Matching size 5, DomSet size 3. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_maximummatching_*.rs examples/reduction_minimumdominatingset_*.rs -git commit -m "examples: use Petersen graph for Matching and DomSet reductions" -``` - ---- - -## Task 4: Coloring + MaxCut (3 files) - -**Files:** -- Modify: `examples/reduction_coloring_to_ilp.rs` -- Modify: `examples/reduction_coloring_to_qubo.rs` -- Modify: `examples/reduction_maxcut_to_spinglass.rs` - -**Step 1: Update Coloring files** - -Petersen has chromatic number 3, so `KColoring::<3, ...>` is correct: -```rust -use problemreductions::topology::small_graphs::petersen; - -let (num_vertices, edges) = petersen(); -let kc = KColoring::<3, SimpleGraph, i32>::new(num_vertices, edges.clone()); -``` - -**IMPORTANT for `coloring_to_qubo`**: KColoring::<3> on Petersen creates a QUBO with 10×3 = 30 variables. BruteForce on 30 variables (2^30 ≈ 1B) is too slow. Either: -- (a) Use a smaller graph for coloring QUBO examples (e.g., `house()` — 5 vertices, χ=3, QUBO=15 vars), or -- (b) Accept slow runtime (~minutes). - -**Recommended**: Use `house()` (5 vertices, 6 edges, χ=3) for `coloring_to_qubo` only. Keep Petersen for `coloring_to_ilp`. - -**Step 1b: Update MaxCut file** - -MaxCut with unit weights on Petersen: -```rust -use problemreductions::topology::small_graphs::petersen; - -let (num_vertices, edges) = petersen(); -let maxcut = MaxCut::<SimpleGraph, i32>::unweighted(num_vertices, edges.clone()); -``` - -**Step 2: Run all 3 examples** - -```bash -cargo run --all-features --example reduction_coloring_to_ilp -cargo run --all-features --example reduction_coloring_to_qubo -cargo run --all-features --example reduction_maxcut_to_spinglass -``` - -Expected: Coloring solutions with 3 colors. MaxCut solution. No panics. Verify `coloring_to_qubo` completes within ~2 minutes. - -**Step 3: Commit** - -```bash -git add examples/reduction_coloring_*.rs examples/reduction_maxcut_*.rs -git commit -m "examples: use Petersen graph for Coloring and MaxCut reductions" -``` - ---- - -## Task 5: MaximumClique → ILP (1 file) - -**Files:** -- Modify: `examples/reduction_maximumclique_to_ilp.rs` - -**Step 1: Update instance** - -```rust -use problemreductions::topology::small_graphs::octahedral; - -// Octahedron: 6 vertices, 12 edges, K_{2,2,2}, clique number = 3 -let (num_vertices, edges) = octahedral(); -let clique = MaximumClique::<SimpleGraph, i32>::new(num_vertices, edges.clone()); -``` - -Update doc comments: "Octahedron graph (K_{2,2,2}) with 6 vertices and 12 edges, max clique size 3." - -**Step 2: Run example** - -```bash -cargo run --all-features --example reduction_maximumclique_to_ilp -``` - -Expected: Clique of size 3. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_maximumclique_to_ilp.rs -git commit -m "examples: use octahedron for MaximumClique reduction" -``` - ---- - -## Task 6: Standalone Set Problems (3 files) - -**Files:** -- Modify: `examples/reduction_maximumsetpacking_to_qubo.rs` -- Modify: `examples/reduction_maximumsetpacking_to_ilp.rs` -- Modify: `examples/reduction_minimumsetcovering_to_ilp.rs` - -**Step 1: Update instances** - -These are standalone set problems (not derived from graph reductions). Replace the trivial 3-set instances with 6 sets over 8 elements: - -For MaximumSetPacking: -```rust -let sets = vec![ - vec![0, 1, 2], // S0 - vec![2, 3, 4], // S1 (overlaps S0 at 2) - vec![4, 5, 6], // S2 (overlaps S1 at 4) - vec![6, 7, 0], // S3 (overlaps S2 at 6, S0 at 0) - vec![1, 3, 5], // S4 (overlaps S0,S1,S2) - vec![0, 4, 7], // S5 (overlaps S0,S1,S3) -]; -let sp = MaximumSetPacking::<i32>::new(sets.clone()); -``` - -For MinimumSetCovering (same 6 sets, universe size 8): -```rust -let sets = vec![ - vec![0, 1, 2], - vec![2, 3, 4], - vec![4, 5, 6], - vec![6, 7, 0], - vec![1, 3, 5], - vec![0, 4, 7], -]; -let sc = MinimumSetCovering::<i32>::new(8, sets.clone()); -``` - -**Step 2: Run all 3 examples** - -```bash -cargo run --all-features --example reduction_maximumsetpacking_to_qubo -cargo run --all-features --example reduction_maximumsetpacking_to_ilp -cargo run --all-features --example reduction_minimumsetcovering_to_ilp -``` - -Expected: MSP finds max packing (disjoint sets), MSC finds min covering. No panics. - -**IMPORTANT for `msp_to_qubo`**: SetPacking with 6 sets → QUBO with 6 variables. BruteForce on 6 vars is instant. Good. - -**Step 3: Commit** - -```bash -git add examples/reduction_maximumsetpacking_*.rs examples/reduction_minimumsetcovering_to_ilp.rs -git commit -m "examples: use 6-set instances for SetPacking and SetCovering" -``` - ---- - -## Task 7: SAT → MIS, Coloring, DomSet, kSAT (4 files) - -**Files:** -- Modify: `examples/reduction_sat_to_maximumindependentset.rs` -- Modify: `examples/reduction_sat_to_coloring.rs` -- Modify: `examples/reduction_sat_to_minimumdominatingset.rs` -- Modify: `examples/reduction_sat_to_ksat.rs` - -**Step 1: Update all 4 files** - -Replace the SAT construction with the shared 3-SAT instance (see Shared Constants above). Use the exact same formula in all 4 files: -```rust -let sat = Satisfiability::<i32>::new( - 5, - vec![ - CNFClause::new(vec![1, 2, -3]), - CNFClause::new(vec![-1, 3, 4]), - CNFClause::new(vec![2, -4, 5]), - CNFClause::new(vec![-2, 3, -5]), - CNFClause::new(vec![1, -3, 5]), - CNFClause::new(vec![-1, -2, 4]), - CNFClause::new(vec![3, -4, -5]), - ], -); -``` - -Update doc comments to reference "5-variable, 7-clause 3-SAT formula". - -**IMPORTANT**: For `sat_to_mis`, the target MIS graph has 7×3 = 21 vertices. BruteForce on 21 variables (2^21 ≈ 2M) is fast. For `sat_to_coloring`, the target has more variables (2×5 + 7×3 = 31 — too slow for BruteForce). If `sat_to_coloring` is too slow, reduce to 5 clauses instead of 7. For `sat_to_mds`, check the target variable count similarly. - -Run each example and verify it completes within ~30 seconds. If any is too slow, trim the formula to fewer clauses for that specific example. - -**Step 2: Run all 4 examples** - -```bash -cargo run --all-features --example reduction_sat_to_maximumindependentset -cargo run --all-features --example reduction_sat_to_coloring -cargo run --all-features --example reduction_sat_to_minimumdominatingset -cargo run --all-features --example reduction_sat_to_ksat -``` - -Expected: SAT solutions found, reductions verified. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_sat_*.rs -git commit -m "examples: use 5-variable 3-SAT instance for SAT reductions" -``` - ---- - -## Task 8: kSAT → QUBO (1 file) - -**Files:** -- Modify: `examples/reduction_ksatisfiability_to_qubo.rs` - -**Step 1: Update instance** - -Use the same 3-SAT formula but as `KSatisfiability::<3, i32>`: -```rust -let clauses = vec![ - CNFClause::new(vec![1, 2, -3]), - CNFClause::new(vec![-1, 3, 4]), - CNFClause::new(vec![2, -4, 5]), - CNFClause::new(vec![-2, 3, -5]), - CNFClause::new(vec![1, -3, 5]), - CNFClause::new(vec![-1, -2, 4]), - CNFClause::new(vec![3, -4, -5]), -]; -let ksat = KSatisfiability::<3, i32>::new(5, clauses); -``` - -**IMPORTANT**: kSAT → QUBO creates auxiliary variables. Check that the QUBO has ≤ ~25 variables. If too many, reduce to fewer clauses. - -**Step 2: Run example** - -```bash -cargo run --all-features --example reduction_ksatisfiability_to_qubo -``` - -Expected: QUBO solutions found, kSAT verified. No panics. Completes within ~1 minute. - -**Step 3: Commit** - -```bash -git add examples/reduction_ksatisfiability_to_qubo.rs -git commit -m "examples: use 5-variable 3-SAT instance for kSAT-to-QUBO" -``` - ---- - -## Task 9: SpinGlass ↔ QUBO ↔ MaxCut (3 files) - -**Files:** -- Modify: `examples/reduction_spinglass_to_qubo.rs` -- Modify: `examples/reduction_qubo_to_spinglass.rs` -- Modify: `examples/reduction_spinglass_to_maxcut.rs` - -**Step 1: Update SpinGlass → QUBO and SpinGlass → MaxCut** - -Both start with a SpinGlass instance. Use the Petersen SpinGlass (see Shared Constants): -```rust -use problemreductions::topology::small_graphs::petersen; - -let (n, edges) = petersen(); -let couplings: Vec<((usize, usize), f64)> = edges.iter().enumerate() - .map(|(i, &(u, v))| ((u, v), if i % 2 == 0 { 1.0 } else { -1.0 })) - .collect(); -let sg = SpinGlass::<SimpleGraph, f64>::new(n, couplings, vec![0.0; n]); -``` - -For `spinglass_to_maxcut`, use `i32` weights instead of `f64`: -```rust -let couplings: Vec<((usize, usize), i32)> = edges.iter().enumerate() - .map(|(i, &(u, v))| ((u, v), if i % 2 == 0 { 1 } else { -1 })) - .collect(); -let sg = SpinGlass::<SimpleGraph, i32>::new(n, couplings, vec![0; n]); -``` - -**Step 1b: Update QUBO → SpinGlass** - -Create a 10-variable QUBO directly. Use a sparse upper-triangular matrix on Petersen edges: -```rust -use problemreductions::topology::small_graphs::petersen; - -let (n, edges) = petersen(); -let mut matrix = vec![vec![0.0; n]; n]; -// Diagonal: linear terms -for i in 0..n { - matrix[i][i] = -1.0 + 0.2 * i as f64; -} -// Off-diagonal: quadratic terms on Petersen edges -for (idx, &(u, v)) in edges.iter().enumerate() { - let (i, j) = if u < v { (u, v) } else { (v, u) }; - matrix[i][j] = if idx % 2 == 0 { 2.0 } else { -1.5 }; -} -let qubo = QUBO::from_matrix(matrix.clone()); -``` - -**Step 2: Run all 3 examples** - -```bash -cargo run --all-features --example reduction_spinglass_to_qubo -cargo run --all-features --example reduction_qubo_to_spinglass -cargo run --all-features --example reduction_spinglass_to_maxcut -``` - -Expected: Solutions found, round-trip verified. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_spinglass_*.rs examples/reduction_qubo_to_spinglass.rs -git commit -m "examples: use Petersen topology for SpinGlass/QUBO/MaxCut" -``` - ---- - -## Task 10: ILP → QUBO (1 file) - -**Files:** -- Modify: `examples/reduction_ilp_to_qubo.rs` - -**Step 1: Update instance** - -Replace the trivial 3-variable ILP with a 6-variable binary knapsack: -```rust -let ilp = ILP::binary( - 6, - vec![ - // Knapsack weight constraint: 3x0 + 2x1 + 5x2 + 4x3 + 2x4 + 3x5 ≤ 10 - LinearConstraint::le( - vec![(0, 3.0), (1, 2.0), (2, 5.0), (3, 4.0), (4, 2.0), (5, 3.0)], - 10.0, - ), - // Category limit: x0 + x1 + x2 ≤ 2 - LinearConstraint::le(vec![(0, 1.0), (1, 1.0), (2, 1.0)], 2.0), - // Category limit: x3 + x4 + x5 ≤ 2 - LinearConstraint::le(vec![(3, 1.0), (4, 1.0), (5, 1.0)], 2.0), - ], - vec![(0, 10.0), (1, 7.0), (2, 12.0), (3, 8.0), (4, 6.0), (5, 9.0)], - ObjectiveSense::Maximize, -); -``` - -**IMPORTANT**: ILP → QUBO adds slack variables. Check the QUBO has ≤ ~25 variables. If too many, reduce the RHS of the knapsack constraint (fewer slack bits needed). - -**Step 2: Run example** - -```bash -cargo run --all-features --example reduction_ilp_to_qubo -``` - -Expected: QUBO solution extracts to a valid knapsack solution. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_ilp_to_qubo.rs -git commit -m "examples: use 6-variable knapsack for ILP-to-QUBO" -``` - ---- - -## Task 11: CircuitSAT → SpinGlass (1 file) - -**Files:** -- Modify: `examples/reduction_circuit_to_spinglass.rs` - -**Step 1: Update instance** - -Replace the single AND gate with a full adder circuit (1-bit addition with carry): -```rust -use problemreductions::models::specialized::Circuit; - -// Full adder: inputs a, b, cin → outputs sum, cout -// sum = a XOR b XOR cin -// cout = (a AND b) OR (cin AND (a XOR b)) -let circuit = Circuit::new(vec![ - // Intermediate: t = a XOR b - Assignment::new( - vec!["t".to_string()], - BooleanExpr::xor(vec![BooleanExpr::var("a"), BooleanExpr::var("b")]), - ), - // sum = t XOR cin - Assignment::new( - vec!["sum".to_string()], - BooleanExpr::xor(vec![BooleanExpr::var("t"), BooleanExpr::var("cin")]), - ), - // ab = a AND b - Assignment::new( - vec!["ab".to_string()], - BooleanExpr::and(vec![BooleanExpr::var("a"), BooleanExpr::var("b")]), - ), - // cin_t = cin AND t - Assignment::new( - vec!["cin_t".to_string()], - BooleanExpr::and(vec![BooleanExpr::var("cin"), BooleanExpr::var("t")]), - ), - // cout = ab OR cin_t - Assignment::new( - vec!["cout".to_string()], - BooleanExpr::or(vec![BooleanExpr::var("ab"), BooleanExpr::var("cin_t")]), - ), -]); -let circuit_sat = CircuitSAT::<i32>::new(circuit); -``` - -This gives ~7 variables (a, b, cin, t, sum, ab, cin_t, cout). BruteForce on 8 variables is instant. - -**Step 2: Run example** - -```bash -cargo run --all-features --example reduction_circuit_to_spinglass -``` - -Expected: SpinGlass solutions found. No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_circuit_to_spinglass.rs -git commit -m "examples: use full adder circuit for CircuitSAT-to-SpinGlass" -``` - ---- - -## Task 12: Factoring → Circuit, ILP (2 files) - -**Files:** -- Modify: `examples/reduction_factoring_to_circuit.rs` -- Modify: `examples/reduction_factoring_to_ilp.rs` - -**Step 1: Update instances** - -Replace `Factoring::new(2, 2, 6)` and `Factoring::new(4, 4, 15)` with: -```rust -let factoring = Factoring::new(3, 3, 35); -// Factor 35 = 5 × 7, 3-bit factors, 6 binary variables -``` - -For `factoring_to_circuit.rs`: update the variable name format strings. The current code uses `format!("p{}", i + 1)` and `format!("q{}", i + 1)` which should still work for 3-bit factors. - -Update doc comments: "Factor 35 = 5 × 7 (m=3 bits, n=3 bits)". - -For `factoring_to_ilp.rs`: the ILPSolver is used (not BruteForce). This should handle 3×3 fine. - -**Step 2: Run both examples** - -```bash -cargo run --all-features --example reduction_factoring_to_circuit -cargo run --all-features --example reduction_factoring_to_ilp -``` - -Expected: Finds factors 5 and 7 (and 7 and 5). No panics. - -**Step 3: Commit** - -```bash -git add examples/reduction_factoring_*.rs -git commit -m "examples: use factoring 35=5×7 for Factoring reductions" -``` - ---- - -## Task 13: Regenerate JSON and Full Verification - -**Files:** -- All `docs/paper/examples/*.json` and `*.result.json` (auto-generated) - -**Step 1: Run all examples to regenerate JSON** - -```bash -make examples -``` - -If no `make examples` target exists, run manually: -```bash -for ex in $(ls examples/reduction_*.rs | sed 's|examples/||;s|\.rs||'); do - cargo run --all-features --example "$ex" -done -``` - -**Step 2: Run full test suite** - -```bash -make test -``` - -Expected: All tests pass. The QUBO ground truth tests in `tests/data/qubo/` use different instances than the examples, so they should not be affected. - -**Step 3: Run clippy** - -```bash -make clippy -``` - -Expected: No warnings. - -**Step 4: Verify JSON files updated** - -```bash -git diff --stat docs/paper/examples/ -``` - -Expected: All 60 JSON files (30 × `.json` + 30 × `.result.json`) show changes. - -**Step 5: Commit generated files** - -```bash -git add docs/paper/examples/ -git commit -m "chore: regenerate example JSON with improved instances" -``` - ---- - -## Parallel Execution Groups - -Tasks 1-12 are independent and can run in parallel. Task 13 depends on all others completing. - -**Group A (graph, can run in parallel):** Tasks 1, 2, 3, 4, 5 -**Group B (non-graph, can run in parallel):** Tasks 6, 7, 8, 9, 10, 11, 12 -**Group C (verification, sequential after A+B):** Task 13 diff --git a/docs/plans/2026-02-10-improve-example-instances.md b/docs/plans/2026-02-10-improve-example-instances.md deleted file mode 100644 index b5b08ca49..000000000 --- a/docs/plans/2026-02-10-improve-example-instances.md +++ /dev/null @@ -1,94 +0,0 @@ -# Improve Example Instances - -**Date**: 2026-02-10 -**Branch**: TBD -**Status**: Design approved - -## Problem - -All 30 reduction examples use trivially small instances (P4 path graph, K3 triangle, 2-3 variable SAT formulas). The P4 path graph alone appears in 8 examples. These produce unserious-looking data for the paper and don't illustrate interesting reduction behavior. - -## Design Decisions - -- **Purpose**: Examples are primarily data generators for the Typst paper (JSON export) -- **Size range**: 6-10 variables per instance -- **Strategy**: One canonical graph (Petersen) for all graph problems, with exceptions only where necessary -- **Solver**: BruteForce, so target problem must have ≤ ~25 variables - -## Instance Plan - -### 1. Petersen Graph (10 vertices, 15 edges) - -Canonical graph for all graph-based problems. 3-regular, non-bipartite, girth 5. - -**Properties**: MIS=4, VC=6, Matching=5 (perfect), DominatingSet=3, ChromaticNumber=3, Clique=2. - -**Edge list**: `(0,1), (0,4), (0,5), (1,2), (1,6), (2,3), (2,7), (3,4), (3,8), (4,9), (5,7), (5,8), (6,8), (6,9), (7,9)` - -**Used by** (20 examples): -- `mis_to_qubo`, `mis_to_ilp`, `mis_to_mvc`, `mis_to_msp` -- `mvc_to_ilp`, `mvc_to_qubo`, `mvc_to_mis`, `mvc_to_msc` -- `mm_to_ilp`, `mm_to_msp` -- `mds_to_ilp` -- `coloring_to_ilp`, `coloring_to_qubo` -- `maxcut_to_spinglass` -- `spinglass_to_maxcut` (MaxCut on Petersen topology) - -### 2. Octahedron (6 vertices, 12 edges) - -For MaximumClique example. Complete tripartite K_{2,2,2}. Clique number = 3. - -**Used by**: `mclique_to_ilp` - -### 3. Random 3-SAT (5 variables, ~7 clauses) - -Hand-picked instance with ~2-4 satisfying assignments. Ratio ~1.4. Compact enough to display inline in the paper. Target MIS graph has 21 vertices (BruteForce feasible). - -**Used by** (4 examples): -- `sat_to_mis`, `sat_to_coloring`, `sat_to_mds`, `sat_to_ksat` - -### 4. Petersen SpinGlass (10 spins, 15 couplings) - -SpinGlass on Petersen graph topology with random ±1 couplings (frustrated). Shared between SpinGlass ↔ QUBO ↔ MaxCut conversions. - -**Used by** (3 examples): -- `spinglass_to_qubo`, `spinglass_to_maxcut`, `qubo_to_spinglass` - -### 5. Knapsack/Assignment ILP (6-8 variables) - -Proper knapsack or assignment problem with non-trivial constraints and slack variables. - -**Used by**: `ilp_to_qubo` - -### 6. 2-bit Adder Circuit - -Multi-gate circuit (replaces single AND gate). Shows meaningful CircuitSAT structure. - -**Used by**: `circuit_to_spinglass` - -### 7. Factor 35 = 5 x 7 (3-bit x 3-bit) - -Product of two primes, 6 binary variables. BruteForce feasible (2^6 = 64). - -**Used by** (2 examples): -- `factoring_to_circuit`, `factoring_to_ilp` - -### 8. 5-variable 3-SAT (reused from #3) - -Same SAT instance used for kSAT → QUBO. - -**Used by**: `ksatisfiability_to_qubo` - -## Paper Display - -For Petersen-based reductions producing 10x10 QUBO matrices: show as compact figures rather than inline matrices, or show key properties (dimension, sparsity, penalty weight) instead of full matrix. - -## Scope - -Only instance data changes. The example structure (create -> reduce -> solve -> extract -> export JSON) stays the same. No API changes. - -## Verification - -```bash -make test clippy export-graph -``` diff --git a/docs/plans/2026-02-10-json-schema-and-interactive-viz.md b/docs/plans/2026-02-10-json-schema-and-interactive-viz.md deleted file mode 100644 index 7818daca9..000000000 --- a/docs/plans/2026-02-10-json-schema-and-interactive-viz.md +++ /dev/null @@ -1,459 +0,0 @@ -# JSON Schema & Interactive Visualization Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Replace manual Rust struct definitions in typst with auto-generated JSON (issue #33), and move the reduction diagram from typst to an interactive Cytoscape.js page in mdBook (issue #34). - -**Architecture:** Use `inventory` crate (already used for `ReductionEntry`) to auto-register problem schema entries. Export to JSON via an example binary. For the interactive diagram, embed Cytoscape.js in an mdBook page with path exploration. - -**Tech Stack:** Rust (`inventory`, `serde_json`), Typst, Cytoscape.js (CDN), mdBook - ---- - -### Task 1: Add `FieldInfo` and `ProblemSchemaEntry` to registry - -**Files:** -- Modify: `src/registry/info.rs` -- Create: `src/registry/schema.rs` -- Modify: `src/registry/mod.rs` - -**Step 1: Add `FieldInfo` struct to `src/registry/info.rs`** - -Add after `ProblemInfo` impl block (after line 198): - -```rust -/// Description of a struct field for JSON schema export. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct FieldInfo { - /// Field name as it appears in the Rust struct. - pub name: &'static str, - /// Type name (e.g., "Vec<W>", "UnGraph<(), ()>"). - pub type_name: &'static str, - /// Human-readable description of what this field represents. - pub description: &'static str, -} -``` - -Add `fields` to `ProblemInfo`: -```rust -pub fields: &'static [FieldInfo], -``` - -Update `ProblemInfo::new` to initialize `fields: &[]` and add builder: -```rust -pub const fn with_fields(mut self, fields: &'static [FieldInfo]) -> Self { - self.fields = fields; - self -} -``` - -**Step 2: Create `src/registry/schema.rs`** - -```rust -//! Problem schema registration via inventory. - -use super::FieldInfo; -use serde::Serialize; - -/// A registered problem schema entry for static inventory registration. -pub struct ProblemSchemaEntry { - /// Problem name (e.g., "IndependentSet"). - pub name: &'static str, - /// Category (e.g., "graph", "optimization"). - pub category: &'static str, - /// Human-readable description. - pub description: &'static str, - /// Struct fields. - pub fields: &'static [FieldInfo], -} - -inventory::collect!(ProblemSchemaEntry); - -/// JSON-serializable problem schema. -#[derive(Debug, Clone, Serialize)] -pub struct ProblemSchemaJson { - pub name: String, - pub category: String, - pub description: String, - pub fields: Vec<FieldInfoJson>, -} - -/// JSON-serializable field info. -#[derive(Debug, Clone, Serialize)] -pub struct FieldInfoJson { - pub name: String, - pub type_name: String, - pub description: String, -} - -/// Collect all registered problem schemas into JSON-serializable form. -pub fn collect_schemas() -> Vec<ProblemSchemaJson> { - let mut schemas: Vec<ProblemSchemaJson> = inventory::iter::<ProblemSchemaEntry> - .into_iter() - .map(|entry| ProblemSchemaJson { - name: entry.name.to_string(), - category: entry.category.to_string(), - description: entry.description.to_string(), - fields: entry - .fields - .iter() - .map(|f| FieldInfoJson { - name: f.name.to_string(), - type_name: f.type_name.to_string(), - description: f.description.to_string(), - }) - .collect(), - }) - .collect(); - schemas.sort_by(|a, b| a.name.cmp(&b.name)); - schemas -} -``` - -**Step 3: Update `src/registry/mod.rs`** - -Add `mod schema;` and re-export `ProblemSchemaEntry`, `ProblemSchemaJson`, `collect_schemas`. - -**Step 4: Run `cargo check`** - -Run: `cargo check` -Expected: Compiles with no errors. - -**Step 5: Commit** - -``` -feat(registry): add FieldInfo and ProblemSchemaEntry for auto-discovery -``` - ---- - -### Task 2: Register all problem schemas via inventory - -**Files:** -- Modify: `src/models/graph/independent_set.rs` -- Modify: `src/models/graph/vertex_covering.rs` -- Modify: `src/models/graph/max_cut.rs` -- Modify: `src/models/graph/kcoloring.rs` -- Modify: `src/models/graph/dominating_set.rs` -- Modify: `src/models/graph/matching.rs` -- Modify: `src/models/graph/clique.rs` -- Modify: `src/models/graph/maximal_is.rs` -- Modify: `src/models/set/set_packing.rs` -- Modify: `src/models/set/set_covering.rs` -- Modify: `src/models/optimization/spin_glass.rs` -- Modify: `src/models/optimization/qubo.rs` -- Modify: `src/models/optimization/ilp.rs` -- Modify: `src/models/satisfiability/sat.rs` -- Modify: `src/models/satisfiability/ksat.rs` -- Modify: `src/models/specialized/circuit.rs` -- Modify: `src/models/specialized/factoring.rs` -- Modify: `src/models/specialized/bmf.rs` -- Modify: `src/models/specialized/biclique_cover.rs` -- Modify: `src/models/specialized/paintshop.rs` - -**Step 1: Add `inventory::submit!` to each problem file** - -Pattern for each file — add at module level (outside impl blocks): - -```rust -use crate::registry::{FieldInfo, ProblemSchemaEntry}; - -inventory::submit! { - ProblemSchemaEntry { - name: "IndependentSet", - category: "graph", - description: "Find maximum weight independent set in a graph", - fields: &[ - FieldInfo { name: "graph", type_name: "UnGraph<(), ()>", description: "The underlying graph G=(V,E)" }, - FieldInfo { name: "weights", type_name: "Vec<W>", description: "Vertex weights w: V -> R" }, - ], - } -} -``` - -Full list of registrations (one per problem): - -| Problem | Category | Fields | -|---------|----------|--------| -| IndependentSet | graph | graph: UnGraph<(), ()>, weights: Vec\<W\> | -| VertexCovering | graph | graph: UnGraph<(), ()>, weights: Vec\<W\> | -| MaxCut | graph | graph: UnGraph<(), W>, edge_weights: Vec\<W\> | -| KColoring | graph | num_colors: usize, graph: UnGraph<(), ()> | -| DominatingSet | graph | graph: UnGraph<(), ()>, weights: Vec\<W\> | -| Matching | graph | graph: UnGraph<(), W>, edge_weights: Vec\<W\> | -| Clique | graph | graph: UnGraph<(), ()>, weights: Vec\<W\> | -| MaximalIS | graph | graph: UnGraph<(), ()>, weights: Vec\<W\> | -| SetPacking | set | sets: Vec<Vec\<usize\>>, weights: Vec\<W\> | -| SetCovering | set | universe_size: usize, sets: Vec<Vec\<usize\>>, weights: Vec\<W\> | -| SpinGlass | optimization | graph: G, couplings: Vec\<W\>, fields: Vec\<W\> | -| QUBO | optimization | num_vars: usize, matrix: Vec<Vec\<W\>> | -| ILP | optimization | num_vars: usize, bounds: Vec\<VarBounds\>, constraints: Vec\<LinearConstraint\>, objective: Vec<(usize, f64)>, sense: ObjectiveSense | -| Satisfiability | satisfiability | num_vars: usize, clauses: Vec\<CNFClause\>, weights: Vec\<W\> | -| KSatisfiability | satisfiability | num_vars: usize, clauses: Vec\<CNFClause\>, weights: Vec\<W\> | -| CircuitSAT | satisfiability | circuit: Circuit, variables: Vec\<String\>, weights: Vec\<W\> | -| Factoring | specialized | m: usize, n: usize, target: u64 | -| BMF | specialized | matrix: Vec<Vec\<bool\>>, m: usize, n: usize, k: usize | -| BicliqueCover | specialized | left_size: usize, right_size: usize, edges: Vec<(usize, usize)>, k: usize | -| PaintShop | specialized | sequence_indices: Vec\<usize\>, car_labels: Vec\<String\>, is_first: Vec\<bool\>, num_cars: usize | - -**Step 2: Run `cargo test`** - -Run: `make test` -Expected: All tests pass. - -**Step 3: Commit** - -``` -feat(models): register problem schemas for all 20 problem types -``` - ---- - -### Task 3: Create export_schemas example and Makefile target - -**Files:** -- Create: `examples/export_schemas.rs` -- Modify: `Makefile` - -**Step 1: Create `examples/export_schemas.rs`** - -```rust -use problemreductions::registry::collect_schemas; -use std::path::Path; - -fn main() { - let schemas = collect_schemas(); - println!("Collected {} problem schemas", schemas.len()); - - let output_path = Path::new("docs/paper/problem_schemas.json"); - if let Some(parent) = output_path.parent() { - std::fs::create_dir_all(parent).expect("Failed to create output directory"); - } - - let json = serde_json::to_string(&schemas).expect("Failed to serialize"); - std::fs::write(output_path, &json).expect("Failed to write file"); - println!("Exported to: {}", output_path.display()); -} -``` - -**Step 2: Run the example** - -Run: `cargo run --example export_schemas` -Expected: Creates `docs/paper/problem_schemas.json` with 20 entries. - -**Step 3: Add Makefile target** - -Add after `export-graph` section: - -```makefile -# Export problem schemas to JSON -export-schemas: - cargo run --example export_schemas -``` - -Update `paper` target to include `export-schemas`: - -```makefile -paper: examples - cargo run --example export_graph - cargo run --example export_schemas - cd docs/paper && typst compile reductions.typ reductions.pdf -``` - -**Step 4: Run `make paper`** - -Run: `make paper` -Expected: Generates JSON and compiles PDF. - -**Step 5: Commit** - -``` -feat: add export_schemas example and Makefile target -``` - ---- - -### Task 4: Update typst to read struct definitions from JSON - -**Files:** -- Modify: `docs/paper/reductions.typ` - -**Step 1: Add JSON loader and rendering function** - -Add after `load-results` (line 22): - -```typst -#let problem-schemas = json("problem_schemas.json") - -// Render a problem's Rust struct from the JSON schema -#let render-struct(name) = { - let schema = problem-schemas.find(s => s.name == name) - if schema == none { return } - let s = schema - let fields = s.fields.map(f => " " + f.name + ": " + f.type_name + ",").join("\n") - raw("pub struct " + name + " {\n" + fields + "\n}", lang: "rust", block: true) -} -``` - -**Step 2: Replace all manual Rust code blocks** - -For each definition in the typst file, replace the manual ` ```rust ... ``` ` struct block with `#render-struct("ProblemName")`. - -Example — for IndependentSet (lines 112-117), replace: -``` - ```rust - pub struct IndependentSet<W = i32> { - graph: UnGraph<(), ()>, - weights: Vec<W>, - } - ``` -``` -with: -``` - #render-struct("IndependentSet") -``` - -Apply the same pattern for all 15+ struct definitions found in the file: -IndependentSet, VertexCovering, MaxCut, Coloring (KColoring), DominatingSet, Matching, SetPacking, SetCovering, SpinGlass, QUBO, ILP (including VarBounds and LinearConstraint), Satisfiability (including CNFClause), KSatisfiability, CircuitSAT, Factoring. - -**Note:** For ILP, Satisfiability, and CircuitSAT which show supporting structs (VarBounds, LinearConstraint, CNFClause) alongside the main struct, register these as separate schema entries or keep the supporting struct definitions inline. - -**Step 3: Run `make paper`** - -Run: `make paper` -Expected: PDF compiles with struct definitions rendered from JSON. - -**Step 4: Commit** - -``` -refactor(paper): render struct definitions from JSON schema (#33) -``` - ---- - -### Task 5: Remove reduction diagram from typst - -**Files:** -- Modify: `docs/paper/reductions.typ` -- Delete: `docs/paper/reduction-diagram.typ` - -**Step 1: Remove the import** - -Remove line 2: -```typst -#import "reduction-diagram.typ": reduction-graph, graph-data -``` - -**Step 2: Remove `graph-data` references** - -Line 90 references `#graph-data.edges.len()` and `#graph-data.nodes.len()`. Rewrite this sentence to not reference the graph data, or load the JSON directly for just the counts: -```typst -#let graph-data = json("reduction_graph.json") -``` -Keep this minimal load only if the edge/node counts are needed in the text. Otherwise remove the sentence referencing the figure. - -**Step 3: Remove the figure** - -Remove lines 96-99: -```typst -#figure( - reduction-graph(width: 18mm, height: 14mm), - caption: [Reduction graph. Colors: ...] -) <fig:reduction-graph> -``` - -Remove the `@fig:reduction-graph` reference from line 90. - -**Step 4: Delete `reduction-diagram.typ`** - -Delete `docs/paper/reduction-diagram.typ`. - -**Step 5: Run `make paper`** - -Run: `make paper` -Expected: PDF compiles without the diagram. - -**Step 6: Commit** - -``` -refactor(paper): remove reduction diagram from typst (#34) -``` - ---- - -### Task 6: Create interactive Cytoscape.js page in mdBook - -**Files:** -- Modify: `docs/src/SUMMARY.md` -- Modify: `docs/src/reductions/graph.md` (replace static mermaid diagram) -- Modify: `Makefile` (add export-graph to doc target) - -**Step 1: Update Makefile doc target** - -```makefile -doc: - cargo run --example export_graph - cp docs/paper/reduction_graph.json docs/src/reductions/ - mdbook build docs -``` - -**Step 2: Replace mermaid diagram in `docs/src/reductions/graph.md`** - -Replace the mermaid code block and legend (lines 7-96) with embedded Cytoscape.js HTML. Keep the Usage, Registered Reductions, and API sections. - -The embedded HTML should include: -- Controls bar: instructions text + "Clear Path" button -- Canvas div for Cytoscape -- Cytoscape.js loaded from CDN (`https://unpkg.com/cytoscape@3/dist/cytoscape.min.js`) -- Inline `<script>` that: - 1. Fetches `reduction_graph.json` (relative path) - 2. Filters to base problem nodes (empty variant) - 3. Deduplicates edges by base name, detects bidirectionality - 4. Creates Cytoscape instance with cose layout - 5. Styles nodes by category color - 6. Adds hover tooltips - 7. Implements two-click path highlighting using `cy.elements().dijkstra()` - 8. Handles directed vs bidirectional edge arrows - -Category colors (matching the original typst diagram): -- graph: `#c8f0c8` -- set: `#f0c8c8` -- optimization: `#f0f0a0` -- satisfiability: `#c8c8f0` -- specialized: `#f0c8e0` - -**Step 3: Test locally** - -Run: `mdbook serve docs --open` -Expected: Interactive diagram renders with pan/zoom, colored nodes, hover tooltips, click-to-highlight paths. - -**Step 4: Commit** - -``` -feat(docs): interactive reduction diagram with Cytoscape.js (#34) -``` - ---- - -### Task 7: Final verification - -**Step 1: Run full check** - -Run: `make test clippy` -Expected: All pass. - -**Step 2: Build paper** - -Run: `make paper` -Expected: PDF compiles. Struct definitions rendered from JSON. No reduction diagram. - -**Step 3: Build docs** - -Run: `make doc` -Expected: mdBook builds. Interactive diagram works. - -**Step 4: Update CLAUDE.md if needed** - -Add `export-schemas` to the commands section if appropriate. - -**Step 5: Commit any final cleanup** diff --git a/docs/plans/2026-02-10-polish-reductions-implementation.md b/docs/plans/2026-02-10-polish-reductions-implementation.md deleted file mode 100644 index af9919067..000000000 --- a/docs/plans/2026-02-10-polish-reductions-implementation.md +++ /dev/null @@ -1,1060 +0,0 @@ -# Polish Reductions.typ Documentation - Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Polish reductions.typ by connecting math to code, adding theorem labels, expanding proofs, and creating 28 example files with JSON export. - -**Architecture:** 5-pass approach: (1) Add theorem labels, (2) Enhance problem definitions, (3) Expand proofs + add links, (4) Create example files, (5) Verify compilation. - -**Tech Stack:** Typst (docs/paper/reductions.typ), Rust (examples/), serde_json (JSON export) - -**Design Reference:** `docs/plans/2026-02-10-polish-reductions-paper.md` - ---- - -## Overview - -This plan implements a comprehensive documentation polish consisting of: -- **Pass 1:** Add `<thm:*>` labels to all theorems for cross-referencing -- **Pass 2:** Enhance 15 problem definitions with field mappings and theorem links -- **Pass 3:** Expand trivial reduction proofs and add GitHub example links -- **Pass 4:** Create 28 example files with JSON export (split existing qubo_reductions.rs) -- **Pass 5:** Verify `make paper` compiles and all examples run - -**Important Notes:** -- pkgref/ contains reference implementations (ProblemReductions.jl, UnitDiskMapping.jl, qubogen) -- Unit Disk Mapping already has export_petersen_mapping.rs (polished) -- Each example exports JSON to docs/paper/examples/ for paper code blocks - ---- - -## PASS 1: Add Theorem Labels - -### Task 1.1: Scan and catalog all theorems - -**Files:** -- Read: `docs/paper/reductions.typ:312-940` - -**Step 1: Extract theorem titles** - -```bash -cd /Users/liujinguo/rcode/problemreductions -grep -n "^#theorem\[" docs/paper/reductions.typ -``` - -Expected: List of line numbers and theorem titles (e.g., "*(IS $arrow.l.r$ VC)*") - -**Step 2: Create theorem-to-label mapping** - -Create temporary file listing all theorems with proposed labels: -- IS ↔ VC → `<thm:is-to-vc>` and `<thm:vc-to-is>` -- IS → SetPacking → `<thm:is-to-setpacking>` -- SpinGlass ↔ QUBO → `<thm:spinglass-to-qubo>` and `<thm:qubo-to-spinglass>` -- etc. - -Save to: `docs/paper/.theorem_labels.txt` (temporary scratch file) - -**Step 3: Verify no duplicate labels** - -Check for uniqueness in the mapping file. - -Expected: All labels unique - - -### Task 1.2: Add labels to trivial reduction theorems - -**Files:** -- Modify: `docs/paper/reductions.typ:314-370` - -**Step 1: Add label to IS ↔ VC theorem** - -Find the theorem block starting with `*(IS $arrow.l.r$ VC)*` and add label after the closing bracket: - -```typst -#theorem[ - *(IS $arrow.l.r$ VC)* $S subset.eq V$ is independent iff $V backslash S$ is a vertex cover, with $|"IS"| + |"VC"| = |V|$. [_Problems:_ @def:independent-set, @def:vertex-cover.] -] <thm:is-to-vc> -``` - -**Step 2: Add label to IS → SetPacking theorem** - -```typst -#theorem[ - *(IS $arrow.r$ Set Packing)* Construct $U = E$, $S_v = {e in E : v in e}$, $w(S_v) = w(v)$. Then $I$ is independent iff ${S_v : v in I}$ is a packing. [_Problems:_ @def:independent-set, @def:set-packing.] -] <thm:is-to-setpacking> -``` - -**Step 3: Add labels to remaining trivial reductions** - -Continue adding labels to: -- VC → SetCovering: `<thm:vc-to-setcovering>` -- Matching → SetPacking: `<thm:matching-to-setpacking>` -- SpinGlass ↔ QUBO: `<thm:spinglass-to-qubo>` (bidirectional) - -**Step 4: Commit trivial reduction labels** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add theorem labels to trivial reductions - -Added <thm:*> labels to IS↔VC, IS→SetPacking, VC→SetCovering, -Matching→SetPacking, SpinGlass↔QUBO for cross-referencing" -``` - - -### Task 1.3: Add labels to penalty-method QUBO theorems - -**Files:** -- Modify: `docs/paper/reductions.typ:384-560` - -**Step 1: Add label to IS → QUBO theorem** - -```typst -#theorem[ - *(IS $arrow.r$ QUBO)* Given $G = (V, E)$ with weights $w$, construct upper-triangular $Q in RR^(n times n)$ with $Q_(i i) = -w_i$ and $Q_(i j) = P$ for $(i,j) in E$ ($i < j$), where $P = 1 + sum_i w_i$. Then minimizing $f(bold(x)) = sum_i Q_(i i) x_i + sum_(i<j) Q_(i j) x_i x_j$ is equivalent to maximizing the IS objective. [_Problems:_ @def:independent-set, @def:qubo.] -] <thm:is-to-qubo> -``` - -**Step 2: Add labels to remaining QUBO reductions** - -Continue with: -- VC → QUBO: `<thm:vc-to-qubo>` -- KColoring → QUBO: `<thm:coloring-to-qubo>` -- SetPacking → QUBO: `<thm:setpacking-to-qubo>` -- 2-SAT → QUBO: `<thm:ksatisfiability-to-qubo>` -- Binary ILP → QUBO: `<thm:ilp-to-qubo>` - -**Step 3: Commit penalty-method labels** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add theorem labels to penalty-method QUBO reductions - -Added labels for IS→QUBO, VC→QUBO, Coloring→QUBO, SetPacking→QUBO, -KSatisfiability→QUBO, ILP→QUBO" -``` - - -### Task 1.4: Add labels to non-trivial reduction theorems - -**Files:** -- Modify: `docs/paper/reductions.typ:562-720` - -**Step 1: Add labels to SAT reduction theorems** - -```typst -#theorem[ - *(SAT $arrow.r$ IS)* @karp1972 Given CNF $phi$ with $m$ clauses, construct graph $G$ such that $phi$ is satisfiable iff $G$ has an IS of size $m$. [_Problems:_ @def:satisfiability, @def:independent-set.] -] <thm:sat-to-is> - -#theorem[ - *(SAT $arrow.r$ 3-Coloring)* @garey1979 Given CNF $phi$, construct graph $G$ such that $phi$ is satisfiable iff $G$ is 3-colorable. [_Problems:_ @def:satisfiability, @def:coloring.] -] <thm:sat-to-coloring> - -#theorem[ - *(SAT $arrow.r$ Dominating Set)* @garey1979 Given CNF $phi$ with $n$ variables and $m$ clauses, $phi$ is satisfiable iff the constructed graph has a dominating set of size $n$. [_Problems:_ @def:satisfiability, @def:dominating-set.] -] <thm:sat-to-dominatingset> - -#theorem[ - *(SAT $arrow.l.r$ $k$-SAT)* @cook1971 @garey1979 Any SAT formula converts to $k$-SAT ($k >= 3$) preserving satisfiability. [_Problems:_ @def:satisfiability, @def:k-sat.] -] <thm:sat-to-ksat> -``` - -**Step 2: Add labels to CircuitSAT and Factoring theorems** - -```typst -#theorem[ - *(CircuitSAT $arrow.r$ Spin Glass)* @whitfield2012 @lucas2014 Each gate maps to a gadget whose ground states encode valid I/O. [_Problems:_ @def:circuit-sat, @def:spin-glass.] -] <thm:circuit-to-spinglass> - -#theorem[ - *(Factoring $arrow.r$ Circuit-SAT)* An array multiplier with output constrained to $N$ is satisfiable iff $N$ factors within bit bounds. _(Folklore; no canonical reference.)_ [_Problems:_ @def:factoring, @def:circuit-sat.] -] <thm:factoring-to-circuit> -``` - -**Step 3: Add labels to SpinGlass ↔ MaxCut and ILP theorems** - -```typst -#theorem[ - *(Spin Glass $arrow.l.r$ Max-Cut)* @barahona1982 @lucas2014 Ground states of Ising models correspond to maximum cuts. [_Problems:_ @def:spin-glass, @def:max-cut.] -] <thm:spinglass-to-maxcut> - -#theorem[ - *(Coloring $arrow.r$ ILP)* The $k$-coloring problem reduces to binary ILP with $|V| dot k$ variables and $|V| + |E| dot k$ constraints. [_Problems:_ @def:coloring, @def:ilp.] -] <thm:coloring-to-ilp> - -#theorem[ - *(Factoring $arrow.r$ ILP)* Integer factorization reduces to binary ILP using McCormick linearization with $O(m n)$ variables and constraints. [_Problems:_ @def:factoring, @def:ilp.] -] <thm:factoring-to-ilp> -``` - -**Step 4: Add label to Unit Disk Mapping theorem** - -```typst -#theorem[ - *(IS $arrow.r$ GridGraph IS)* @nguyen2023 Any MIS problem on a general graph $G$ can be reduced to MIS on a unit disk graph (King's subgraph) with at most quadratic overhead in the number of vertices. [_Problem:_ @def:independent-set.] -] <thm:is-to-gridgraph> -``` - -**Step 5: Commit non-trivial reduction labels** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add theorem labels to non-trivial reductions - -Added labels for SAT→IS, SAT→Coloring, SAT→DominatingSet, SAT↔KSAT, -CircuitSAT→SpinGlass, Factoring→Circuit, SpinGlass↔MaxCut, -Coloring→ILP, Factoring→ILP, IS→GridGraph" -``` - - -### Task 1.5: Verify all theorem labels added - -**Step 1: Count theorems and labels** - -```bash -cd /Users/liujinguo/rcode/problemreductions -echo "Theorems:" && grep -c "^#theorem\[" docs/paper/reductions.typ -echo "Labels:" && grep -c "] <thm:" docs/paper/reductions.typ -``` - -Expected: Same count for both (approximately 28 theorems) - -**Step 2: Check for label format consistency** - -```bash -grep "] <thm:" docs/paper/reductions.typ | sed 's/.*<thm:\(.*\)>/\1/' | sort -``` - -Expected: All labels use lowercase with hyphens, no duplicates - -**Step 3: Commit verification notes** - -Update `.theorem_labels.txt` with final mapping, commit as documentation. - -```bash -git add docs/paper/.theorem_labels.txt -git commit -m "docs: add theorem label mapping for reference" -``` - ---- - -## PASS 2: Enhance Problem Definitions - -### Task 2.1: Enhance Independent Set definition - -**Files:** -- Modify: `docs/paper/reductions.typ:65-78` - -**Step 1: Add field mapping paragraph after struct** - -Find the Independent Set definition and add after the Rust struct: - -```typst -#definition("Independent Set (IS)")[ - Given $G = (V, E)$ with vertex weights $w: V -> RR$, find $S subset.eq V$ maximizing $sum_(v in S) w(v)$ such that no two vertices in $S$ are adjacent: $forall u, v in S: (u, v) in.not E$. - - ```rust - pub struct IndependentSet<W = i32> { - graph: UnGraph<(), ()>, // The underlying graph - weights: Vec<W>, // Weights for each vertex - } - ``` - - Where `graph` represents $G = (V, E)$ with vertices indexed $0..n-1$, and `weights` stores vertex weights $w: V -> RR$ indexed by vertex ID. The solution is a subset $S subset.eq V$ represented as a `Vec<usize>` of vertex indices. - - _Implemented reductions:_ IS→SetPacking (@thm:is-to-setpacking), IS→QUBO (@thm:is-to-qubo), IS→ILP (@thm:is-to-ilp), VC→IS (@thm:is-to-vc), SAT→IS (@thm:sat-to-is). -] <def:independent-set> -``` - -**Step 2: Commit Independent Set enhancement** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance Independent Set definition with field mappings - -Added field mapping paragraph and replaced problem links with -theorem references for IS definition" -``` - - -### Task 2.2: Enhance Vertex Cover definition - -**Files:** -- Modify: `docs/paper/reductions.typ:80-93` - -**Step 1: Add field mapping and theorem links** - -```typst -#definition("Vertex Cover (VC)")[ - Given $G = (V, E)$ with vertex weights $w: V -> RR$, find $S subset.eq V$ minimizing $sum_(v in S) w(v)$ such that every edge has at least one endpoint in $S$: $forall (u, v) in E: u in S or v in S$. - - ```rust - pub struct VertexCovering<W = i32> { - graph: UnGraph<(), ()>, // The underlying graph - weights: Vec<W>, // Weights for each vertex - } - ``` - - Where `graph` represents $G = (V, E)$ with vertices indexed $0..n-1$, and `weights` stores vertex weights $w: V -> RR$ indexed by vertex ID. The solution is a subset $S subset.eq V$ represented as a `Vec<usize>` of vertex indices covering all edges. - - _Implemented reductions:_ VC→IS (@thm:is-to-vc), VC→SetCovering (@thm:vc-to-setcovering), VC→QUBO (@thm:vc-to-qubo), VC→ILP (@thm:vc-to-ilp), IS→VC (@thm:is-to-vc). -] <def:vertex-cover> -``` - -**Step 2: Commit Vertex Cover enhancement** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance Vertex Cover definition - -Added field mapping paragraph and theorem references" -``` - - -### Task 2.3: Enhance remaining graph problem definitions - -**Files:** -- Modify: `docs/paper/reductions.typ:95-153` - -**Step 1: Enhance Max-Cut, Graph Coloring, Dominating Set, Matching definitions** - -For each definition: -1. Add field mapping paragraph after struct -2. Replace "Reduces to/from" with "Implemented reductions" using theorem labels -3. Keep existing struct code block - -**Step 2: Commit graph problem enhancements** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance graph problem definitions - -Added field mappings and theorem references for Max-Cut, Coloring, -DominatingSet, Matching, Unit Disk Graph" -``` - - -### Task 2.4: Enhance set problem definitions - -**Files:** -- Modify: `docs/paper/reductions.typ:155-184` - -**Step 1: Enhance Set Packing and Set Covering** - -Add field mapping paragraphs: -- Set Packing: Explain `sets` as collection, `weights` as set weights -- Set Covering: Explain `universe_size`, `sets`, `weights` - -**Step 2: Commit set problem enhancements** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance set problem definitions - -Added field mappings and theorem references for SetPacking and SetCovering" -``` - - -### Task 2.5: Enhance optimization problem definitions - -**Files:** -- Modify: `docs/paper/reductions.typ:186-242` - -**Step 1: Enhance Spin Glass, QUBO, ILP definitions** - -- SpinGlass: Explain `num_spins`, `interactions` (J_ij), `fields` (h_i) -- QUBO: Explain `num_vars`, `matrix` (upper triangular Q) -- ILP: Explain `num_vars`, `bounds`, `constraints`, `objective`, `sense` - -**Step 2: Commit optimization problem enhancements** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance optimization problem definitions - -Added field mappings and theorem references for SpinGlass, QUBO, ILP" -``` - - -### Task 2.6: Enhance satisfiability problem definitions - -**Files:** -- Modify: `docs/paper/reductions.typ:244-310` - -**Step 1: Enhance SAT, K-SAT, Circuit-SAT, Factoring definitions** - -- SAT: Explain `num_vars`, `clauses` (CNFClause), `weights` -- K-SAT: Similar to SAT but with K literals per clause -- Circuit-SAT: Explain `circuit`, `variables`, `weights` -- Factoring: Explain `m`, `n`, `target` - -**Step 2: Commit satisfiability problem enhancements** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: enhance satisfiability problem definitions - -Added field mappings and theorem references for SAT, K-SAT, -CircuitSAT, Factoring" -``` - - -### Task 2.7: Verify all problem definitions enhanced - -**Step 1: Count field mapping paragraphs** - -```bash -grep -c "Where \`" docs/paper/reductions.typ -``` - -Expected: 15 (one per problem definition) - -**Step 2: Check all use theorem references** - -```bash -grep "_Implemented reductions:_" docs/paper/reductions.typ | wc -l -``` - -Expected: 15 - -**Step 3: Commit verification checkpoint** - -```bash -git commit --allow-empty -m "checkpoint: Pass 2 complete - all problem definitions enhanced" -``` - ---- - -## PASS 3: Expand Proofs and Add Example Links - -### Task 3.1: Expand trivial reduction proofs - -**Files:** -- Modify: `docs/paper/reductions.typ:316-370` - -**Step 1: Expand IS ↔ VC proof** - -Find the proof block and add variable mapping section at end: - -```typst -#proof[ - ($arrow.r.double$) If $S$ is independent, for any $(u, v) in E$, at most one endpoint lies in $S$, so $V backslash S$ covers all edges. ($arrow.l.double$) If $C$ is a cover, for any $u, v in V backslash C$, $(u, v) in.not E$, so $V backslash C$ is independent. - - _Variable mapping:_ Given IS instance $(G, w)$, create VC instance $(G, w)$ with identical graph and weights. Solution extraction: for VC solution $C$, return $S = V backslash C$. The complement operation preserves optimality since $|S| + |C| = |V|$ is constant. -] -``` - -**Step 2: Expand remaining trivial proofs** - -Add variable mapping sections to: -- IS → SetPacking: Explain edge set mapping -- VC → SetCovering: Explain edge coverage mapping -- Matching → SetPacking: Explain endpoint mapping -- SpinGlass ↔ QUBO: Already has formula expansion, just verify - -**Step 3: Commit expanded trivial proofs** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: expand trivial reduction proofs with variable mappings - -Added explicit variable mapping explanations to IS↔VC, IS→SetPacking, -VC→SetCovering, Matching→SetPacking proofs" -``` - - -### Task 3.2: Add GitHub links to trivial reductions - -**Files:** -- Modify: `docs/paper/reductions.typ:316-370` - -**Step 1: Add example link after IS ↔ VC proof** - -After the proof block, before the code example: - -```typst -See [reduction example](https://github.com/CodingThrust/problem-reductions/blob/main/examples/reduction_is_to_vc.rs). -``` - -**Step 2: Add links to all trivial reductions** - -Continue for: -- IS → SetPacking: `examples/reduction_is_to_setpacking.rs` -- VC → SetCovering: `examples/reduction_vc_to_setcovering.rs` -- Matching → SetPacking: `examples/reduction_matching_to_setpacking.rs` -- SpinGlass ↔ QUBO: `examples/reduction_spinglass_to_qubo.rs` and `reduction_qubo_to_spinglass.rs` - -**Step 3: Commit GitHub links** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add GitHub example links to trivial reductions" -``` - - -### Task 3.3: Add GitHub links to QUBO reductions - -**Files:** -- Modify: `docs/paper/reductions.typ:384-560` - -**Step 1: Add links after each QUBO reduction proof** - -After each proof/code example: -- IS → QUBO: `examples/reduction_is_to_qubo.rs` -- VC → QUBO: `examples/reduction_vc_to_qubo.rs` -- KColoring → QUBO: `examples/reduction_coloring_to_qubo.rs` -- SetPacking → QUBO: `examples/reduction_setpacking_to_qubo.rs` -- 2-SAT → QUBO: `examples/reduction_ksatisfiability_to_qubo.rs` -- Binary ILP → QUBO: `examples/reduction_ilp_to_qubo.rs` - -**Step 2: Commit QUBO reduction links** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add GitHub example links to QUBO reductions" -``` - - -### Task 3.4: Add GitHub links to non-trivial reductions - -**Files:** -- Modify: `docs/paper/reductions.typ:562-770` - -**Step 1: Add links to SAT reduction theorems** - -- SAT → IS: `examples/reduction_sat_to_is.rs` -- SAT → 3-Coloring: `examples/reduction_sat_to_coloring.rs` -- SAT → Dominating Set: `examples/reduction_sat_to_dominatingset.rs` -- SAT ↔ K-SAT: `examples/reduction_sat_to_ksat.rs` - -**Step 2: Add links to remaining reductions** - -- CircuitSAT → SpinGlass: `examples/reduction_circuit_to_spinglass.rs` -- Factoring → Circuit: `examples/reduction_factoring_to_circuit.rs` -- SpinGlass ↔ MaxCut: `examples/reduction_spinglass_to_maxcut.rs` and `reduction_maxcut_to_spinglass.rs` -- Coloring → ILP: `examples/reduction_coloring_to_ilp.rs` -- Factoring → ILP: `examples/reduction_factoring_to_ilp.rs` - -**Step 3: Add link to Unit Disk Mapping** - -After the Unit Disk Mapping theorem proof: - -```typst -See [unit disk mapping example](https://github.com/CodingThrust/problem-reductions/blob/main/examples/export_petersen_mapping.rs). -``` - -**Step 4: Commit non-trivial reduction links** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add GitHub example links to non-trivial reductions - -Added links for SAT, CircuitSAT, Factoring, SpinGlass/MaxCut, ILP -reductions and Unit Disk Mapping" -``` - - -### Task 3.5: Verify all theorems have example links - -**Step 1: Count GitHub links** - -```bash -grep -c "See \[.*example\](https://github.com" docs/paper/reductions.typ -``` - -Expected: 28+ (one per reduction theorem) - -**Step 2: Verify link format consistency** - -```bash -grep "See \[.*example\]" docs/paper/reductions.typ -``` - -Expected: All use same format with `/blob/main/examples/` - -**Step 3: Commit verification checkpoint** - -```bash -git commit --allow-empty -m "checkpoint: Pass 3 complete - all proofs expanded and linked" -``` - ---- - -## PASS 4: Create Example Files - -### Task 4.1: Split qubo_reductions.rs into separate files - -**Files:** -- Read: `examples/qubo_reductions.rs` -- Create: 6 new files in `examples/` - -**Step 1: Create reduction_is_to_qubo.rs** - -Extract the `demo_independent_set()` function and create standalone example following the template from design doc. Include: -- Detailed docstring with mathematical equivalence -- Problem transformation metrics output -- JSON export to `docs/paper/examples/is_to_qubo.json` - -**Step 2: Test the example compiles and runs** - -```bash -cargo check --example reduction_is_to_qubo -cargo run --example reduction_is_to_qubo -``` - -Expected: Compiles, runs, outputs metrics, exports JSON - -**Step 3: Commit IS → QUBO example** - -```bash -git add examples/reduction_is_to_qubo.rs -git commit -m "feat: add IS→QUBO reduction example with JSON export - -Extracted from qubo_reductions.rs, added detailed docstring -and JSON export for paper integration" -``` - -**Step 4: Repeat for remaining 5 QUBO examples** - -Create in order: -- `reduction_vc_to_qubo.rs` from `demo_vertex_covering()` -- `reduction_coloring_to_qubo.rs` from `demo_coloring()` -- `reduction_setpacking_to_qubo.rs` from `demo_set_packing()` -- `reduction_ksatisfiability_to_qubo.rs` from `demo_ksat()` -- `reduction_ilp_to_qubo.rs` from `demo_ilp()` - -Commit each separately. - -**Step 5: Remove or rename original qubo_reductions.rs** - -```bash -# Option: rename as tutorial -git mv examples/qubo_reductions.rs examples/tutorial_qubo_reductions.rs -# Or option: remove if redundant -# git rm examples/qubo_reductions.rs - -git commit -m "refactor: rename qubo_reductions.rs to tutorial - -Separated into individual reduction examples, keeping original -as comprehensive tutorial" -``` - - -### Task 4.2: Create trivial reduction examples (IS↔VC, SetPacking, etc.) - -**Files:** -- Create: `examples/reduction_is_to_vc.rs` -- Create: `examples/reduction_vc_to_is.rs` -- Create: `examples/reduction_is_to_setpacking.rs` -- Create: `examples/reduction_matching_to_setpacking.rs` -- Create: `examples/reduction_vc_to_setcovering.rs` -- Create: `examples/reduction_spinglass_to_qubo.rs` (if not already created) -- Create: `examples/reduction_qubo_to_spinglass.rs` -- Create: `examples/reduction_spinglass_to_maxcut.rs` -- Create: `examples/reduction_maxcut_to_spinglass.rs` - -**Step 1: Create reduction_is_to_vc.rs** - -Use UnitDiskMapping.jl's 5-vertex demo graph or Petersen graph. Follow template with: -- Docstring explaining complement relationship -- Transformation metrics -- JSON export with graph structure and solutions - -```rust -//! # Independent Set to Vertex Cover Reduction -//! -//! ## Mathematical Equivalence -//! S ⊆ V is an independent set iff V \ S is a vertex cover. -//! Proof: If S is independent, no edge has both endpoints in S, -//! so every edge has at least one endpoint in V \ S. -//! -//! ## This Example -//! Demonstrates the complement relationship using the Petersen graph: -//! - Instance: Petersen graph (10 vertices, 15 edges) -//! - Maximum IS size: 4 -//! - Minimum VC size: 6 (complement property: 4 + 6 = 10) -//! - Reference: Based on UnitDiskMapping.jl Petersen example -//! -//! ## Output -//! Exports `docs/paper/examples/is_to_vc.json` for use in paper code blocks. -//! -//! See docs/paper/reductions.typ for the full reduction specification. - -use problemreductions::prelude::*; -use serde::Serialize; -use std::fs; - -#[derive(Serialize)] -struct ExampleData { - source_problem: String, - target_problem: String, - graph_vertices: usize, - graph_edges: Vec<(usize, usize)>, - source_size: usize, - target_size: usize, - source_solution: Vec<usize>, - target_solution: Vec<usize>, -} - -fn main() { - // Petersen graph (10 vertices, 15 edges) - let edges = vec![ - (0, 1), (1, 2), (2, 3), (3, 4), (4, 0), // outer pentagon - (5, 7), (7, 9), (9, 6), (6, 8), (8, 5), // inner star - (0, 5), (1, 6), (2, 7), (3, 8), (4, 9), // spokes - ]; - let is = IndependentSet::<i32>::new(10, edges.clone()); - - println!("\n=== Problem Transformation ==="); - println!("Source: {} with {} variables", "IndependentSet", is.num_variables()); - - // Reduce to VC - let reduction = ReduceTo::<VertexCovering<i32>>::reduce_to(&is); - let vc = reduction.target_problem(); - println!("Target: {} with {} variables", "VertexCovering", vc.num_variables()); - - // Solve - let solver = BruteForce::new(); - let vc_solutions = solver.find_best(vc); - println!("\n=== Solution ==="); - println!("Target solutions found: {}", vc_solutions.len()); - - // Extract - let is_solution = reduction.extract_solution(&vc_solutions[0]); - println!("Source solution: {:?}", is_solution); - - let size = is.solution_size(&is_solution); - println!("Solution size: {:?}", size); - assert!(size.is_valid); - println!("\n✓ Reduction verified successfully"); - - // Export JSON - let example_data = ExampleData { - source_problem: "IndependentSet".to_string(), - target_problem: "VertexCovering".to_string(), - graph_vertices: 10, - graph_edges: edges, - source_size: is.num_variables(), - target_size: vc.num_variables(), - source_solution: is_solution, - target_solution: vc_solutions[0].clone(), - }; - - fs::create_dir_all("docs/paper/examples").unwrap(); - let json = serde_json::to_string_pretty(&example_data).unwrap(); - fs::write("docs/paper/examples/is_to_vc.json", json).unwrap(); - println!(" Exported: docs/paper/examples/is_to_vc.json"); -} -``` - -**Step 2: Test and commit** - -```bash -cargo check --example reduction_is_to_vc -cargo run --example reduction_is_to_vc -git add examples/reduction_is_to_vc.rs -git commit -m "feat: add IS→VC reduction example" -``` - -**Step 3: Create remaining trivial examples** - -Follow same pattern for VC→IS, IS→SetPacking, etc. Commit each separately. - - -### Task 4.3: Create ILP reduction examples - -**Files:** -- Create 9 ILP reduction examples in `examples/` - -Create following the same pattern: -- `reduction_is_to_ilp.rs` -- `reduction_vc_to_ilp.rs` -- `reduction_matching_to_ilp.rs` -- `reduction_setpacking_to_ilp.rs` -- `reduction_setcovering_to_ilp.rs` -- `reduction_dominatingset_to_ilp.rs` -- `reduction_clique_to_ilp.rs` -- `reduction_coloring_to_ilp.rs` (already may exist) -- `reduction_factoring_to_ilp.rs` (already may exist) - -Each should: -- Use small instances (5-10 variables) -- Include `--features ilp` note in docstring -- Export ILP constraints to JSON for paper - -Commit each separately. - - -### Task 4.4: Create SAT and non-trivial reduction examples - -**Files:** -- Create remaining examples in `examples/` - -Create: -- `reduction_sat_to_is.rs` - use 3-4 variable SAT formula -- `reduction_sat_to_coloring.rs` - small SAT to coloring -- `reduction_sat_to_dominatingset.rs` - small SAT instance -- `reduction_sat_to_ksat.rs` - SAT to 3-SAT conversion -- `reduction_circuit_to_spinglass.rs` - small circuit -- `reduction_factoring_to_circuit.rs` - factor 6 = 2×3 - -Each should reference pkgref/ instances where applicable and export JSON. - -Commit each separately. - - -### Task 4.5: Verify all 28 examples compile and run - -**Step 1: Test all examples compile** - -```bash -cd /Users/liujinguo/rcode/problemreductions -cargo check --examples -``` - -Expected: All examples compile without errors - -**Step 2: Run all examples and verify JSON output** - -```bash -for example in examples/reduction_*.rs; do - name=$(basename "$example" .rs) - echo "Running $name..." - cargo run --example "$name" || echo "FAILED: $name" -done - -ls -lh docs/paper/examples/*.json -``` - -Expected: All examples run, JSON files created - -**Step 3: Commit verification script** - -Create `scripts/test_examples.sh`: - -```bash -#!/bin/bash -set -e -cd "$(dirname "$0")/.." -echo "Testing all reduction examples..." -for example in examples/reduction_*.rs; do - name=$(basename "$example" .rs) - cargo run --quiet --example "$name" -done -echo "✓ All examples passed" -``` - -```bash -chmod +x scripts/test_examples.sh -git add scripts/test_examples.sh -git commit -m "test: add example verification script" -``` - ---- - -## PASS 5: Final Verification - -### Task 5.1: Verify paper compiles - -**Step 1: Build the paper** - -```bash -cd /Users/liujinguo/rcode/problemreductions -make paper -``` - -Expected: Typst compiles without errors, generates PDF - -**Step 2: Check for broken references** - -```bash -grep -n "@thm:" docs/paper/reductions.typ | grep -v "^[0-9]*:#theorem" -``` - -Expected: All theorem references resolve (no broken links) - -**Step 3: Commit if fixes needed** - -If any issues found, fix and commit: - -```bash -git add docs/paper/reductions.typ -git commit -m "fix: resolve broken theorem references" -``` - - -### Task 5.2: Run full test suite - -**Step 1: Run Rust tests** - -```bash -make test -``` - -Expected: All tests pass - -**Step 2: Run clippy** - -```bash -make clippy -``` - -Expected: No warnings - -**Step 3: Run all examples** - -```bash -./scripts/test_examples.sh -``` - -Expected: All examples run successfully - - -### Task 5.3: Generate final checklist report - -**Step 1: Verify success criteria** - -Create `docs/paper/VERIFICATION.md`: - -```markdown -# Reductions.typ Polish - Verification Report - -Date: 2026-02-10 - -## Success Criteria - -- [x] 15 problem definitions have field mapping paragraphs -- [x] All problem definitions link to theorem labels (not problem definitions) -- [x] 28 theorems have labels and GitHub example links -- [x] Trivial reduction proofs explain variable mappings explicitly -- [x] 28 example files created with detailed docstrings -- [x] All examples use reference package instances where applicable -- [x] All examples export JSON to `docs/paper/examples/` -- [x] `docs/paper/examples/` added to `.gitignore` (already done) -- [x] Existing `qubo_reductions.rs` split into 6 separate files -- [x] `make paper` compiles successfully -- [x] All example files compile and run successfully - -## Statistics - -- Problem definitions enhanced: 15 -- Theorems labeled: 28 -- Example files created: 28 -- JSON exports: 28 -- Total commits: ~50-60 - -## Files Modified - -- `docs/paper/reductions.typ` - main paper file -- `examples/` - 28 new example files -- `examples/qubo_reductions.rs` - renamed to tutorial -- `.gitignore` - already updated - -## Next Steps - -- Review generated PDF for formatting -- Verify all GitHub links work after PR merge -- Consider adding CI check for example JSON generation -``` - -**Step 2: Commit verification report** - -```bash -git add docs/paper/VERIFICATION.md -git commit -m "docs: add verification report for reductions.typ polish" -``` - - -### Task 5.4: Final cleanup and summary commit - -**Step 1: Remove temporary files** - -```bash -rm -f docs/paper/.theorem_labels.txt -``` - -**Step 2: Create summary commit** - -```bash -git commit --allow-empty -m "feat: complete reductions.typ documentation polish - -Implemented 5-pass documentation enhancement: - -Pass 1: Added theorem labels (<thm:*>) to all 28 reduction theorems -Pass 2: Enhanced 15 problem definitions with field mappings and theorem links -Pass 3: Expanded trivial reduction proofs and added GitHub example links -Pass 4: Created 28 standalone example files with JSON export -Pass 5: Verified compilation and all examples run successfully - -All examples follow consistent format: -- Detailed docstrings with mathematical context -- Transformation metrics output -- JSON export to docs/paper/examples/ -- Based on reference package instances where applicable - -See docs/paper/VERIFICATION.md for complete checklist." -``` - - -### Task 5.5: Push and create PR (if in worktree) - -**Step 1: Push changes** - -```bash -git push origin HEAD -``` - -**Step 2: Create PR** - -```bash -gh pr create --title "Polish reductions.typ documentation" \ - --body "Implements design from docs/plans/2026-02-10-polish-reductions-paper.md - -## Changes -- Added theorem labels to all 28 reductions for cross-referencing -- Enhanced 15 problem definitions with field mappings -- Expanded trivial reduction proofs with variable mapping explanations -- Created 28 standalone example files with JSON export -- Split qubo_reductions.rs into 6 separate files -- All examples reference pkgref/ instances where applicable - -## Verification -- ✓ make paper compiles successfully -- ✓ All 28 examples compile and run -- ✓ JSON exports generated -- ✓ All tests pass -- ✓ No clippy warnings - -See docs/paper/VERIFICATION.md for complete checklist." -``` - -Expected: PR created and ready for review - ---- - -## Notes - -**Reference Package Usage:** -- pkgref/UnitDiskMapping.jl/examples/ - Petersen graph, 5-vertex demo -- pkgref/qubogen/tests/ - QUBO test cases with known matrices -- pkgref/ProblemReductions.jl/examples/ - Factoring example - -**JSON Export Format:** -Each example exports structured data including: -- Problem names and sizes -- Input instance (graph edges, formulas, matrices) -- Solutions (source and target) -- Verification results - -**Commit Strategy:** -- Small, focused commits (one enhancement per commit) -- ~50-60 commits total across 5 passes -- Checkpoint commits after each pass -- Final summary commit - -**Testing:** -- Verify each example individually as created -- Run full suite at end -- Check paper compiles after each pass - diff --git a/docs/plans/2026-02-10-polish-reductions-paper.md b/docs/plans/2026-02-10-polish-reductions-paper.md deleted file mode 100644 index cdf3341c9..000000000 --- a/docs/plans/2026-02-10-polish-reductions-paper.md +++ /dev/null @@ -1,342 +0,0 @@ -# Polish reductions.typ Documentation - -**Date:** 2026-02-10 -**Status:** Design validated, ready for implementation - -## Objectives - -1. Connect mathematical symbols to program fields in problem definitions -2. Link from problem definitions to reduction theorems (not other problems) -3. Explain trivial reductions explicitly with variable mappings -4. Create standalone example files for all 28 reductions with GitHub links - -## Design - -### 1. Problem Definition Enhancement - -For each of the 15 problem definitions in Section 2, add two modifications: - -**A) Field Mapping Paragraph** - -After the Rust struct, add a paragraph explaining the correspondence between mathematical notation and struct fields: - -```typst -#definition("Independent Set (IS)")[ - Given $G = (V, E)$ with vertex weights $w: V -> RR$, find $S subset.eq V$ maximizing... - - ```rust - pub struct IndependentSet<W = i32> { - graph: UnGraph<(), ()>, - weights: Vec<W>, - } - ``` - - Where `graph` represents $G = (V, E)$ with vertices indexed $0..n-1$, and `weights` stores vertex weights $w: V -> RR$ indexed by vertex ID. The solution is a subset $S subset.eq V$ represented as a `Vec<usize>` of vertex indices. - - _Implemented reductions:_ IS→SetPacking (@thm:is-to-setpacking), IS→QUBO (@thm:is-to-qubo), VC→IS (@thm:vc-to-is), SAT→IS (@thm:sat-to-is), SetPacking→IS (@thm:setpacking-to-is). -] <def:independent-set> -``` - -**B) Link to Theorems Instead of Problems** - -Replace current: -```typst -_Reduces to:_ Set Packing (@def:set-packing), QUBO (@def:qubo). -_Reduces from:_ Vertex Cover (@def:vertex-cover), SAT (@def:satisfiability). -``` - -With: -```typst -_Implemented reductions:_ IS→SetPacking (@thm:is-to-setpacking), IS→QUBO (@thm:is-to-qubo), VC→IS (@thm:vc-to-is), SAT→IS (@thm:sat-to-is). -``` - -### 2. Theorem Enhancement - -**A) Add Labels to All Theorems** - -Every reduction theorem gets a label for cross-referencing: - -```typst -#theorem[ - *(IS $arrow.l.r$ VC)* ... -] <thm:is-to-vc> - -#theorem[ - *(IS $arrow.r$ Set Packing)* ... -] <thm:is-to-setpacking> -``` - -Label format: `<thm:source-to-target>` using lowercase problem names with hyphens. - -**B) Expand Trivial Reduction Proofs** - -For trivial reductions (complement, isomorphism), add explicit variable mapping explanation: - -```typst -#theorem[ - *(IS $arrow.l.r$ VC)* $S subset.eq V$ is independent iff $V backslash S$ is a vertex cover, with $|"IS"| + |"VC"| = |V|$. -] <thm:is-to-vc> - -#proof[ - ($arrow.r.double$) If $S$ is independent, no edge has both endpoints in $S$, so every edge has at least one endpoint in $V backslash S$, making it a cover. - - ($arrow.l.double$) If $C$ is a cover, for any $u, v in V backslash C$, edge $(u,v)$ cannot exist (else uncovered), so $V backslash C$ is independent. - - _Variable mapping:_ Given IS instance $(G, w)$, create VC instance $(G, w)$ with identical graph and weights. Solution extraction: for VC solution $C$, return $S = V backslash C$. The complement operation preserves optimality since $|S| + |C| = |V|$ is constant. -] -``` - -**C) Add GitHub Links to Examples** - -After each proof or embedded example, add: - -```typst -See [reduction example](https://github.com/CodingThrust/problem-reductions/blob/main/examples/reduction_is_to_vc.rs). -``` - -### 3. Example Files Creation - -Create 28 example files in `examples/` directory with flat naming structure: - -#### Example File Template - -```rust -//! # [Source] to [Target] Reduction -//! -//! ## Mathematical Equivalence -//! [Explain the mathematical relationship and why it works - 2-4 sentences] -//! [Reference to the mathematical proof if helpful] -//! -//! ## This Example -//! [Describe the specific instance used - graph structure, problem size, expected results] -//! - Instance: [e.g., "5-vertex graph from edges [(0,1), (1,2), ...]" or "Petersen graph"] -//! - Source: [expected optimal value] -//! - Target: [expected optimal value] -//! - Reference: [cite pkgref source if applicable, e.g., "Based on qubogen test case"] -//! -//! ## Output -//! Exports `docs/paper/examples/[source]_to_[target].json` for use in paper code blocks. -//! -//! See docs/paper/reductions.typ for the full reduction specification. - -use problemreductions::prelude::*; -use serde::Serialize; -use std::fs; -use std::path::Path; - -#[derive(Serialize)] -struct ExampleData { - source_problem: String, - target_problem: String, - source_size: usize, - target_size: usize, - source_solution: Vec<usize>, - target_solution: Vec<usize>, - // Include problem-specific fields as needed -} - -fn main() { - // 1. Create source problem - // Use instances from reference packages where available: - // - ProblemReductions.jl examples (Petersen graph, demo graphs) - // - qubogen test cases (small graphs with known solutions) - // - UnitDiskMapping.jl examples - let source = SourceProblem::new(...); - - // 2. Reduce to target - let reduction = ReduceTo::<TargetProblem>::reduce_to(&source); - let target = reduction.target_problem(); - - // 3. Print problem transformation metrics - println!("\n=== Problem Transformation ==="); - println!("Source: {} with {} variables", - SourceProblem::NAME, source.num_variables()); - println!("Target: {} with {} variables", - TargetProblem::NAME, target.num_variables()); - - // 4. Solve target problem - let solver = BruteForce::new(); - let target_solutions = solver.find_best(target); - println!("\n=== Solution ==="); - println!("Target solutions found: {}", target_solutions.len()); - - // 5. Extract source solution - let source_solution = reduction.extract_solution(&target_solutions[0]); - println!("Source solution: {:?}", source_solution); - - // 6. Verify and print result - let size = source.solution_size(&source_solution); - println!("Solution size: {:?}", size); - assert!(size.is_valid); - println!("\n✓ Reduction verified successfully"); - - // 7. Export to JSON for paper - let example_data = ExampleData { - source_problem: SourceProblem::NAME.to_string(), - target_problem: TargetProblem::NAME.to_string(), - source_size: source.num_variables(), - target_size: target.num_variables(), - source_solution: source_solution.clone(), - target_solution: target_solutions[0].clone(), - // Add problem-specific fields - }; - - let json = serde_json::to_string_pretty(&example_data).unwrap(); - fs::create_dir_all("docs/paper/examples").unwrap(); - let path = Path::new("docs/paper/examples/[source]_to_[target].json"); - fs::write(path, json).unwrap(); - println!(" Exported: {}", path.display()); -} -``` - -#### File Manifest (28 files) - -**Note:** Unit Disk Mapping (IS → GridGraph IS) already has an example at `examples/export_petersen_mapping.rs`. We will link to it from the paper but not create a new example file. - -**Note:** Existing `examples/qubo_reductions.rs` will be split into 6 separate files following our naming convention. The original file can be deleted or kept as a tutorial (to be decided during implementation). - -**Trivial/Complement (6 files):** -1. `reduction_is_to_vc.rs` -2. `reduction_vc_to_is.rs` -3. `reduction_spinglass_to_qubo.rs` -4. `reduction_qubo_to_spinglass.rs` -5. `reduction_spinglass_to_maxcut.rs` -6. `reduction_maxcut_to_spinglass.rs` - -**Graph → Set (3 files):** -7. `reduction_is_to_setpacking.rs` -8. `reduction_matching_to_setpacking.rs` -9. `reduction_vc_to_setcovering.rs` - -**Penalty-method QUBO (6 files):** -10. `reduction_is_to_qubo.rs` -11. `reduction_vc_to_qubo.rs` -12. `reduction_coloring_to_qubo.rs` -13. `reduction_setpacking_to_qubo.rs` -14. `reduction_ksatisfiability_to_qubo.rs` -15. `reduction_ilp_to_qubo.rs` - -**ILP formulations (9 files):** -16. `reduction_coloring_to_ilp.rs` -17. `reduction_factoring_to_ilp.rs` -18. `reduction_is_to_ilp.rs` -19. `reduction_vc_to_ilp.rs` -20. `reduction_matching_to_ilp.rs` -21. `reduction_setpacking_to_ilp.rs` -22. `reduction_setcovering_to_ilp.rs` -23. `reduction_dominatingset_to_ilp.rs` -24. `reduction_clique_to_ilp.rs` - -**Non-trivial (6 files):** -25. `reduction_sat_to_is.rs` -26. `reduction_sat_to_coloring.rs` -27. `reduction_sat_to_dominatingset.rs` -28. `reduction_sat_to_ksat.rs` -29. `reduction_circuit_to_spinglass.rs` -30. `reduction_factoring_to_circuit.rs` - -### 4. Reference Package Integration - -Use instances from reference packages for cross-verification and consistency: - -**Available in `pkgref/` (cloned from GitHub):** - -1. **ProblemReductions.jl** (`pkgref/ProblemReductions.jl/examples/`) - - Petersen graph examples - - Factoring → Circuit → SpinGlass - - Educational narrative style - -2. **UnitDiskMapping.jl** (`pkgref/UnitDiskMapping.jl/examples/`) - - 5-vertex demo graph: edges `[(1,2), (2,4), (3,4), (1,3), (4,5), (1,5)]` - - Petersen graph mapping - - Comprehensive tutorial examples - -3. **qubogen** (`pkgref/qubogen/tests/`) - - Small test instances (5 nodes) with known QUBO matrices - - Graph coloring: 5 nodes, 3 colors - - Max-Cut, MVC, Set Packing test cases - - Max-2-SAT examples - -**Instance Selection Strategy:** - -- **Graph problems**: Use Petersen graph (10 vertices) or UnitDiskMapping's 5-vertex demo graph -- **QUBO reductions**: Cross-reference with qubogen test cases where applicable -- **SAT problems**: Small formulas (3-4 variables) with known solutions -- **Factoring**: Use 6 = 2×3 from ProblemReductions.jl example -- **Document reference source** in example docstring when using external instance - -### 5. Implementation Workflow - -**Pass 1: Add theorem labels** -- Scan all `#theorem[...]` blocks in Section 3 -- Add `<thm:source-to-target>` labels -- Build mapping: reduction → label - -**Pass 2: Enhance problem definitions** -- For each problem in Section 2: - - Add field mapping paragraph after struct - - Replace "Reduces to/from" with "Implemented reductions" + theorem refs - -**Pass 3: Enhance theorem proofs** -- Expand trivial reduction proofs with variable mapping -- Add GitHub links after all theorems - - For Unit Disk Mapping (IS → GridGraph IS): link to existing `examples/export_petersen_mapping.rs` - - For other reductions: link to new `examples/reduction_*.rs` files - -**Pass 4: Create example files** -- Split existing `examples/qubo_reductions.rs` into 6 separate files: - - `reduction_is_to_qubo.rs` - - `reduction_vc_to_qubo.rs` - - `reduction_coloring_to_qubo.rs` - - `reduction_setpacking_to_qubo.rs` - - `reduction_ksatisfiability_to_qubo.rs` - - `reduction_ilp_to_qubo.rs` -- Extract embedded examples from paper to standalone files -- For each new example: - - Check `pkgref/` for matching instances in reference packages - - Use reference instances where available for cross-verification - - Document source in docstring (e.g., "Based on qubogen test case") - - Add detailed output showing problem transformation metrics - - Export JSON to `docs/paper/examples/[source]_to_[target].json` -- Each file: detailed docstring + closed-loop verification + JSON export - -**Pass 5: Verification** -- All theorems labeled -- All problems link to theorems -- All theorems link to examples -- Run `make paper` - must compile without errors - -## Success Criteria - -- [ ] 15 problem definitions have field mapping paragraphs -- [ ] All problem definitions link to theorem labels (not problem definitions) -- [ ] 28 theorems have labels and GitHub example links -- [ ] Trivial reduction proofs explain variable mappings explicitly -- [ ] 28 example files created with detailed docstrings -- [ ] All examples use reference package instances where applicable -- [ ] All examples export JSON to `docs/paper/examples/` -- [ ] `docs/paper/examples/` added to `.gitignore` (generated files) -- [ ] Existing `qubo_reductions.rs` split into 6 separate files -- [ ] `make paper` compiles successfully -- [ ] All example files compile and run successfully - -## Dependencies - -- Repository: https://github.com/CodingThrust/problem-reductions -- Paper file: `docs/paper/reductions.typ` -- Examples directory: `examples/` (already exists) -- Reduction rules: 28 files in `src/rules/` - -## Notes - -- **Docstrings**: Explain math and example instance, NOT reduction algorithm (kept in paper) -- **Instance selection**: Prefer instances from reference packages (ProblemReductions.jl, UnitDiskMapping.jl, qubogen) for cross-verification -- **Output style**: Inspired by UnitDiskMapping.jl - show problem transformation metrics and verification details -- **JSON export**: Each example exports `docs/paper/examples/[source]_to_[target].json` containing: - - Problem names and sizes - - Source and target solutions - - Problem-specific data (graphs, matrices, formulas) - - Used for embedding code examples in the paper -- **Separation of concerns**: Examples demonstrate mechanics, paper provides mathematical specification -- **GitHub links**: Use path `/blob/main/examples/reduction_*.rs` -- **Reference packages**: Located in `pkgref/` (gitignored, cloned for development reference) diff --git a/docs/plans/2026-02-10-unified-json-schema.md b/docs/plans/2026-02-10-unified-json-schema.md deleted file mode 100644 index 3a22b1d4a..000000000 --- a/docs/plans/2026-02-10-unified-json-schema.md +++ /dev/null @@ -1,277 +0,0 @@ -# Unified JSON Schema for Reduction Examples - -**Date**: 2026-02-10 -**Status**: Draft -**Scope**: All 30 example JSON files in `docs/paper/examples/` - -## Problem - -Three inconsistent JSON schemas exist across 30 example files: - -1. **Flat schema** (e.g., `sat_to_ksat.json`): `source_num_variables`, `target_num_variables`, `source_solution`, `target_solution` at top level -2. **QUBO rich schema** (e.g., `is_to_qubo.json`): `source_instance`, `qubo`, `optimal_solutions` with domain-specific details -3. **Nested schema** (e.g., `matching_to_setpacking.json`): `source`, `target`, `solution` objects - -This forces Typst's `load-example()` to normalize across all three formats at load time. - -## Design Decisions (from brainstorming) - -1. **Two files per example**: `<name>.json` (reduction structure) and `<name>.result.json` (runtime solutions) -2. **Typed instance objects**: Each problem type has its own natural fields (`num_vertices`/`edges` for graphs, `num_vars`/`matrix` for QUBO, `clauses` for SAT) -3. **No field duplication**: No synthetic `num_variables` when the instance already has `num_vertices` or `num_vars` -4. **Raw configs only**: Solution arrays are `[0,1,0,1]`, no interpretation strings like `"V0=Red"` -5. **Polynomial overhead**: Matches `ReductionOverhead` from `src/rules/registry.rs` -6. **Variant dict**: Matches `reduction_graph.json` node format - -## Schema: `<name>.json` (Reduction File) - -```json -{ - "source": { - "problem": "IndependentSet", - "variant": { "graph": "SimpleGraph", "weight": "Unweighted" }, - "instance": { - "num_vertices": 4, - "num_edges": 3, - "edges": [[0,1], [1,2], [2,3]] - } - }, - "target": { - "problem": "QUBO", - "variant": { "graph": "SimpleGraph", "weight": "f64" }, - "instance": { - "num_vars": 4, - "matrix": [[-1.0, 5.0, 0.0, 0.0], ...] - } - }, - "overhead": [ - { "field": "num_vars", "polynomial": [{ "coefficient": 1.0, "variables": [["num_vertices", 1]] }] } - ] -} -``` - -### `source` / `target` object - -| Field | Type | Description | -|-------|------|-------------| -| `problem` | string | Problem name matching `Problem::NAME` (e.g., `"IndependentSet"`, `"QUBO"`, `"KSatisfiability<3>"`) | -| `variant` | object | Key-value pairs matching `reduction_graph.json` node variant (e.g., `{"graph": "SimpleGraph", "weight": "Unweighted"}`) | -| `instance` | object | Problem-specific fields (see Instance Schemas below) | - -### `overhead` array - -Directly mirrors `ReductionOverhead::output_size: Vec<(&str, Polynomial)>` from `src/rules/registry.rs`. - -Each element maps one output size field to a polynomial of input size variables: - -```json -{ - "field": "num_vars", - "polynomial": [ - { "coefficient": 1.0, "variables": [["num_vertices", 1]] } - ] -} -``` - -Each polynomial entry is a monomial (mirrors `Monomial` from `src/polynomial.rs`): - -| Field | Type | Description | -|-------|------|-------------| -| `coefficient` | float | Scalar multiplier | -| `variables` | array of `[name, exponent]` | Variable-exponent pairs (empty = constant) | - -The polynomial is the sum of all monomials: `Σ (coefficient × Π variable^exponent)`. - -**Examples matching actual code declarations:** - -| Code (`#[reduction(overhead = ...)]`) | JSON `overhead` | -|---------------------------------------|-----------------| -| `poly!(num_vertices)` | `[{"coefficient": 1.0, "variables": [["num_vertices", 1]]}]` | -| `poly!(7 * num_clauses)` | `[{"coefficient": 7.0, "variables": [["num_clauses", 1]]}]` | -| `poly!(num_bits_first^2)` | `[{"coefficient": 1.0, "variables": [["num_bits_first", 2]]}]` | -| `poly!(3 * num_vars)` | `[{"coefficient": 3.0, "variables": [["num_vars", 1]]}]` | -| `poly!(3)` (constant) | `[{"coefficient": 3.0, "variables": []}]` | - -**Multi-field overhead** (e.g., SAT → IS has both `num_vertices` and `num_edges`): - -```json -"overhead": [ - { "field": "num_vertices", "polynomial": [{ "coefficient": 7.0, "variables": [["num_clauses", 1]] }] }, - { "field": "num_edges", "polynomial": [{ "coefficient": 21.0, "variables": [["num_clauses", 1]] }] } -] -``` - -### Instance Schemas (by problem type) - -Each problem type uses its natural fields from `problem_size()`. No generic `num_variables` wrapper. - -**Graph problems** (IndependentSet, VertexCovering, DominatingSet, MaxCut, Clique, Matching): -```json -{ "num_vertices": 4, "num_edges": 3, "edges": [[0,1], [1,2], [2,3]] } -``` -Optional: `"weights": [1, 2, 3, 4]` for weighted variants. - -**KColoring**: -```json -{ "num_vertices": 3, "num_edges": 3, "num_colors": 3, "edges": [[0,1], [1,2], [0,2]] } -``` - -**SAT / Satisfiability**: -```json -{ "num_vars": 4, "num_clauses": 2, "clauses": [[1, -2, 3], [-1, 4]] } -``` - -**KSatisfiability<K>**: -```json -{ "num_vars": 8, "num_clauses": 6, "k": 3, "clauses": [[1, -2, 3], ...] } -``` - -**QUBO**: -```json -{ "num_vars": 4, "matrix": [[-1.0, 5.0, 0.0, 0.0], ...] } -``` - -**SpinGlass**: -```json -{ "num_spins": 4, "num_interactions": 3, "interactions": [[0,1,1.0], [1,2,-1.0], ...] } -``` - -**SetPacking / SetCovering**: -```json -{ "num_sets": 4, "num_elements": 3, "sets": [[0], [0,1], [1,2], [2]] } -``` -Optional: `"weights": [1, 1, 1, 1]` for weighted variants. - -**ILP**: -```json -{ "num_vars": 4, "num_constraints": 2, "objective": [1.0, 2.0, 3.0, 4.0], "constraints": [...] } -``` - -**CircuitSAT**: -```json -{ "num_gates": 5, "num_assignments": 8, "gates": [...] } -``` - -**Factoring**: -```json -{ "number": 15, "num_bits_first": 2, "num_bits_second": 2 } -``` - -## Schema: `<name>.result.json` (Results File) - -```json -{ - "solutions": [ - { - "source_config": [1, 0, 1, 0], - "target_config": [1, 0, 1, 0] - } - ] -} -``` - -| Field | Type | Description | -|-------|------|-------------| -| `solutions` | array | One or more optimal solution pairs | -| `solutions[].source_config` | array of int | Raw variable assignment for source problem | -| `solutions[].target_config` | array of int | Raw variable assignment for target problem | - -No interpretation fields (no `"coloring": ["V0=Red", ...]`, no `"selected_vertices": [1,3]`). Consumers derive meaning from config + instance. - -## Implementation Plan - -### Step 1: Create `ExampleData` Rust struct - -Add a shared serialization module (e.g., `examples/shared/schema.rs` or a helper in `src/`) that all examples import: - -```rust -#[derive(Serialize)] -struct ProblemSide { - problem: String, - variant: HashMap<String, String>, - instance: serde_json::Value, -} - -#[derive(Serialize)] -struct OverheadEntry { - field: String, - polynomial: Vec<MonomialJson>, -} - -#[derive(Serialize)] -struct MonomialJson { - coefficient: f64, - variables: Vec<(String, u8)>, -} - -#[derive(Serialize)] -struct ReductionData { - source: ProblemSide, - target: ProblemSide, - overhead: Vec<OverheadEntry>, -} - -#[derive(Serialize)] -struct SolutionPair { - source_config: Vec<usize>, - target_config: Vec<usize>, -} - -#[derive(Serialize)] -struct ResultData { - solutions: Vec<SolutionPair>, -} -``` - -Add helper methods to build `ProblemSide` from any `Problem` impl and `OverheadEntry` from `ReductionOverhead`. - -### Step 2: Update all 30 example files - -Replace ad-hoc serialization with the shared struct. Each example: -1. Creates source problem -2. Reduces to target -3. Solves target, extracts solutions -4. Builds `ReductionData` + `ResultData` -5. Writes `<name>.json` and `<name>.result.json` - -### Step 3: Update Typst `load-example()` - -Replace the 3-schema normalization in `reductions.typ` with direct field access. Since all JSON files now share the same schema, `load-example()` becomes trivial: - -```typst -#let load-example(name) = json("examples/" + name + ".json") -#let load-results(name) = json("examples/" + name + ".result.json") -``` - -Update all `reduction-example()` calls and the resource estimation table to use `data.source.instance.num_vertices` etc. - -### Step 4: Update integration test assertions - -If `tests/suites/examples.rs` checks JSON structure, update to match new schema. - -### Step 5: Verify - -```bash -make examples # All 30 examples regenerate both files -make paper # Typst compiles with new schema -make test # All tests pass -make clippy # No warnings -``` - -## File Impact - -| Files | Count | Action | -|-------|-------|--------| -| `examples/shared/schema.rs` (or similar) | 1 | New: shared serialization structs | -| `examples/reduction_*.rs` | 30 | Update: use shared schema | -| `docs/paper/examples/*.json` | 30 | Regenerated: unified schema | -| `docs/paper/examples/*.result.json` | 30 | New: split solution data | -| `docs/paper/reductions.typ` | 1 | Update: simplify `load-example()` | - -## Migration Notes - -- Old JSON files are fully replaced (not backwards-compatible) -- The `overhead` field is new — sourced from each reduction's `#[reduction(overhead = ...)]` macro -- Polynomial serialization is a 1:1 mapping from `Polynomial { terms: Vec<Monomial> }` in Rust to `[{coefficient, variables}]` in JSON -- Problem names use `Problem::NAME` exactly (e.g., `"KSatisfiability<3>"` not `"3-SAT"`) -- Variant dicts match `reduction_graph.json` nodes diff --git a/docs/plans/2026-02-12-solution-size-enum.md b/docs/plans/2026-02-12-solution-size-enum.md deleted file mode 100644 index e49d52081..000000000 --- a/docs/plans/2026-02-12-solution-size-enum.md +++ /dev/null @@ -1,711 +0,0 @@ -# SolutionSize Enum Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add explicit `SolutionSize<T>` enum for validity tracking in optimization problems, replacing magic MIN/MAX values. - -**Architecture:** Introduce `SolutionSize::Valid(T)` and `SolutionSize::Invalid` enum. Optimization problems return `SolutionSize<W>` as their Metric. The `OptimizationProblem` trait provides `is_better(&self, a, b) -> bool` for direction-aware comparison. Satisfaction problems keep `Metric = bool` unchanged. - -**Tech Stack:** Rust, serde (for serialization) - ---- - -## Task 1: Add SolutionSize enum to types.rs - -**Files:** -- Modify: `src/types.rs` -- Test: `src/unit_tests/types.rs` - -**Step 1: Write the failing test** - -Add to `src/unit_tests/types.rs`: - -```rust -#[test] -fn test_solution_size_valid() { - let size: SolutionSize<i32> = SolutionSize::Valid(42); - assert!(size.is_valid()); - assert_eq!(size.size(), Some(&42)); -} - -#[test] -fn test_solution_size_invalid() { - let size: SolutionSize<i32> = SolutionSize::Invalid; - assert!(!size.is_valid()); - assert_eq!(size.size(), None); -} - -#[test] -fn test_solution_size_unwrap() { - let valid: SolutionSize<i32> = SolutionSize::Valid(10); - assert_eq!(valid.unwrap(), 10); -} - -#[test] -#[should_panic(expected = "called unwrap on Invalid")] -fn test_solution_size_unwrap_panics() { - let invalid: SolutionSize<i32> = SolutionSize::Invalid; - invalid.unwrap(); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test test_solution_size --lib` -Expected: FAIL with "cannot find type `SolutionSize`" - -**Step 3: Write the implementation** - -Add to `src/types.rs`: - -```rust -/// Result of evaluating a constrained optimization problem. -/// -/// For optimization problems with constraints (like MaximumIndependentSet), -/// configurations may be infeasible. This enum explicitly represents validity. -/// -/// # Example -/// -/// ``` -/// use problemreductions::types::SolutionSize; -/// -/// let valid = SolutionSize::Valid(42); -/// assert!(valid.is_valid()); -/// assert_eq!(valid.size(), Some(&42)); -/// -/// let invalid: SolutionSize<i32> = SolutionSize::Invalid; -/// assert!(!invalid.is_valid()); -/// ``` -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] -pub enum SolutionSize<T> { - /// A valid (feasible) solution with the given objective value. - Valid(T), - /// An invalid (infeasible) solution that violates constraints. - Invalid, -} - -impl<T> SolutionSize<T> { - /// Returns true if this is a valid solution. - pub fn is_valid(&self) -> bool { - matches!(self, SolutionSize::Valid(_)) - } - - /// Returns the size if valid, None if invalid. - pub fn size(&self) -> Option<&T> { - match self { - SolutionSize::Valid(t) => Some(t), - SolutionSize::Invalid => None, - } - } - - /// Unwraps the size, panicking if invalid. - pub fn unwrap(self) -> T { - match self { - SolutionSize::Valid(t) => t, - SolutionSize::Invalid => panic!("called unwrap on Invalid SolutionSize"), - } - } - - /// Maps the inner value if valid. - pub fn map<U, F: FnOnce(T) -> U>(self, f: F) -> SolutionSize<U> { - match self { - SolutionSize::Valid(t) => SolutionSize::Valid(f(t)), - SolutionSize::Invalid => SolutionSize::Invalid, - } - } -} - -impl<T: Default> Default for SolutionSize<T> { - fn default() -> Self { - SolutionSize::Invalid - } -} -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test test_solution_size --lib` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/types.rs src/unit_tests/types.rs -git commit -m "feat: add SolutionSize enum for explicit validity tracking" -``` - ---- - -## Task 2: Add is_better method to OptimizationProblem trait - -**Files:** -- Modify: `src/traits.rs` -- Modify: `src/types.rs` (export SolutionSize) -- Test: `src/unit_tests/traits.rs` - -**Step 1: Write the failing test** - -Add to `src/unit_tests/traits.rs`: - -```rust -use crate::types::{Direction, SolutionSize}; - -#[test] -fn test_is_better_maximize_valid_vs_valid() { - // For maximization: larger is better - let a = SolutionSize::Valid(10); - let b = SolutionSize::Valid(5); - assert!(is_better(&a, &b, Direction::Maximize)); - assert!(!is_better(&b, &a, Direction::Maximize)); -} - -#[test] -fn test_is_better_minimize_valid_vs_valid() { - // For minimization: smaller is better - let a = SolutionSize::Valid(5); - let b = SolutionSize::Valid(10); - assert!(is_better(&a, &b, Direction::Minimize)); - assert!(!is_better(&b, &a, Direction::Minimize)); -} - -#[test] -fn test_is_better_valid_vs_invalid() { - // Valid is always better than invalid - let valid = SolutionSize::Valid(0); - let invalid: SolutionSize<i32> = SolutionSize::Invalid; - assert!(is_better(&valid, &invalid, Direction::Maximize)); - assert!(is_better(&valid, &invalid, Direction::Minimize)); - assert!(!is_better(&invalid, &valid, Direction::Maximize)); - assert!(!is_better(&invalid, &valid, Direction::Minimize)); -} - -#[test] -fn test_is_better_invalid_vs_invalid() { - // Neither invalid is better - let a: SolutionSize<i32> = SolutionSize::Invalid; - let b: SolutionSize<i32> = SolutionSize::Invalid; - assert!(!is_better(&a, &b, Direction::Maximize)); - assert!(!is_better(&a, &b, Direction::Minimize)); -} - -#[test] -fn test_is_better_equal_valid() { - // Equal values: neither is better - let a = SolutionSize::Valid(5); - let b = SolutionSize::Valid(5); - assert!(!is_better(&a, &b, Direction::Maximize)); - assert!(!is_better(&a, &b, Direction::Minimize)); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test test_is_better --lib` -Expected: FAIL with "cannot find function `is_better`" - -**Step 3: Write the implementation** - -Add to `src/types.rs`: - -```rust -impl<T: Ord> SolutionSize<T> { - /// Returns true if self is a better solution than other for the given direction. - /// - /// - For maximization: larger values are better - /// - For minimization: smaller values are better - /// - Valid solutions are always better than invalid ones - /// - Two invalid solutions are equally bad (neither is better) - pub fn is_better(&self, other: &Self, direction: Direction) -> bool { - match (self, other) { - (SolutionSize::Valid(a), SolutionSize::Valid(b)) => match direction { - Direction::Maximize => a > b, - Direction::Minimize => a < b, - }, - (SolutionSize::Valid(_), SolutionSize::Invalid) => true, - (SolutionSize::Invalid, SolutionSize::Valid(_)) => false, - (SolutionSize::Invalid, SolutionSize::Invalid) => false, - } - } -} -``` - -Update test to use method: - -```rust -fn is_better<T: Ord>(a: &SolutionSize<T>, b: &SolutionSize<T>, dir: Direction) -> bool { - a.is_better(b, dir) -} -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test test_is_better --lib` -Expected: PASS - -**Step 5: Update exports** - -In `src/lib.rs`, add `SolutionSize` to prelude and re-exports: - -```rust -pub use types::{Direction, NumericSize, NumericSizeBounds, ProblemSize, SolutionSize, Unweighted, Weights}; -``` - -In `src/prelude.rs` section of `src/lib.rs`: - -```rust -pub use crate::types::{Direction, NumericSize, NumericSizeBounds, NumericWeight, ProblemSize, SolutionSize, Unweighted, Weights}; -``` - -**Step 6: Commit** - -```bash -git add src/types.rs src/traits.rs src/lib.rs src/unit_tests/traits.rs -git commit -m "feat: add is_better method to SolutionSize for direction-aware comparison" -``` - ---- - -## Task 3: Update Solver trait and BruteForce implementation - -**Files:** -- Modify: `src/solvers/mod.rs` -- Modify: `src/solvers/brute_force.rs` -- Test: `src/unit_tests/solvers/brute_force.rs` - -**Step 1: Update Solver trait** - -In `src/solvers/mod.rs`, change `find_best` signature: - -```rust -use crate::traits::{OptimizationProblem, Problem}; -use crate::types::SolutionSize; - -/// Trait for problem solvers. -pub trait Solver { - /// Find best solution(s) for an optimization problem. - /// - /// Returns all configurations that achieve the optimal metric value. - /// Returns empty vec if all configurations are invalid. - fn find_best<P>(&self, problem: &P) -> Vec<Vec<usize>> - where - P: OptimizationProblem, - P::Metric: Clone; - - /// Find any satisfying solution for a satisfaction problem (Metric = bool). - fn find_satisfying<P: Problem<Metric = bool>>(&self, problem: &P) -> Option<Vec<usize>>; -} -``` - -Note: Remove `find_all_satisfying` from the trait (internal only). - -**Step 2: Update BruteForce implementation** - -In `src/solvers/brute_force.rs`: - -```rust -use crate::config::DimsIterator; -use crate::solvers::Solver; -use crate::traits::{OptimizationProblem, Problem}; -use crate::types::SolutionSize; - -impl Solver for BruteForce { - fn find_best<P>(&self, problem: &P) -> Vec<Vec<usize>> - where - P: OptimizationProblem, - P::Metric: Clone, - { - self.find_all_best(problem) - } - - fn find_satisfying<P: Problem<Metric = bool>>(&self, problem: &P) -> Option<Vec<usize>> { - let dims = problem.dims(); - if dims.is_empty() { - return None; - } - DimsIterator::new(dims).find(|config| problem.evaluate(config)) - } -} - -impl BruteForce { - /// Internal: find all optimal solutions. - fn find_all_best<P>(&self, problem: &P) -> Vec<Vec<usize>> - where - P: OptimizationProblem, - P::Metric: Clone, - { - let dims = problem.dims(); - if dims.is_empty() { - return vec![]; - } - - let iter = DimsIterator::new(dims); - let mut best_solutions: Vec<Vec<usize>> = vec![]; - let mut best_metric: Option<P::Metric> = None; - - for config in iter { - let metric = problem.evaluate(&config); - - let dominated = match &best_metric { - None => false, - Some(current_best) => problem.is_better(current_best, &metric), - }; - - if dominated { - continue; - } - - let dominates = match &best_metric { - None => true, - Some(current_best) => problem.is_better(&metric, current_best), - }; - - if dominates { - best_metric = Some(metric); - best_solutions.clear(); - best_solutions.push(config); - } else if best_metric.is_some() { - // Equal quality - add to solutions - best_solutions.push(config); - } - } - - best_solutions - } - - /// Find all satisfying solutions (internal, used for testing). - pub(crate) fn find_all_satisfying<P: Problem<Metric = bool>>( - &self, - problem: &P, - ) -> Vec<Vec<usize>> { - let dims = problem.dims(); - if dims.is_empty() { - return vec![]; - } - DimsIterator::new(dims) - .filter(|config| problem.evaluate(config)) - .collect() - } -} -``` - -**Step 3: Run tests** - -Run: `cargo test --lib` -Expected: Many failures (models still use old Metric type) - -**Step 4: Commit intermediate progress** - -```bash -git add src/solvers/mod.rs src/solvers/brute_force.rs -git commit -m "refactor: update Solver trait for SolutionSize-based metrics" -``` - ---- - -## Task 4: Add is_better to OptimizationProblem trait - -**Files:** -- Modify: `src/traits.rs` - -**Step 1: Update OptimizationProblem trait** - -```rust -/// Extension for problems with a numeric objective to optimize. -pub trait OptimizationProblem: Problem { - /// Whether to maximize or minimize the metric. - fn direction(&self) -> crate::types::Direction; - - /// Returns true if metric `a` is better than metric `b` for this problem. - fn is_better(&self, a: &Self::Metric, b: &Self::Metric) -> bool; -} -``` - -**Step 2: Commit** - -```bash -git add src/traits.rs -git commit -m "feat: add is_better method to OptimizationProblem trait" -``` - ---- - -## Task 5: Update MaximumIndependentSet model - -**Files:** -- Modify: `src/models/graph/maximum_independent_set.rs` -- Test: `src/unit_tests/models/graph/maximum_independent_set.rs` - -**Step 1: Update imports and Problem impl** - -```rust -use crate::types::{Direction, SolutionSize}; - -impl<G, W> Problem for MaximumIndependentSet<G, W> -where - G: Graph, - W: Clone + Default + PartialOrd + Ord + num_traits::Num + num_traits::Zero + std::ops::AddAssign + 'static, -{ - const NAME: &'static str = "MaximumIndependentSet"; - type Metric = SolutionSize<W>; - - fn variant() -> Vec<(&'static str, &'static str)> { - vec![ - ("graph", crate::variant::short_type_name::<G>()), - ("weight", crate::variant::short_type_name::<W>()), - ] - } - - fn dims(&self) -> Vec<usize> { - vec![2; self.graph.num_vertices()] - } - - fn evaluate(&self, config: &[usize]) -> SolutionSize<W> { - if !is_independent_set_config(&self.graph, config) { - return SolutionSize::Invalid; - } - let mut total = W::zero(); - for (i, &selected) in config.iter().enumerate() { - if selected == 1 { - total += self.weights[i].clone(); - } - } - SolutionSize::Valid(total) - } -} - -impl<G, W> OptimizationProblem for MaximumIndependentSet<G, W> -where - G: Graph, - W: Clone + Default + PartialOrd + Ord + num_traits::Num + num_traits::Zero + std::ops::AddAssign + 'static, -{ - fn direction(&self) -> Direction { - Direction::Maximize - } - - fn is_better(&self, a: &Self::Metric, b: &Self::Metric) -> bool { - a.is_better(b, self.direction()) - } -} -``` - -**Step 2: Update unit tests** - -In `src/unit_tests/models/graph/maximum_independent_set.rs`, update tests to use `SolutionSize`: - -```rust -use crate::types::SolutionSize; - -#[test] -fn test_evaluate_valid() { - let problem = MaximumIndependentSet::<SimpleGraph, i32>::new(3, vec![(0, 1)]); - // Select vertex 2 only (not adjacent to anything selected) - let config = vec![0, 0, 1]; - assert_eq!(problem.evaluate(&config), SolutionSize::Valid(1)); -} - -#[test] -fn test_evaluate_invalid() { - let problem = MaximumIndependentSet::<SimpleGraph, i32>::new(3, vec![(0, 1)]); - // Select both 0 and 1 (adjacent - invalid) - let config = vec![1, 1, 0]; - assert_eq!(problem.evaluate(&config), SolutionSize::Invalid); -} -``` - -**Step 3: Run tests** - -Run: `cargo test maximum_independent_set --lib` -Expected: PASS - -**Step 4: Commit** - -```bash -git add src/models/graph/maximum_independent_set.rs src/unit_tests/models/graph/maximum_independent_set.rs -git commit -m "refactor: update MaximumIndependentSet to use SolutionSize" -``` - ---- - -## Task 6: Update MinimumVertexCover model - -**Files:** -- Modify: `src/models/graph/minimum_vertex_cover.rs` -- Test: `src/unit_tests/models/graph/minimum_vertex_cover.rs` - -**Step 1: Update Problem and OptimizationProblem impl** - -Same pattern as Task 5, but: -- `evaluate` returns `SolutionSize::Invalid` when not a valid cover -- `direction()` returns `Direction::Minimize` - -```rust -fn evaluate(&self, config: &[usize]) -> SolutionSize<W> { - if !is_vertex_cover_config(&self.graph, config) { - return SolutionSize::Invalid; - } - let mut total = W::zero(); - for (i, &selected) in config.iter().enumerate() { - if selected == 1 { - total += self.weights[i].clone(); - } - } - SolutionSize::Valid(total) -} -``` - -**Step 2: Update tests, run, commit** - -```bash -git add src/models/graph/minimum_vertex_cover.rs src/unit_tests/models/graph/minimum_vertex_cover.rs -git commit -m "refactor: update MinimumVertexCover to use SolutionSize" -``` - ---- - -## Task 7: Update remaining graph models - -**Files:** -- Modify: `src/models/graph/max_cut.rs` -- Modify: `src/models/graph/minimum_dominating_set.rs` -- Modify: `src/models/graph/maximal_is.rs` -- Modify: `src/models/graph/maximum_matching.rs` -- Modify: `src/models/graph/maximum_clique.rs` - -For each model: -1. Change `type Metric = W` to `type Metric = SolutionSize<W>` -2. Update `evaluate` to return `SolutionSize::Valid(value)` or `SolutionSize::Invalid` -3. Add `is_better` method to `OptimizationProblem` impl -4. Update corresponding unit tests - -**Note:** MaxCut may have no invalid configurations (all cuts are valid), so it always returns `SolutionSize::Valid(cut_value)`. - -**Commit after each model:** - -```bash -git commit -m "refactor: update <ModelName> to use SolutionSize" -``` - ---- - -## Task 8: Update set models - -**Files:** -- Modify: `src/models/set/maximum_set_packing.rs` -- Modify: `src/models/set/minimum_set_covering.rs` - -Same pattern as graph models. - ---- - -## Task 9: Update optimization models - -**Files:** -- Modify: `src/models/optimization/spin_glass.rs` -- Modify: `src/models/optimization/qubo.rs` -- Modify: `src/models/optimization/ilp.rs` - -**Note:** SpinGlass and QUBO are unconstrained - they always return `SolutionSize::Valid(energy)`. ILP has constraints. - ---- - -## Task 10: Update specialized models - -**Files:** -- Modify: `src/models/specialized/paintshop.rs` -- Modify: `src/models/specialized/bmf.rs` -- Modify: `src/models/specialized/biclique_cover.rs` - -Skip satisfaction models (Factoring, CircuitSAT) - they keep `Metric = bool`. - ---- - -## Task 11: Update reduction rules - -**Files:** -- All files in `src/rules/` that use `evaluate()` or compare metrics - -Key changes: -- Update `extract_solution` methods if they check validity -- Update any code that compares metric values directly - ---- - -## Task 12: Update examples - -**Files:** -- All files in `examples/` - -Update patterns: -- Change `problem.evaluate(&config) > i32::MIN` to `problem.evaluate(&config).is_valid()` -- Change `metric.is_min_bound()` to `!metric.is_valid()` -- Use `metric.unwrap()` or `metric.size()` to get the value - ---- - -## Task 13: Update integration tests - -**Files:** -- `tests/suites/integration.rs` -- `tests/suites/reductions.rs` - -Same patterns as examples. - ---- - -## Task 14: Update benchmarks - -**Files:** -- `benches/solver_benchmarks.rs` - -Ensure benchmarks compile with new API. - ---- - -## Task 15: Final verification - -**Step 1: Run all tests** - -```bash -make test -``` -Expected: All pass - -**Step 2: Run clippy** - -```bash -make clippy -``` -Expected: No warnings - -**Step 3: Run examples** - -```bash -cargo run --example reduction_maximumindependentset_to_qubo -``` -Expected: Success, JSON output unchanged - -**Step 4: Final commit** - -```bash -git add -A -git commit -m "feat: complete SolutionSize migration for explicit validity tracking" -``` - ---- - -## Summary - -| Task | Description | Files | -|------|-------------|-------| -| 1 | Add SolutionSize enum | types.rs | -| 2 | Add is_better method | types.rs, traits.rs | -| 3 | Update Solver trait | solvers/*.rs | -| 4 | Update OptimizationProblem | traits.rs | -| 5-6 | Update MIS, MVC | graph/*.rs | -| 7 | Update other graph models | graph/*.rs | -| 8 | Update set models | set/*.rs | -| 9 | Update optimization models | optimization/*.rs | -| 10 | Update specialized models | specialized/*.rs | -| 11 | Update reduction rules | rules/*.rs | -| 12-14 | Update examples, tests, benchmarks | examples/, tests/, benches/ | -| 15 | Final verification | - | diff --git a/docs/plans/2026-02-12-trait-refactoring-design.md b/docs/plans/2026-02-12-trait-refactoring-design.md deleted file mode 100644 index 2d51fdb21..000000000 --- a/docs/plans/2026-02-12-trait-refactoring-design.md +++ /dev/null @@ -1,303 +0,0 @@ -# Trait System Refactoring Design - -**Goal:** Simplify types and interfaces to lower the barrier for contributors. - -**Approach:** Trait system redesign (Approach B) — addresses root causes of complexity without hiding them behind macros. - -## 1. `NumericSize` Bound - -Replace the repeated `where W: Clone + Default + PartialOrd + Num + Zero + AddAssign + 'static` with a single supertrait. Eliminates 15+ copy-pasted bound lists. - -```rust -pub trait NumericSize: - Clone + Default + PartialOrd + num_traits::Num + num_traits::Zero - + std::ops::AddAssign + 'static -{} - -// Blanket impl: any type meeting the bounds is automatically NumericSize. -impl<T> NumericSize for T -where - T: Clone + Default + PartialOrd + num_traits::Num + num_traits::Zero - + std::ops::AddAssign + 'static, -{} -``` - -Problems needing extra bounds add them locally: `W: Weights` where `W::Size: NumericSize + Mul<Output = W::Size>`. - -## 2. `Weights` Trait - -Replaces the current weight type parameter `W`. Separates two concepts that were conflated: -- **Weight storage** — how weights are stored (`Unweighted`, `Vec<i32>`, `Vec<f64>`) -- **Objective value type** — what type the metric is (`i32`, `f64`) - -```rust -pub trait Weights: Clone + 'static { - const NAME: &'static str; - type Size: NumericSize; - fn weight(&self, index: usize) -> Self::Size; - fn len(&self) -> usize; -} -``` - -### Implementations - -**`Unweighted`** — zero-data storage, every element has unit weight: - -```rust -#[derive(Clone, Debug, Serialize, Deserialize)] -pub struct Unweighted(pub usize); // stores only the count - -impl Weights for Unweighted { - const NAME: &'static str = "Unweighted"; - type Size = i32; - fn weight(&self, _index: usize) -> i32 { 1 } - fn len(&self) -> usize { self.0 } -} -``` - -**`Vec<i32>` and `Vec<f64>`** — concrete weighted storage: - -```rust -impl Weights for Vec<i32> { - const NAME: &'static str = "Weighted<i32>"; - type Size = i32; - fn weight(&self, index: usize) -> i32 { self[index] } - fn len(&self) -> usize { self.len() } -} - -impl Weights for Vec<f64> { - const NAME: &'static str = "Weighted<f64>"; - type Size = f64; - fn weight(&self, index: usize) -> f64 { self[index] } - fn len(&self) -> usize { self.len() } -} -``` - -### Type-level distinction - -The type reflects whether a problem is weighted: -- `MaximumIndependentSet<SimpleGraph, Unweighted>` — unweighted -- `MaximumIndependentSet<SimpleGraph, Vec<i32>>` — weighted with integers -- `MaximumIndependentSet<SimpleGraph, Vec<f64>>` — weighted with floats - -Constructors make this ergonomic: -```rust -let mis = MaximumIndependentSet::new(graph); // -> MIS<_, Unweighted> -let mis = MaximumIndependentSet::with_weights(graph, vec![3, 1, 4]); // -> MIS<_, Vec<i32>> -``` - -## 3. `Problem` Trait (Minimal Function-like Object) - -A problem is a function from configuration to metric. Two methods + one constant: - -```rust -pub trait Problem: Clone { - const NAME: &'static str; - type Metric: Clone; - - /// Configuration space dimensions. Each entry is the cardinality - /// of that variable (e.g., [2, 2, 2] = 3 binary variables). - fn dims(&self) -> Vec<usize>; - - /// Evaluate the problem on a configuration. - fn evaluate(&self, config: &[usize]) -> Self::Metric; -} -``` - -`num_variables()` is derived: `self.dims().len()`. - -### `OptimizationProblem` extension - -Optimization problems add a direction (maximize or minimize): - -```rust -pub trait OptimizationProblem: Problem -where - Self::Metric: NumericSize, -{ - fn direction(&self) -> Direction; -} - -pub enum Direction { - Maximize, - Minimize, -} -``` - -### How problems implement this - -**Satisfaction problems** — metric is `bool`: -```rust -impl<W: Weights> Problem for Satisfiability<W> { - type Metric = bool; - fn evaluate(&self, config: &[usize]) -> bool { - self.clauses.iter().all(|c| c.is_satisfied(config)) - } -} -``` - -**Optimization problems** — metric is numeric, invalid configs return worst value: -```rust -impl<G: Graph, W: Weights> Problem for MaximumIndependentSet<G, W> { - type Metric = W::Size; - fn evaluate(&self, config: &[usize]) -> W::Size { - if !self.is_independent(config) { - return f64::NEG_INFINITY; // not favored by maximize - } - self.total_weight(config) - } -} -``` - -**All-valid problems** — every config is feasible: -```rust -impl<W: Weights> Problem for QUBO<W> { - type Metric = W::Size; - fn evaluate(&self, config: &[usize]) -> W::Size { - self.compute_energy(config) - } -} -``` - -### Problem categorization - -| Problem | `Metric` | `OptimizationProblem` | Invalid handling | -|---------|----------|----------------------|-----------------| -| SAT | `bool` | No | N/A (all configs valid) | -| KColoring | `bool` | No | N/A (all configs valid) | -| MIS | `W::Size` | Yes (Maximize) | `-inf` | -| VertexCover | `W::Size` | Yes (Minimize) | `+inf` | -| QUBO | `W::Size` | Yes (Minimize) | N/A (all configs valid) | -| SpinGlass | `W::Size` | Yes (Minimize) | N/A (all configs valid) | -| MaxCut | `W::Size` | Yes (Maximize) | N/A (all configs valid) | -| MAX-SAT | `W::Size` | Yes (Maximize) | N/A (all configs valid) | - -## 4. Standardized Type Parameters - -| Category | Pattern | Example | -|----------|---------|---------| -| Graph + weighted | `<G: Graph, W: Weights>` | `MaximumIndependentSet<G, W>` | -| Non-graph + weighted | `<W: Weights>` | `QUBO<W>`, `Satisfiability<W>` | -| Decision (no weight) | `<G: Graph>` | `KColoring<G>` (k is runtime field) | - -KColoring's const generic `K` becomes a runtime field `k: usize`. - -## 5. `ReductionResult` and `ReduceTo` (Simplified) - -```rust -pub trait ReductionResult: Clone { - type Source: Problem; - type Target: Problem; - - /// The reduced problem instance. - fn target_problem(&self) -> &Self::Target; - - /// Map a target solution back to a source solution. - fn extract_solution(&self, target_config: &[usize]) -> Vec<usize>; -} - -pub trait ReduceTo<T: Problem>: Problem { - type Result: ReductionResult<Source = Self, Target = T>; - fn reduce_to(&self) -> Self::Result; -} -``` - -Removed `source_size()` and `target_size()`. Overhead is tracked in the `#[reduction]` macro attribute. Instance sizes available via `dims()`. - -## 6. `#[reduction]` Macro (Trait-bound Extraction) - -The macro identifies graph/weight types by inspecting **trait bounds**, not parameter positions: -- `G: Graph` bound → graph type, uses `Graph::NAME` -- `W: Weights` bound → weights type, uses `Weights::NAME` -- Source/target names: extracted from type signature (`ReduceTo<Target> for Source`) - -No heuristics, no hardcoded type name lists, no silent fallbacks. - -```rust -#[reduction(overhead = { ... })] -impl<G: Graph, W: Weights> ReduceTo<MinimumVertexCover<G, W>> - for MaximumIndependentSet<G, W> -{ - type Result = ReductionMISToVC<G, W>; - fn reduce_to(&self) -> Self::Result { ... } -} -``` - -Only the `overhead` attribute is required. Everything else is derived from types. - -Variant IDs are constructed in the registry from `Graph::NAME` and `Weights::NAME`: -``` -"MaximumIndependentSet" // SimpleGraph + Unweighted (defaults) -"MaximumIndependentSet/GridGraph" // non-default graph -"MaximumIndependentSet/Weighted" // non-default weight -"MaximumIndependentSet/GridGraph/Weighted" // both non-default -``` - -## 7. What's Removed - -| Removed | Replaced by | -|---------|------------| -| `Unweighted` marker struct | `Unweighted(usize)` real weight vector | -| `EnergyMode` enum | `Direction` on `OptimizationProblem` | -| `SolutionSize<T>` struct | `evaluate()` return value directly | -| `ConstraintSatisfactionProblem` trait | Removed entirely | -| `variant()` method | Derived from `Graph::NAME` + `Weights::NAME` | -| `solution_size()` | `evaluate()` | -| `is_valid()` | Folded into `evaluate()` (returns -inf/+inf) | -| `num_flavors()` | `dims()` (per-variable) | -| `num_variables()` | `dims().len()` | -| `problem_size()` on core trait | Removed | -| `set_weights()` / `is_weighted()` | Removed | -| `source_size()` / `target_size()` on `ReductionResult` | Removed, use `dims()` | -| Hardcoded weight type list in macro | Trait-bound inspection | -| Position-based type param inference in macro | Trait-bound inspection | - -## 8. Contributor Experience After Refactoring - -### Adding a new problem (2 methods + 1 constant) - -```rust -pub struct MyProblem<G: Graph, W: Weights> { - graph: G, - weights: W, -} - -impl<G: Graph, W: Weights> Problem for MyProblem<G, W> { - const NAME: &'static str = "MyProblem"; - type Metric = W::Size; - - fn dims(&self) -> Vec<usize> { - vec![2; self.graph.num_vertices()] - } - - fn evaluate(&self, config: &[usize]) -> W::Size { - // compute objective, return -inf for invalid if maximizing - } -} - -impl<G: Graph, W: Weights> OptimizationProblem for MyProblem<G, W> { - fn direction(&self) -> Direction { Direction::Maximize } -} -``` - -### Adding a new reduction (2 methods) - -```rust -#[derive(Clone)] -pub struct ReductionAToB<W: Weights> { - target: ProblemB<W>, -} - -impl<W: Weights> ReductionResult for ReductionAToB<W> { - type Source = ProblemA<W>; - type Target = ProblemB<W>; - fn target_problem(&self) -> &Self::Target { &self.target } - fn extract_solution(&self, target_config: &[usize]) -> Vec<usize> { /* ... */ } -} - -#[reduction(overhead = { ... })] -impl<W: Weights> ReduceTo<ProblemB<W>> for ProblemA<W> { - type Result = ReductionAToB<W>; - fn reduce_to(&self) -> Self::Result { /* ... */ } -} -``` diff --git a/docs/plans/2026-02-12-trait-refactoring-impl.md b/docs/plans/2026-02-12-trait-refactoring-impl.md deleted file mode 100644 index d456e2cd4..000000000 --- a/docs/plans/2026-02-12-trait-refactoring-impl.md +++ /dev/null @@ -1,793 +0,0 @@ -# Trait System Refactoring Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Refactor the core type system to simplify Problem/Reduction traits, making it easier for contributors to add problems and reductions. - -**Architecture:** Replace the current 8-method `Problem` trait + `ConstraintSatisfactionProblem` with a minimal 2-method `Problem` trait (`dims` + `evaluate`) plus an `OptimizationProblem` extension. Introduce a `Weights` trait to separate weight storage from objective types. Simplify the proc macro to use trait-bound inspection instead of heuristics. - -**Tech Stack:** Rust, proc-macro2/syn/quote (proc macro crate), inventory (static registration), serde, num-traits - -**Design doc:** `docs/plans/2026-02-12-trait-refactoring-design.md` - ---- - -## Task 1: Add `NumericSize` trait and `Weights` trait to `src/types.rs` - -**Files:** -- Modify: `src/types.rs` - -**Step 1: Write failing test** - -Add to `src/unit_tests/types.rs`: - -```rust -#[test] -fn test_numeric_size_blanket_impl() { - fn assert_numeric_size<T: NumericSize>() {} - assert_numeric_size::<i32>(); - assert_numeric_size::<i64>(); - assert_numeric_size::<f64>(); -} - -#[test] -fn test_unweighted_weights_trait() { - let w = Unweighted(5); - assert_eq!(w.len(), 5); - assert_eq!(w.weight(0), 1); - assert_eq!(w.weight(4), 1); - assert_eq!(Unweighted::NAME, "Unweighted"); -} - -#[test] -fn test_vec_i32_weights_trait() { - let w = vec![3, 1, 4]; - assert_eq!(w.len(), 3); - assert_eq!(w.weight(0), 3); - assert_eq!(w.weight(2), 4); - assert_eq!(<Vec<i32> as Weights>::NAME, "Weighted<i32>"); -} - -#[test] -fn test_vec_f64_weights_trait() { - let w = vec![1.5, 2.5]; - assert_eq!(w.len(), 2); - assert_eq!(w.weight(1), 2.5); - assert_eq!(<Vec<f64> as Weights>::NAME, "Weighted<f64>"); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test --lib test_numeric_size_blanket_impl test_unweighted_weights_trait test_vec_i32_weights_trait test_vec_f64_weights_trait` -Expected: FAIL — `NumericSize`, `Weights` not defined - -**Step 3: Implement `NumericSize`, `Weights`, and refactored `Unweighted`** - -In `src/types.rs`, add after the existing `NumericWeight` trait (we keep `NumericWeight` temporarily for backwards compat): - -```rust -/// Bound for objective value types (i32, f64, etc.) -pub trait NumericSize: - Clone + Default + PartialOrd + num_traits::Num + num_traits::Zero - + std::ops::AddAssign + 'static -{} - -impl<T> NumericSize for T -where - T: Clone + Default + PartialOrd + num_traits::Num + num_traits::Zero - + std::ops::AddAssign + 'static, -{} - -/// Trait for weight storage. Separates weight storage from objective value type. -pub trait Weights: Clone + 'static { - /// Name for variant metadata (e.g., "Unweighted", "Weighted<i32>"). - const NAME: &'static str; - /// The objective/metric type derived from these weights. - type Size: NumericSize; - /// Get the weight at a given index. - fn weight(&self, index: usize) -> Self::Size; - /// Number of weights. - fn len(&self) -> usize; - /// Whether the weight vector is empty. - fn is_empty(&self) -> bool { self.len() == 0 } -} -``` - -Change `Unweighted` from a zero-sized marker to a real weight vector: - -```rust -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default, Serialize, Deserialize)] -pub struct Unweighted(pub usize); - -impl Weights for Unweighted { - const NAME: &'static str = "Unweighted"; - type Size = i32; - fn weight(&self, _index: usize) -> i32 { 1 } - fn len(&self) -> usize { self.0 } -} - -impl Weights for Vec<i32> { - const NAME: &'static str = "Weighted<i32>"; - type Size = i32; - fn weight(&self, index: usize) -> i32 { self[index] } - fn len(&self) -> usize { self.len() } -} - -impl Weights for Vec<f64> { - const NAME: &'static str = "Weighted<f64>"; - type Size = f64; - fn weight(&self, index: usize) -> f64 { self[index] } - fn len(&self) -> usize { self.len() } -} -``` - -Keep `Unweighted::get()` method for backwards compat during migration. - -**Step 4: Run test to verify it passes** - -Run: `cargo test --lib test_numeric_size_blanket_impl test_unweighted_weights_trait test_vec_i32_weights_trait test_vec_f64_weights_trait` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/types.rs src/unit_tests/types.rs -git commit -m "feat: add NumericSize trait, Weights trait, and refactored Unweighted" -``` - ---- - -## Task 2: Add new `Problem` and `OptimizationProblem` traits to `src/traits.rs` - -**Files:** -- Modify: `src/traits.rs` -- Modify: `src/unit_tests/traits.rs` - -**Step 1: Write failing test** - -Add to `src/unit_tests/traits.rs`: - -```rust -use crate::types::{Direction, Weights}; - -#[derive(Clone)] -struct TestSatProblem { - num_vars: usize, - satisfying: Vec<Vec<usize>>, -} - -impl crate::traits::ProblemV2 for TestSatProblem { - const NAME: &'static str = "TestSat"; - type Metric = bool; - fn dims(&self) -> Vec<usize> { vec![2; self.num_vars] } - fn evaluate(&self, config: &[usize]) -> bool { - self.satisfying.iter().any(|s| s == config) - } -} - -#[test] -fn test_problem_v2_sat() { - let p = TestSatProblem { - num_vars: 2, - satisfying: vec![vec![1, 0], vec![0, 1]], - }; - assert_eq!(p.dims(), vec![2, 2]); - assert!(p.evaluate(&[1, 0])); - assert!(!p.evaluate(&[0, 0])); -} - -#[derive(Clone)] -struct TestOptProblem { - weights: Vec<i32>, -} - -impl crate::traits::ProblemV2 for TestOptProblem { - const NAME: &'static str = "TestOpt"; - type Metric = i32; - fn dims(&self) -> Vec<usize> { vec![2; self.weights.len()] } - fn evaluate(&self, config: &[usize]) -> i32 { - config.iter().enumerate() - .map(|(i, &v)| if v == 1 { self.weights[i] } else { 0 }) - .sum() - } -} - -impl crate::traits::OptimizationProblemV2 for TestOptProblem { - fn direction(&self) -> Direction { Direction::Maximize } -} - -#[test] -fn test_optimization_problem_v2() { - let p = TestOptProblem { weights: vec![3, 1, 4] }; - assert_eq!(p.evaluate(&[1, 0, 1]), 7); - assert_eq!(p.direction(), Direction::Maximize); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test --lib test_problem_v2_sat test_optimization_problem_v2` -Expected: FAIL — `ProblemV2`, `OptimizationProblemV2`, `Direction` not defined - -**Step 3: Implement new traits** - -In `src/types.rs`, add `Direction` enum: - -```rust -/// Optimization direction. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)] -pub enum Direction { - /// Maximize the objective value. - Maximize, - /// Minimize the objective value. - Minimize, -} -``` - -In `src/traits.rs`, add new traits (keep old ones during migration): - -```rust -use crate::types::Direction; - -/// Minimal problem trait — a problem is a function from configuration to metric. -pub trait ProblemV2: Clone { - /// Base name of this problem type. - const NAME: &'static str; - /// The evaluation metric type. - type Metric: Clone; - /// Configuration space dimensions. Each entry is the cardinality of that variable. - fn dims(&self) -> Vec<usize>; - /// Evaluate the problem on a configuration. - fn evaluate(&self, config: &[usize]) -> Self::Metric; - /// Number of variables (derived from dims). - fn num_variables(&self) -> usize { self.dims().len() } - /// Returns variant attributes derived from type parameters. - /// Used for generating variant IDs in the reduction graph schema. - /// Returns pairs like `[("graph", "SimpleGraph"), ("weight", "i32")]`. - fn variant() -> Vec<(&'static str, &'static str)>; -} - -/// Extension for problems with a numeric objective to optimize. -pub trait OptimizationProblemV2: ProblemV2 -where - Self::Metric: crate::types::NumericSize, -{ - /// Whether to maximize or minimize the metric. - fn direction(&self) -> Direction; -} -``` - -NOTE: We use `ProblemV2`/`OptimizationProblemV2` as temporary names. After all models are migrated (Task 6+), we rename to `Problem`/`OptimizationProblem` and remove old traits. - -**Step 4: Run test to verify it passes** - -Run: `cargo test --lib test_problem_v2_sat test_optimization_problem_v2` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/traits.rs src/types.rs src/unit_tests/traits.rs -git commit -m "feat: add ProblemV2, OptimizationProblemV2, and Direction" -``` - ---- - -## Task 3: Add new `ReductionResult` and `ReduceTo` traits in `src/rules/traits.rs` - -**Files:** -- Modify: `src/rules/traits.rs` -- Modify: `src/unit_tests/rules/traits.rs` - -**Step 1: Write failing test** - -Add to `src/unit_tests/rules/traits.rs`: - -```rust -use crate::traits::ProblemV2; -use crate::rules::traits::{ReductionResultV2, ReduceToV2}; - -#[derive(Clone)] -struct SourceProblem; -#[derive(Clone)] -struct TargetProblem; - -impl ProblemV2 for SourceProblem { - const NAME: &'static str = "Source"; - type Metric = i32; - fn dims(&self) -> Vec<usize> { vec![2, 2] } - fn evaluate(&self, config: &[usize]) -> i32 { (config[0] + config[1]) as i32 } -} - -impl ProblemV2 for TargetProblem { - const NAME: &'static str = "Target"; - type Metric = i32; - fn dims(&self) -> Vec<usize> { vec![2, 2] } - fn evaluate(&self, config: &[usize]) -> i32 { (config[0] + config[1]) as i32 } -} - -#[derive(Clone)] -struct TestReduction { target: TargetProblem } - -impl ReductionResultV2 for TestReduction { - type Source = SourceProblem; - type Target = TargetProblem; - fn target_problem(&self) -> &TargetProblem { &self.target } - fn extract_solution(&self, target_config: &[usize]) -> Vec<usize> { - target_config.to_vec() - } -} - -impl ReduceToV2<TargetProblem> for SourceProblem { - type Result = TestReduction; - fn reduce_to(&self) -> TestReduction { - TestReduction { target: TargetProblem } - } -} - -#[test] -fn test_reduction_v2() { - let source = SourceProblem; - let result = source.reduce_to(); - let target = result.target_problem(); - assert_eq!(target.evaluate(&[1, 1]), 2); - assert_eq!(result.extract_solution(&[1, 0]), vec![1, 0]); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test --lib test_reduction_v2` -Expected: FAIL — `ReductionResultV2`, `ReduceToV2` not defined - -**Step 3: Implement new reduction traits** - -In `src/rules/traits.rs`, add (keep old traits): - -```rust -use crate::traits::ProblemV2; - -/// Simplified reduction result — just target problem and solution extraction. -pub trait ReductionResultV2: Clone { - type Source: ProblemV2; - type Target: ProblemV2; - fn target_problem(&self) -> &Self::Target; - fn extract_solution(&self, target_config: &[usize]) -> Vec<usize>; -} - -/// Simplified reduction trait. -pub trait ReduceToV2<T: ProblemV2>: ProblemV2 { - type Result: ReductionResultV2<Source = Self, Target = T>; - fn reduce_to(&self) -> Self::Result; -} -``` - -Update `src/rules/mod.rs` to also export new traits: - -```rust -pub use traits::{ReduceTo, ReductionResult, ReduceToV2, ReductionResultV2}; -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test --lib test_reduction_v2` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/rules/traits.rs src/rules/mod.rs src/unit_tests/rules/traits.rs -git commit -m "feat: add ReductionResultV2 and ReduceToV2 traits" -``` - ---- - -## Task 4: Migrate one model as proof-of-concept — `MaximumIndependentSet` - -This task validates the full migration pattern. All subsequent model migrations follow the same steps. - -**Files:** -- Modify: `src/models/graph/maximum_independent_set.rs` -- Modify: `src/unit_tests/models/graph/maximum_independent_set.rs` - -**Step 1: Write failing test for new trait impl** - -Add to `src/unit_tests/models/graph/maximum_independent_set.rs`: - -```rust -#[test] -fn test_mis_problem_v2() { - use crate::traits::ProblemV2; - use crate::types::Direction; - - // Triangle graph with unit weights - let p = MaximumIndependentSet::<SimpleGraph, Vec<i32>>::with_weights( - 3, vec![(0, 1), (1, 2), (0, 2)], vec![1, 1, 1], - ); - assert_eq!(p.dims(), vec![2, 2, 2]); - // Valid IS: select vertex 0 only - assert_eq!(p.evaluate(&[1, 0, 0]), 1); - // Invalid IS: select adjacent 0,1 -> should return i32::MIN (neg inf for integers) - assert_eq!(p.evaluate(&[1, 1, 0]), i32::MIN); - assert_eq!(p.direction(), Direction::Maximize); -} - -#[test] -fn test_mis_unweighted_v2() { - use crate::traits::ProblemV2; - use crate::types::Unweighted; - - let p = MaximumIndependentSet::<SimpleGraph, Unweighted>::new_unweighted( - 3, vec![(0, 1), (1, 2), (0, 2)], - ); - assert_eq!(p.dims(), vec![2, 2, 2]); - assert_eq!(p.evaluate(&[1, 0, 0]), 1); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test --lib test_mis_problem_v2 test_mis_unweighted_v2` -Expected: FAIL - -**Step 3: Add `ProblemV2` and `OptimizationProblemV2` impls to MIS** - -In `src/models/graph/maximum_independent_set.rs`, add: - -```rust -use crate::traits::{ProblemV2, OptimizationProblemV2}; -use crate::types::{Direction, Weights}; - -impl MaximumIndependentSet<SimpleGraph, Unweighted> { - pub fn new_unweighted(num_vertices: usize, edges: Vec<(usize, usize)>) -> Self { - let graph = SimpleGraph::new(num_vertices, edges); - Self { graph, weights: Unweighted(num_vertices) } - } -} - -impl<G, W> ProblemV2 for MaximumIndependentSet<G, W> -where - G: Graph, - W: Weights, -{ - const NAME: &'static str = "MaximumIndependentSet"; - type Metric = W::Size; - - fn dims(&self) -> Vec<usize> { - vec![2; self.graph.num_vertices()] - } - - fn evaluate(&self, config: &[usize]) -> W::Size { - if !is_independent_set_config(&self.graph, config) { - // Return worst value for maximization - // For i32: i32::MIN, for f64: f64::NEG_INFINITY - return <W::Size as num_traits::Bounded>::min_value(); - } - let mut total = W::Size::zero(); - for (i, &selected) in config.iter().enumerate() { - if selected == 1 { - total += self.weights.weight(i); - } - } - total - } -} - -impl<G, W> OptimizationProblemV2 for MaximumIndependentSet<G, W> -where - G: Graph, - W: Weights, - W::Size: crate::types::NumericSize, -{ - fn direction(&self) -> Direction { - Direction::Maximize - } -} -``` - -NOTE: The "return worst value" pattern requires `num_traits::Bounded`. Add this bound to `NumericSize`: - -In `src/types.rs`, update `NumericSize`: -```rust -pub trait NumericSize: - Clone + Default + PartialOrd + num_traits::Num + num_traits::Zero - + num_traits::Bounded + std::ops::AddAssign + 'static -{} -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test --lib test_mis_problem_v2 test_mis_unweighted_v2` -Expected: PASS - -**Step 5: Run full test suite to verify nothing is broken** - -Run: `make test` -Expected: All existing tests still PASS (old traits untouched) - -**Step 6: Commit** - -```bash -git add src/types.rs src/traits.rs src/models/graph/maximum_independent_set.rs src/unit_tests/models/graph/maximum_independent_set.rs -git commit -m "feat: add ProblemV2 impl for MaximumIndependentSet (proof of concept)" -``` - ---- - -## Task 5: Migrate remaining graph models - -Repeat the Task 4 pattern for each graph model. Each should implement `ProblemV2` (and `OptimizationProblemV2` where applicable). - -**Files to modify (one per sub-step):** -- `src/models/graph/minimum_vertex_cover.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/graph/maximum_clique.rs` — `ProblemV2` + `OptimizationProblemV2` (Maximize) -- `src/models/graph/max_cut.rs` — `ProblemV2` + `OptimizationProblemV2` (Maximize) -- `src/models/graph/maximum_matching.rs` — `ProblemV2` + `OptimizationProblemV2` (Maximize) -- `src/models/graph/minimum_dominating_set.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/graph/maximal_is.rs` — `ProblemV2` + `OptimizationProblemV2` (Maximize) -- `src/models/graph/kcoloring.rs` — `ProblemV2` only (Metric = bool, no `OptimizationProblemV2`). Remove `PhantomData<W>`, change to `KColoring<G>` with runtime `k: usize` field. - -For **KColoring** specifically, the struct changes to: - -```rust -pub struct KColoring<G> { - graph: G, - k: usize, -} - -impl<G: Graph> ProblemV2 for KColoring<G> { - const NAME: &'static str = "KColoring"; - type Metric = bool; - fn dims(&self) -> Vec<usize> { vec![self.k; self.graph.num_vertices()] } - fn evaluate(&self, config: &[usize]) -> bool { self.is_valid_coloring(config) } -} -``` - -NOTE: KColoring changes its type signature (`<const K, G, W>` -> `<G>`), which breaks existing reductions that reference it. Keep the old struct as a type alias during migration: -```rust -pub type KColoringLegacy<const K: usize, G, W> = KColoring<G>; -``` - -**Commit after each model:** one commit per model file. - ---- - -## Task 6: Migrate optimization models - -**Files to modify:** -- `src/models/optimization/qubo.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize), `W: Weights` where `W::Size: Mul<Output = W::Size>` -- `src/models/optimization/spin_glass.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/optimization/ilp.rs` — `ProblemV2` + `OptimizationProblemV2` (uses `ObjectiveSense`) - -**Commit after each model.** - ---- - -## Task 7: Migrate satisfiability models - -**Files to modify:** -- `src/models/satisfiability/sat.rs` — `ProblemV2` only (Metric = bool for SAT, or `W::Size` for MAX-SAT) -- `src/models/satisfiability/ksat.rs` — `ProblemV2` only (Metric = bool) -- `src/models/specialized/circuit.rs` — `ProblemV2` only (Metric = bool) -- `src/models/specialized/factoring.rs` — `ProblemV2` only (Metric = bool) - -**Commit after each model.** - ---- - -## Task 8: Migrate set models and remaining specialized models - -**Files to modify:** -- `src/models/set/minimum_set_covering.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/set/maximum_set_packing.rs` — `ProblemV2` + `OptimizationProblemV2` (Maximize) -- `src/models/specialized/paintshop.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/specialized/biclique_cover.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) -- `src/models/specialized/bmf.rs` — `ProblemV2` + `OptimizationProblemV2` (Minimize) - -**Commit after each model.** - ---- - -## Task 9: Update solvers to use new traits - -**Files:** -- Modify: `src/solvers/mod.rs` -- Modify: `src/solvers/brute_force.rs` -- Modify: `src/unit_tests/solvers/brute_force.rs` - -**Step 1: Add new `SolverV2` trait** - -In `src/solvers/mod.rs`: - -```rust -use crate::traits::{ProblemV2, OptimizationProblemV2}; -use crate::types::Direction; - -pub trait SolverV2 { - /// Find best solution(s) for an optimization problem. - fn find_best_optimization<P: OptimizationProblemV2>( - &self, problem: &P, - ) -> Vec<Vec<usize>> - where P::Metric: crate::types::NumericSize; - - /// Find any satisfying solution for a satisfaction problem (Metric = bool). - fn find_satisfying<P: ProblemV2<Metric = bool>>( - &self, problem: &P, - ) -> Option<Vec<usize>>; -} -``` - -**Step 2: Implement for `BruteForce`** - -In `src/solvers/brute_force.rs`, add `SolverV2` impl that uses `evaluate()` and `direction()` instead of `solution_size()` and `energy_mode()`. - -**Step 3: Test with new traits** - -Add tests in `src/unit_tests/solvers/brute_force.rs` using `ProblemV2`-based problems. - -**Step 4: Commit** - -```bash -git add src/solvers/ -git commit -m "feat: add SolverV2 using ProblemV2/OptimizationProblemV2" -``` - ---- - -## Task 10: Update proc macro for trait-bound inspection - -**Files:** -- Modify: `problemreductions-macros/src/lib.rs` - -**Step 1: Replace type extraction heuristics with trait-bound inspection** - -Replace `extract_graph_type()`, `extract_weight_type()`, `is_weight_type()`, `get_weight_name()` with: - -```rust -/// Inspect impl block's generic params and their bounds to identify roles. -fn extract_roles_from_bounds(impl_block: &ItemImpl) -> (Option<String>, Option<String>) { - let mut graph_type = None; - let mut weight_type = None; - - for param in &impl_block.generics.params { - if let syn::GenericParam::Type(type_param) = param { - for bound in &type_param.bounds { - if let syn::TypeParamBound::Trait(trait_bound) = bound { - let trait_name = trait_bound.path.segments.last() - .map(|s| s.ident.to_string()); - match trait_name.as_deref() { - Some("Graph") => graph_type = Some(type_param.ident.to_string()), - Some("Weights") => weight_type = Some(type_param.ident.to_string()), - _ => {} - } - } - } - } - } - - (graph_type, weight_type) -} -``` - -For concrete types in the signature (e.g., `SimpleGraph` in `ReduceTo<QUBO<f64>> for MIS<SimpleGraph, i32>`), match them against the type arguments and use string literals. - -**Step 2: Update `generate_reduction_entry` to use new extraction** - -Remove all the old `extract_graph_type`, `is_weight_type`, `get_weight_name` functions. The new logic: -1. Call `extract_roles_from_bounds()` to find which generic params are Graph/Weights -2. For generic params: use the trait's `NAME` constant at registration time -3. For concrete types in signatures: use literal strings -4. Emit compile error if structure is ambiguous - -**Step 3: Test** - -Run: `make test` -Expected: All existing reductions still compile and register correctly - -**Step 4: Commit** - -```bash -git add problemreductions-macros/src/lib.rs -git commit -m "refactor: replace macro heuristics with trait-bound inspection" -``` - ---- - -## Task 11: Swap old traits for new — the rename - -Once all models, solvers, and reductions implement the V2 traits, perform the swap. - -**Files:** -- Modify: `src/traits.rs` — rename `ProblemV2` -> `Problem`, `OptimizationProblemV2` -> `OptimizationProblem`, remove old `Problem` and `ConstraintSatisfactionProblem`. **Keep `fn variant() -> Vec<(&'static str, &'static str)>` in the Problem trait** for schema/registry variant ID generation. -- Modify: `src/rules/traits.rs` — rename `ReductionResultV2` -> `ReductionResult`, `ReduceToV2` -> `ReduceTo`, remove old traits -- Modify: `src/types.rs` — remove `EnergyMode`, `SolutionSize`, `LocalConstraint`, `LocalSolutionSize`, `NumericWeight`, old `Unweighted`. Remove `csp_solution_size()`. **Add `NumericSizeBounds` trait** for bound-checking in solvers. -- Modify: `src/lib.rs` — update prelude and re-exports -- Modify: `src/variant.rs` — **KEEP** `short_type_name` and `const_usize_str` (still used by `Problem::variant()` impls) -- Modify: ALL model files — remove old `Problem` / `CSP` impls, keep only new impls. **Each model must implement `fn variant()`** returning type parameter metadata. -- Modify: ALL rule files — update to use new traits -- Modify: ALL solver files — remove old `Solver` trait, keep `SolverV2` renamed to `Solver` -- Modify: ALL test files — update imports -- Modify: ALL example files — update to use new API (`solution_size` -> `evaluate`, keep `variant()` calls) - -**This is the largest task.** Break it into sub-steps: - -1. Rename traits in `src/traits.rs` and `src/rules/traits.rs` -2. Update `src/types.rs` (remove dead types) -3. Update `src/lib.rs` prelude -4. Update each model file (remove old impls) -5. Update each rule file -6. Update each solver file -7. Update each test file -8. Update each example file -9. Run `make test clippy` after each batch - -**Commit frequently** — at minimum one commit per sub-step. - ---- - -## Task 12: Clean up and verify - -**Step 1: Run full test suite** - -```bash -make test -``` - -**Step 2: Run clippy** - -```bash -make clippy -``` - -**Step 3: Check formatting** - -```bash -make fmt-check -``` - -**Step 4: Run coverage** - -```bash -make coverage -``` -Expected: >95% coverage - -**Step 5: Regenerate reduction graph** - -```bash -cargo run --example export_graph -``` - -**Step 6: Build docs** - -```bash -make doc -``` - -**Step 7: Final commit** - -```bash -git add -A -git commit -m "chore: cleanup after trait refactoring" -``` - ---- - -## Migration Strategy Summary - -The key principle is **parallel existence**: new traits (`ProblemV2`, `OptimizationProblemV2`, `ReductionResultV2`, `ReduceToV2`) coexist with old traits throughout the migration. This means: - -- The codebase compiles and all tests pass at every commit -- Models can be migrated one at a time -- The final rename (Task 11) is the only "big bang" change - -**Dependency order:** -1. Types (`NumericSize`, `Weights`, `Direction`) — no dependencies -2. Traits (`ProblemV2`, `OptimizationProblemV2`) — depends on types -3. Reduction traits (`ReductionResultV2`, `ReduceToV2`) — depends on `ProblemV2` -4. Models — depends on all above -5. Solvers — depends on models + traits -6. Proc macro — independent (just registration metadata) -7. Rename — depends on everything being migrated -8. Cleanup — depends on rename diff --git a/docs/plans/2026-02-13-documentation-improvements-design.md b/docs/plans/2026-02-13-documentation-improvements-design.md deleted file mode 100644 index 814a4a8d3..000000000 --- a/docs/plans/2026-02-13-documentation-improvements-design.md +++ /dev/null @@ -1,136 +0,0 @@ -# Documentation Improvements Design - -## Goal - -Improve the "Getting Started" and "Architecture" sections of the mdBook documentation. - -- **Getting Started**: For all audiences (researchers, developers, students, contributors) -- **Architecture**: For contributors and developers - -## Getting Started - -### Current Problems - -- Jumps straight into code without explaining what the library does -- No high-level workflow overview -- Missing JSON resource documentation - -### Proposed Structure - -1. **What This Library Does** (~50 words) - - One paragraph explaining: "Reduce hard problems to solver-friendly forms" - - Link to Introduction page for the interactive reduction graph - -2. **Installation** (keep existing content) - -3. **The Reduction Workflow** - - Cetz diagram showing: `Problem A → reduce → Problem B → solve → extract → Solution for A` - - One complete code example walking through each step with comments - - Brief mention: chaining reductions works the same way - - Note: automated reduction path optimization for connected problems coming in the future - -4. **Solvers** (brief, with links) - - `BruteForce` — for small instances (<20 variables) - - `ILPSolver` — for larger instances (requires `ilp` feature) - - Link to API docs for details - -5. **JSON Resources** (new section) - - `reduction_graph.json` — all problems and reduction edges; useful for tooling, visualization, and research - - `problem_schemas.json` — field definitions for each problem type - - Location: `docs/src/reductions/` in the built docs, or generate via `cargo run --example export_graph` - -6. **Next Steps** - - Link to Architecture for internals - - Link to API Reference for full documentation - -## Architecture - -### Current Problems - -- Outdated trait references (`solution_size()` → `evaluate()`, `ConstraintSatisfactionProblem` removed) -- No visual diagram of module relationships -- Unclear entry points for contributors - -### Proposed Structure - -1. **Module Overview** (new) - - Cetz diagram showing module relationships: - ``` - models/ ←→ rules/ → registry/ - ↑ - solvers/ - ``` - - One sentence description per module - -2. **Trait Hierarchy** (updated) - - Cetz diagram showing trait relationships - - Updated to current API: - - `Problem`: `NAME`, `Metric`, `dims()`, `evaluate()`, `variant()`, `num_variables()` - - `OptimizationProblem`: `Value`, `direction()` - - Remove references to `ConstraintSatisfactionProblem` - -3. **Problems** (streamlined) - - Keep graph types table (SimpleGraph, GridGraph, UnitDiskGraph, HyperGraph) - - Keep variant ID explanation - - Update code examples to use `evaluate()` instead of `solution_size()` - -4. **Reductions** (streamlined) - - Keep reduce → solve → extract explanation - - Update `#[reduction]` macro example to current syntax - - Keep overhead tracking mention - -5. **Registry** (keep mostly as-is) - - JSON schema details are good - - Keep `reduction_graph.json` and `problem_schemas.json` schema examples - -6. **Solvers** (keep mostly as-is) - - Update trait signature if needed - -7. **Contributing** (new section, replaces scattered links) - - Priority order: - - a. **Recommended: Issue-based workflow** - - Open an issue using [Problem template](link) or [Rule template](link) - - Fill in all sections (definition, algorithm, size overhead, example instance) - - AI handles implementation automatically - - b. **Optional: Plan + automated PR** - - Use `superpowers:brainstorming` to create a detailed plan - - Create PR with `[action]` prefix in description to trigger automated implementation - - c. **Last resort: Manual implementation** - - See `adding-models.md` for adding problem types - - See `adding-reductions.md` for adding reduction rules - - See `testing.md` for test requirements - -## Diagrams - -Three separate Cetz diagrams in Typst, output to `docs/src/static/`: - -1. **`module-overview.typ`** → `module-overview.svg` - - Shows relationships between `src/models/`, `src/rules/`, `src/registry/`, `src/solvers/` - - Arrows showing data flow and dependencies - -2. **`trait-hierarchy.typ`** → `trait-hierarchy.svg` - - `Problem` trait with key methods - - `OptimizationProblem` extension - - Type parameters (`Metric`, `Value`) - -3. **`reduction-workflow.typ`** → `reduction-workflow.svg` - - Linear flow: Create Problem → Reduce → Solve Target → Extract Solution - - Shows the round-trip nature - -## Files to Modify - -- `docs/src/getting-started.md` — rewrite -- `docs/src/arch.md` — update -- `docs/src/static/module-overview.typ` — new -- `docs/src/static/trait-hierarchy.typ` — new -- `docs/src/static/reduction-workflow.typ` — new - -## Out of Scope - -- Introduction page (already has reduction graph) -- API reference (auto-generated) -- CLAUDE.md (separate concern) diff --git a/docs/plans/2026-02-13-documentation-improvements-impl.md b/docs/plans/2026-02-13-documentation-improvements-impl.md deleted file mode 100644 index b0665f754..000000000 --- a/docs/plans/2026-02-13-documentation-improvements-impl.md +++ /dev/null @@ -1,642 +0,0 @@ -# Documentation Improvements Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Improve Getting Started and Architecture documentation with diagrams, updated API references, and clearer structure. - -**Architecture:** Three Typst diagrams (using fletcher) compiled to SVG, then referenced in markdown docs. Getting Started focuses on workflow; Architecture focuses on internals and contribution paths. - -**Tech Stack:** Typst with fletcher, mdBook markdown, SVG output - ---- - -## Task 1: Create Reduction Workflow Diagram - -**Files:** -- Create: `docs/src/static/reduction-workflow.typ` -- Output: `docs/src/static/reduction-workflow.svg`, `docs/src/static/reduction-workflow-dark.svg` - -**Step 1: Create the Typst diagram** - -Create `docs/src/static/reduction-workflow.typ`: - -```typst -#import "@preview/fletcher:0.5.8" as fletcher: diagram, node, edge -#set page(width: auto, height: auto, margin: (top: 5pt, bottom: 5pt, left: 5pt, right: 5pt), fill: none) -#set text(font: "Noto Sans CJK SC") - -#let reduction-workflow(dark: false) = { - let (fg, box-fill) = if dark { - (rgb("#e2e8f0"), rgb("#1e293b")) - } else { - (rgb("#1e293b"), rgb("#f8fafc")) - } - - set text(fill: fg, size: 10pt) - - diagram( - node-stroke: 1.5pt + fg, - edge-stroke: 1.5pt, - spacing: (20mm, 10mm), - - let accent = rgb("#3b82f6"), - let success = rgb("#22c55e"), - - // Nodes - node((0, 0), box(width: 28mm, align(center)[*Problem A*\ #text(size: 8pt)[source problem]]), fill: box-fill, corner-radius: 6pt, inset: 10pt, name: <a>), - node((1, 0), box(width: 28mm, align(center)[*Problem B*\ #text(size: 8pt)[target problem]]), fill: box-fill, corner-radius: 6pt, inset: 10pt, name: <b>), - node((2, 0), box(width: 28mm, align(center)[*Solution B*\ #text(size: 8pt)[solver output]]), fill: box-fill, corner-radius: 6pt, inset: 10pt, name: <sol-b>), - node((1, 1), box(width: 28mm, align(center)[*Solution A*\ #text(size: 8pt)[extracted result]]), fill: rgb("#dcfce7"), stroke: 1.5pt + success, corner-radius: 6pt, inset: 10pt, name: <sol-a>), - - // Edges with labels - edge(<a>, <b>, "->", stroke: 1.5pt + accent, label: text(size: 9pt)[`reduce_to()`], label-pos: 0.5, label-side: center), - edge(<b>, <sol-b>, "->", stroke: 1.5pt + accent, label: text(size: 9pt)[`find_best()`], label-pos: 0.5, label-side: center), - edge(<sol-b>, <sol-a>, "->", stroke: 1.5pt + success, label: text(size: 9pt)[`extract_solution()`], label-pos: 0.5, label-side: center), - ) -} - -#let standalone-dark = sys.inputs.at("dark", default: "false") == "true" -#reduction-workflow(dark: standalone-dark) -``` - -**Step 2: Compile to SVG (light and dark)** - -```bash -cd docs/src/static -typst compile reduction-workflow.typ --input dark=false reduction-workflow.svg -typst compile reduction-workflow.typ --input dark=true reduction-workflow-dark.svg -``` - -**Step 3: Verify output** - -```bash -ls -la docs/src/static/reduction-workflow*.svg -``` - -Expected: Two SVG files created - -**Step 4: Commit** - -```bash -git add docs/src/static/reduction-workflow.typ docs/src/static/reduction-workflow.svg docs/src/static/reduction-workflow-dark.svg -git commit -m "docs: add reduction workflow diagram" -``` - ---- - -## Task 2: Create Module Overview Diagram - -**Files:** -- Create: `docs/src/static/module-overview.typ` -- Output: `docs/src/static/module-overview.svg`, `docs/src/static/module-overview-dark.svg` - -**Step 1: Create the Typst diagram** - -Create `docs/src/static/module-overview.typ`: - -```typst -#import "@preview/fletcher:0.5.8" as fletcher: diagram, node, edge -#set page(width: auto, height: auto, margin: (top: 5pt, bottom: 5pt, left: 5pt, right: 5pt), fill: none) -#set text(font: "Noto Sans CJK SC") - -#let module-overview(dark: false) = { - let (fg, box-fill) = if dark { - (rgb("#e2e8f0"), rgb("#1e293b")) - } else { - (rgb("#1e293b"), rgb("#f8fafc")) - } - - set text(fill: fg, size: 10pt) - - diagram( - node-stroke: 1.5pt + fg, - edge-stroke: 1.5pt, - spacing: (25mm, 15mm), - - let model-color = rgb("#c8f0c8"), - let rule-color = rgb("#c8c8f0"), - let registry-color = rgb("#f0f0a0"), - let solver-color = rgb("#f0c8c8"), - - // Module nodes - node((0, 0), box(width: 30mm, align(center)[*models/*\ #text(size: 8pt)[Problem types]]), fill: model-color, corner-radius: 6pt, inset: 10pt, name: <models>), - node((1, 0), box(width: 30mm, align(center)[*rules/*\ #text(size: 8pt)[Reductions]]), fill: rule-color, corner-radius: 6pt, inset: 10pt, name: <rules>), - node((2, 0), box(width: 30mm, align(center)[*registry/*\ #text(size: 8pt)[Graph metadata]]), fill: registry-color, corner-radius: 6pt, inset: 10pt, name: <registry>), - node((1, 1), box(width: 30mm, align(center)[*solvers/*\ #text(size: 8pt)[BruteForce, ILP]]), fill: solver-color, corner-radius: 6pt, inset: 10pt, name: <solvers>), - - // Relationships - edge(<models>, <rules>, "<->", label: text(size: 8pt)[imports], label-side: center), - edge(<rules>, <registry>, "->", label: text(size: 8pt)[registers], label-side: center), - edge(<solvers>, <models>, "->", label: text(size: 8pt)[solves], label-side: center), - ) -} - -#let standalone-dark = sys.inputs.at("dark", default: "false") == "true" -#module-overview(dark: standalone-dark) -``` - -**Step 2: Compile to SVG** - -```bash -cd docs/src/static -typst compile module-overview.typ --input dark=false module-overview.svg -typst compile module-overview.typ --input dark=true module-overview-dark.svg -``` - -**Step 3: Verify output** - -```bash -ls -la docs/src/static/module-overview*.svg -``` - -**Step 4: Commit** - -```bash -git add docs/src/static/module-overview.typ docs/src/static/module-overview.svg docs/src/static/module-overview-dark.svg -git commit -m "docs: add module overview diagram" -``` - ---- - -## Task 3: Create Trait Hierarchy Diagram - -**Files:** -- Create: `docs/src/static/trait-hierarchy.typ` -- Output: `docs/src/static/trait-hierarchy.svg`, `docs/src/static/trait-hierarchy-dark.svg` - -**Step 1: Create the Typst diagram** - -Create `docs/src/static/trait-hierarchy.typ`: - -```typst -#import "@preview/fletcher:0.5.8" as fletcher: diagram, node, edge -#set page(width: auto, height: auto, margin: (top: 5pt, bottom: 5pt, left: 5pt, right: 5pt), fill: none) -#set text(font: "Noto Sans CJK SC") - -#let trait-hierarchy(dark: false) = { - let (fg, box-fill) = if dark { - (rgb("#e2e8f0"), rgb("#1e293b")) - } else { - (rgb("#1e293b"), rgb("#f8fafc")) - } - - set text(fill: fg, size: 9pt) - - diagram( - node-stroke: 1.5pt + fg, - edge-stroke: 1.5pt, - spacing: (8mm, 12mm), - - let trait-fill = rgb("#e0e7ff"), - let type-fill = rgb("#fef3c7"), - - // Problem trait (main) - node((0, 0), box(width: 55mm, align(left)[ - *trait Problem*\ - #text(size: 8pt, fill: rgb("#6b7280"))[ - `const NAME: &str`\ - `type Metric: Clone`\ - `fn dims() -> Vec<usize>`\ - `fn evaluate(&config) -> Metric`\ - `fn variant() -> Vec<(&str, &str)>` - ] - ]), fill: trait-fill, corner-radius: 6pt, inset: 10pt, name: <problem>), - - // OptimizationProblem trait - node((0, 1), box(width: 55mm, align(left)[ - *trait OptimizationProblem*\ - #text(size: 8pt, fill: rgb("#6b7280"))[ - `type Value: PartialOrd + Clone`\ - `fn direction() -> Direction`\ - #text(style: "italic")[requires `Metric = SolutionSize<Value>`] - ] - ]), fill: trait-fill, corner-radius: 6pt, inset: 10pt, name: <opt>), - - // Type boxes on the right - node((1.3, 0), box(width: 38mm, align(left)[ - *SolutionSize\<T\>*\ - #text(size: 8pt, fill: rgb("#6b7280"))[`Valid(T) | Invalid`] - ]), fill: type-fill, corner-radius: 6pt, inset: 8pt, name: <solsize>), - - node((1.3, 1), box(width: 38mm, align(left)[ - *Direction*\ - #text(size: 8pt, fill: rgb("#6b7280"))[`Maximize | Minimize`] - ]), fill: type-fill, corner-radius: 6pt, inset: 8pt, name: <dir>), - - // Inheritance arrow - edge(<opt>, <problem>, "->", stroke: 1.5pt + fg, label: text(size: 8pt)[extends], label-side: center), - - // Type associations (dashed) - edge(<problem>, <solsize>, "-->", stroke: (paint: fg, dash: "dashed")), - edge(<opt>, <dir>, "-->", stroke: (paint: fg, dash: "dashed")), - ) -} - -#let standalone-dark = sys.inputs.at("dark", default: "false") == "true" -#trait-hierarchy(dark: standalone-dark) -``` - -**Step 2: Compile to SVG** - -```bash -cd docs/src/static -typst compile trait-hierarchy.typ --input dark=false trait-hierarchy.svg -typst compile trait-hierarchy.typ --input dark=true trait-hierarchy-dark.svg -``` - -**Step 3: Verify output** - -```bash -ls -la docs/src/static/trait-hierarchy*.svg -``` - -**Step 4: Commit** - -```bash -git add docs/src/static/trait-hierarchy.typ docs/src/static/trait-hierarchy.svg docs/src/static/trait-hierarchy-dark.svg -git commit -m "docs: add trait hierarchy diagram" -``` - ---- - -## Task 4: Rewrite Getting Started - -**Files:** -- Modify: `docs/src/getting-started.md` - -**Step 1: Rewrite the file** - -Replace contents of `docs/src/getting-started.md` with: - -```markdown -# Getting Started - -## What This Library Does - -**problemreductions** transforms hard computational problems into forms that efficient solvers can handle. You define a problem, reduce it to another problem type (like QUBO or ILP), solve the reduced problem, and extract the solution back. The [interactive reduction graph](./introduction.html) shows all available problem types and transformations. - -## Installation - -Add to your `Cargo.toml`: - -```toml -[dependencies] -problemreductions = "0.1" -``` - -## The Reduction Workflow - -The core workflow is: **create** a problem, **reduce** it to a target, **solve** the target, and **extract** the solution back. - -<div class="theme-light-only"> - -![Reduction Workflow](static/reduction-workflow.svg) - -</div> -<div class="theme-dark-only"> - -![Reduction Workflow](static/reduction-workflow-dark.svg) - -</div> - -### Complete Example - -```rust -use problemreductions::prelude::*; - -// 1. Create: Independent Set on a path graph (4 vertices) -let problem = MaximumIndependentSet::<i32>::new(4, vec![(0, 1), (1, 2), (2, 3)]); - -// 2. Reduce: Transform to Minimum Vertex Cover -let reduction = ReduceTo::<MinimumVertexCover<i32>>::reduce_to(&problem); -let target = reduction.target_problem(); - -// 3. Solve: Find optimal solution to the target problem -let solver = BruteForce::new(); -let target_solutions = solver.find_best(target); - -// 4. Extract: Map solution back to original problem -let solution = reduction.extract_solution(&target_solutions[0]); - -// Verify: solution is valid for the original problem -let metric = problem.evaluate(&solution); -assert!(metric.is_valid()); -``` - -### Chaining Reductions - -Reductions can be chained. Each step preserves the solution mapping: - -```rust -use problemreductions::prelude::*; - -// SetPacking -> IndependentSet -> VertexCover -let sp = MaximumSetPacking::<i32>::new(vec![vec![0, 1], vec![1, 2], vec![2, 3]]); - -let r1 = ReduceTo::<MaximumIndependentSet<i32>>::reduce_to(&sp); -let r2 = ReduceTo::<MinimumVertexCover<i32>>::reduce_to(r1.target_problem()); - -// Solve final target, extract back through chain -let solver = BruteForce::new(); -let vc_sol = solver.find_best(r2.target_problem()); -let is_sol = r2.extract_solution(&vc_sol[0]); -let sp_sol = r1.extract_solution(&is_sol); -``` - -## Solvers - -Two solvers are available: - -| Solver | Use Case | Notes | -|--------|----------|-------| -| [`BruteForce`](api/problemreductions/solvers/struct.BruteForce.html) | Small instances (<20 variables) | Enumerates all configurations | -| [`ILPSolver`](api/problemreductions/solvers/ilp/struct.ILPSolver.html) | Larger instances | Requires `ilp` feature flag | - -Enable ILP support: - -```toml -[dependencies] -problemreductions = { version = "0.1", features = ["ilp"] } -``` - -**Future:** Automated reduction path optimization will find the best route between any two connected problems. - -## JSON Resources - -The library exports machine-readable metadata useful for tooling and research: - -| File | Contents | Use Case | -|------|----------|----------| -| [`reduction_graph.json`](reductions/reduction_graph.json) | All problem variants and reduction edges | Visualization, path finding, research | -| [`problem_schemas.json`](reductions/problem_schemas.json) | Field definitions for each problem type | Code generation, validation | - -Generate locally: - -```bash -cargo run --example export_graph # reduction_graph.json -cargo run --example export_schemas # problem_schemas.json -``` - -## Next Steps - -- Explore the [interactive reduction graph](./introduction.html) to discover available reductions -- Read the [Architecture](./arch.md) guide for implementation details -- Browse the [API Reference](./api.html) for full documentation -``` - -**Step 2: Verify markdown renders** - -```bash -cd docs && mdbook build && echo "Build successful" -``` - -**Step 3: Commit** - -```bash -git add docs/src/getting-started.md -git commit -m "docs: rewrite Getting Started with workflow focus" -``` - ---- - -## Task 5: Update Architecture - Module Overview Section - -**Files:** -- Modify: `docs/src/arch.md` (partial update) - -**Step 1: Read current file to get line numbers** - -```bash -head -60 docs/src/arch.md -``` - -**Step 2: Replace the beginning of arch.md (lines 1-58) with new Module Overview** - -Replace the beginning of `docs/src/arch.md` up through the Problems section header with: - -```markdown -# Architecture - -This guide covers the library internals for contributors and developers extending the library. - -## Module Overview - -<div class="theme-light-only"> - -![Module Overview](static/module-overview.svg) - -</div> -<div class="theme-dark-only"> - -![Module Overview](static/module-overview-dark.svg) - -</div> - -| Module | Purpose | -|--------|---------| -| `src/models/` | Problem type implementations (SAT, Graph, Set, Optimization) | -| `src/rules/` | Reduction rules with `ReduceTo` implementations | -| `src/registry/` | Compile-time reduction graph metadata | -| `src/solvers/` | BruteForce and ILP solvers | -| `src/traits.rs` | Core `Problem` and `OptimizationProblem` traits | -| `src/types.rs` | Shared types (`SolutionSize`, `Direction`, `ProblemSize`) | - -## Trait Hierarchy - -<div class="theme-light-only"> - -![Trait Hierarchy](static/trait-hierarchy.svg) - -</div> -<div class="theme-dark-only"> - -![Trait Hierarchy](static/trait-hierarchy-dark.svg) - -</div> - -Every problem implements `Problem`. Optimization problems additionally implement `OptimizationProblem`. - -```rust -pub trait Problem: Clone { - const NAME: &'static str; // e.g., "MaximumIndependentSet" - type Metric: Clone; // SolutionSize<W> or bool - fn dims(&self) -> Vec<usize>; // config space: [2, 2, 2] for 3 binary vars - fn evaluate(&self, config: &[usize]) -> Self::Metric; - fn variant() -> Vec<(&'static str, &'static str)>; -} - -pub trait OptimizationProblem: Problem<Metric = SolutionSize<Self::Value>> { - type Value: PartialOrd + Clone; // i32, f64, etc. - fn direction(&self) -> Direction; // Maximize or Minimize -} -``` - -**Key types:** -- `SolutionSize<T>`: `Valid(T)` for feasible solutions, `Invalid` for constraint violations -- `Direction`: `Maximize` or `Minimize` - -## Problems -``` - -**Step 3: Commit partial update** - -```bash -git add docs/src/arch.md -git commit -m "docs: update Architecture with module overview and trait hierarchy" -``` - ---- - -## Task 6: Update Architecture - Problems and Rules Sections - -**Files:** -- Modify: `docs/src/arch.md` (continue update) - -**Step 1: Update the Problems section** - -Find and update the Problems section to fix outdated API references. Replace `solution_size(&config)` with `evaluate(&config)`: - -Old: -```rust -let config = vec![1, 0, 1, 0]; -let result = problem.solution_size(&config); -// result.is_valid: bool -// result.size: objective value -``` - -New: -```rust -let config = vec![1, 0, 1, 0]; -let result = problem.evaluate(&config); -// result.is_valid() -> bool -// result.size() -> Option<&T> -``` - -**Step 2: Update the Rules section** - -Replace the outdated reduction example: - -Old: -```rust -let reduction = problem.reduce_to::<QUBO<f64>>(); -``` - -New: -```rust -let reduction = ReduceTo::<QUBO<f64>>::reduce_to(&problem); -``` - -**Step 3: Remove reference to ConstraintSatisfactionProblem** - -Delete the line: "For problems with explicit constraints, also implement `ConstraintSatisfactionProblem`." - -**Step 4: Commit** - -```bash -git add docs/src/arch.md -git commit -m "docs: fix outdated API references in Architecture" -``` - ---- - -## Task 7: Add Contributing Section to Architecture - -**Files:** -- Modify: `docs/src/arch.md` (append) - -**Step 1: Add Contributing section at the end of arch.md** - -Append to `docs/src/arch.md`: - -```markdown - -## Contributing - -### Recommended: Issue-Based Workflow - -The easiest way to contribute is through GitHub issues: - -1. **Open an issue** using the [Problem](https://github.com/CodingThrust/problem-reductions/issues/new?template=problem.md) or [Rule](https://github.com/CodingThrust/problem-reductions/issues/new?template=rule.md) template -2. **Fill in all sections** — definition, algorithm, size overhead, example instance -3. **AI handles implementation** — automated tools generate the code from your specification - -### Optional: Plan + Automated PR - -For more control over the implementation: - -1. Use `superpowers:brainstorming` to create a detailed plan -2. Create a PR with `[action]` prefix in the description -3. Automated implementation is triggered from your plan - -### Manual Implementation - -When automation isn't suitable: - -- **Adding a problem:** See [adding-models.md](https://github.com/CodingThrust/problem-reductions/blob/main/.claude/rules/adding-models.md) -- **Adding a reduction:** See [adding-reductions.md](https://github.com/CodingThrust/problem-reductions/blob/main/.claude/rules/adding-reductions.md) -- **Testing requirements:** See [testing.md](https://github.com/CodingThrust/problem-reductions/blob/main/.claude/rules/testing.md) - -Run `make test clippy` before submitting PRs. -``` - -**Step 2: Commit** - -```bash -git add docs/src/arch.md -git commit -m "docs: add Contributing section to Architecture" -``` - ---- - -## Task 8: Build and Verify Documentation - -**Files:** -- None (verification only) - -**Step 1: Build the mdBook** - -```bash -make doc -``` - -**Step 2: Verify diagrams are included** - -```bash -ls -la docs/book/static/*.svg | grep -E "(reduction-workflow|module-overview|trait-hierarchy)" -``` - -Expected: All 6 SVG files present (light + dark for each) - -**Step 3: Open locally and visual check** - -```bash -open docs/book/getting-started.html -open docs/book/arch.html -``` - -Verify: -- Diagrams render correctly -- Light/dark theme switching works -- Links work - -**Step 4: Final commit if any fixes needed** - -```bash -git status -# If clean, done. If changes needed, fix and commit. -``` - ---- - -## Summary - -| Task | Description | -|------|-------------| -| 1 | Create reduction workflow diagram (Typst → SVG) | -| 2 | Create module overview diagram | -| 3 | Create trait hierarchy diagram | -| 4 | Rewrite Getting Started | -| 5 | Update Architecture - Module Overview | -| 6 | Update Architecture - Fix outdated API | -| 7 | Add Contributing section | -| 8 | Build and verify | diff --git a/docs/plans/2026-02-13-hamiltonian-cycle-model.md b/docs/plans/2026-02-13-hamiltonian-cycle-model.md deleted file mode 100644 index ad0d7c6e0..000000000 --- a/docs/plans/2026-02-13-hamiltonian-cycle-model.md +++ /dev/null @@ -1,728 +0,0 @@ -# HamiltonianCycle Model Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Implement the HamiltonianCycle optimization problem model — given a weighted undirected graph, find a minimum-weight cycle visiting every vertex exactly once. - -**Architecture:** Follow the `MaximumMatching` model pattern (edge-based binary variables with edge weights). The struct `HamiltonianCycle<G, W>` stores a graph and edge weights. `dims()` returns `[2; num_edges]`. `evaluate()` checks if selected edges form a valid Hamiltonian cycle (degree-2 at every vertex, single connected cycle, exactly |V| edges) then returns the total weight. Direction is `Minimize`. - -**Tech Stack:** Rust, serde, num_traits, inventory (for schema registration) - ---- - -## Task 1: Write failing tests for HamiltonianCycle model - -**Files:** -- Create: `src/unit_tests/models/graph/hamiltonian_cycle.rs` - -**Step 1: Write the failing tests** - -Create the test file with comprehensive tests covering creation, evaluation, brute-force solving, and edge cases. These tests follow the patterns in `src/unit_tests/models/graph/maximum_matching.rs` and use the four example instances from the issue. - -```rust -use super::*; -use crate::solvers::BruteForce; -use crate::traits::{OptimizationProblem, Problem}; -use crate::types::{Direction, SolutionSize}; - -#[test] -fn test_hamiltonian_cycle_creation() { - // K4 complete graph - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - assert_eq!(problem.num_vertices(), 4); - assert_eq!(problem.num_edges(), 6); - assert_eq!(problem.dims().len(), 6); -} - -#[test] -fn test_hamiltonian_cycle_unweighted() { - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - assert!(!problem.is_weighted()); - assert_eq!(problem.num_vertices(), 5); - assert_eq!(problem.num_edges(), 5); -} - -#[test] -fn test_hamiltonian_cycle_weighted() { - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - assert!(problem.is_weighted()); -} - -#[test] -fn test_evaluate_valid_cycle() { - // C5 cycle graph with unit weights: all 5 edges form the only Hamiltonian cycle - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - // Select all edges → valid Hamiltonian cycle, cost = 5 - assert_eq!(problem.evaluate(&[1, 1, 1, 1, 1]), SolutionSize::Valid(5)); -} - -#[test] -fn test_evaluate_invalid_degree() { - // K4: select 3 edges incident to vertex 0 → degree > 2 at vertex 0 - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - // edges: 0-1, 0-2, 0-3, 1-2, 1-3, 2-3 - // Select first 3 edges (all incident to 0): degree(0)=3 → Invalid - assert_eq!(problem.evaluate(&[1, 1, 1, 0, 0, 0]), SolutionSize::Invalid); -} - -#[test] -fn test_evaluate_invalid_not_connected() { - // 6 vertices, two disjoint triangles: 0-1-2-0 and 3-4-5-3 - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 6, - vec![ - (0, 1), (1, 2), (0, 2), - (3, 4), (4, 5), (3, 5), - ], - ); - // Select all 6 edges: two disjoint cycles, not a single Hamiltonian cycle - assert_eq!(problem.evaluate(&[1, 1, 1, 1, 1, 1]), SolutionSize::Invalid); -} - -#[test] -fn test_evaluate_invalid_wrong_edge_count() { - // C5 with only 4 edges selected → not enough edges - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - assert_eq!(problem.evaluate(&[1, 1, 1, 1, 0]), SolutionSize::Invalid); -} - -#[test] -fn test_evaluate_no_edges_selected() { - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - assert_eq!(problem.evaluate(&[0, 0, 0, 0, 0]), SolutionSize::Invalid); -} - -#[test] -fn test_brute_force_k4() { - // Instance 1 from issue: K4 with weights - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - let solver = BruteForce::new(); - let solutions = solver.find_all_best(&problem); - assert!(!solutions.is_empty()); - // Optimal cycle: 0→1→3→2→0, cost = 10+25+30+15 = 80 - for sol in &solutions { - assert_eq!(problem.evaluate(sol), SolutionSize::Valid(80)); - } -} - -#[test] -fn test_brute_force_path_graph_no_solution() { - // Instance 2 from issue: path graph, no Hamiltonian cycle exists - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 4, - vec![(0, 1), (1, 2), (2, 3)], - ); - let solver = BruteForce::new(); - let solutions = solver.find_all_best(&problem); - assert!(solutions.is_empty()); -} - -#[test] -fn test_brute_force_c5_unique_solution() { - // Instance 3 from issue: C5 cycle graph, unique Hamiltonian cycle - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - let solver = BruteForce::new(); - let solutions = solver.find_all_best(&problem); - assert_eq!(solutions.len(), 1); - assert_eq!(solutions[0], vec![1, 1, 1, 1, 1]); - assert_eq!(problem.evaluate(&solutions[0]), SolutionSize::Valid(5)); -} - -#[test] -fn test_brute_force_bipartite_no_solution() { - // Instance 4 from issue: K_{2,3} bipartite, no Hamiltonian cycle - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 2), (0, 3), (0, 4), (1, 2), (1, 3), (1, 4)], - ); - let solver = BruteForce::new(); - let solutions = solver.find_all_best(&problem); - assert!(solutions.is_empty()); -} - -#[test] -fn test_direction() { - let problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 3, - vec![(0, 1), (1, 2), (0, 2)], - ); - assert_eq!(problem.direction(), Direction::Minimize); -} - -#[test] -fn test_problem_name() { - assert_eq!( - <HamiltonianCycle<SimpleGraph, i32> as Problem>::NAME, - "HamiltonianCycle" - ); -} - -#[test] -fn test_is_hamiltonian_cycle_function() { - // Triangle: selecting all 3 edges is a valid Hamiltonian cycle - assert!(is_hamiltonian_cycle( - 3, - &[(0, 1), (1, 2), (0, 2)], - &[true, true, true] - )); - // Path: not a cycle - assert!(!is_hamiltonian_cycle( - 3, - &[(0, 1), (1, 2)], - &[true, true] - )); -} - -#[test] -fn test_set_weights() { - let mut problem = HamiltonianCycle::<SimpleGraph, i32>::unweighted( - 3, - vec![(0, 1), (1, 2), (0, 2)], - ); - problem.set_weights(vec![5, 10, 15]); - assert_eq!(problem.weights(), vec![5, 10, 15]); -} - -#[test] -fn test_edges() { - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 3, - vec![(0, 1, 10), (1, 2, 20), (0, 2, 30)], - ); - let edges = problem.edges(); - assert_eq!(edges.len(), 3); -} - -#[test] -fn test_from_graph() { - let graph = SimpleGraph::new(3, vec![(0, 1), (1, 2), (0, 2)]); - let problem = HamiltonianCycle::<SimpleGraph, i32>::from_graph(graph, vec![10, 20, 30]); - assert_eq!(problem.num_vertices(), 3); - assert_eq!(problem.weights(), vec![10, 20, 30]); -} - -#[test] -fn test_from_graph_unit_weights() { - let graph = SimpleGraph::new(3, vec![(0, 1), (1, 2), (0, 2)]); - let problem = HamiltonianCycle::<SimpleGraph, i32>::from_graph_unit_weights(graph); - assert_eq!(problem.weights(), vec![1, 1, 1]); -} - -#[test] -fn test_brute_force_triangle_weighted() { - // Triangle with weights: unique Hamiltonian cycle using all edges - let problem = HamiltonianCycle::<SimpleGraph, i32>::new( - 3, - vec![(0, 1, 5), (1, 2, 10), (0, 2, 15)], - ); - let solver = BruteForce::new(); - let solutions = solver.find_all_best(&problem); - assert_eq!(solutions.len(), 1); - assert_eq!(solutions[0], vec![1, 1, 1]); - assert_eq!(problem.evaluate(&solutions[0]), SolutionSize::Valid(30)); -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test hamiltonian_cycle -- --no-run 2>&1 | head -20` -Expected: Compilation error — `HamiltonianCycle` type doesn't exist yet. - ---- - -## Task 2: Implement HamiltonianCycle model - -**Files:** -- Create: `src/models/graph/hamiltonian_cycle.rs` -- Modify: `src/models/graph/mod.rs` - -**Step 1: Write the implementation** - -Create `src/models/graph/hamiltonian_cycle.rs`: - -```rust -//! Hamiltonian Cycle problem implementation. -//! -//! The Hamiltonian Cycle problem asks for a minimum-weight cycle -//! that visits every vertex exactly once. - -use crate::registry::{FieldInfo, ProblemSchemaEntry}; -use crate::topology::{Graph, SimpleGraph}; -use crate::traits::{OptimizationProblem, Problem}; -use crate::types::{Direction, SolutionSize}; -use serde::{Deserialize, Serialize}; - -inventory::submit! { - ProblemSchemaEntry { - name: "HamiltonianCycle", - description: "Find minimum weight Hamiltonian cycle in a graph", - fields: &[ - FieldInfo { name: "graph", type_name: "G", description: "The underlying graph G=(V,E)" }, - FieldInfo { name: "edge_weights", type_name: "Vec<W>", description: "Edge weights w: E -> R" }, - ], - } -} - -/// The Hamiltonian Cycle problem. -/// -/// Given a weighted graph G = (V, E) with edge weights w_e, -/// find a cycle that visits every vertex exactly once and -/// minimizes the total edge weight. -/// -/// # Representation -/// -/// Each edge is assigned a binary variable: -/// - 0: edge is not in the cycle -/// - 1: edge is in the cycle -/// -/// A valid Hamiltonian cycle requires: -/// - Exactly 2 selected edges incident to each vertex (degree constraint) -/// - Selected edges form a single connected cycle (no subtours) -/// - Exactly |V| edges are selected -/// -/// # Type Parameters -/// -/// * `G` - The graph type (e.g., `SimpleGraph`, `GridGraph`) -/// * `W` - The weight type for edges (e.g., `i32`, `f64`) -#[derive(Debug, Clone, Serialize, Deserialize)] -pub struct HamiltonianCycle<G, W> { - /// The underlying graph. - graph: G, - /// Weights for each edge (in edge index order). - edge_weights: Vec<W>, -} - -impl<W: Clone + Default> HamiltonianCycle<SimpleGraph, W> { - /// Create a new HamiltonianCycle problem. - /// - /// # Arguments - /// * `num_vertices` - Number of vertices - /// * `edges` - List of weighted edges as (u, v, weight) triples - pub fn new(num_vertices: usize, edges: Vec<(usize, usize, W)>) -> Self { - let mut edge_list = Vec::new(); - let mut edge_weights = Vec::new(); - for (u, v, w) in edges { - edge_list.push((u, v)); - edge_weights.push(w); - } - let graph = SimpleGraph::new(num_vertices, edge_list); - Self { - graph, - edge_weights, - } - } - - /// Create a HamiltonianCycle problem with unit weights. - pub fn unweighted(num_vertices: usize, edges: Vec<(usize, usize)>) -> Self - where - W: From<i32>, - { - let edge_weights = vec![W::from(1); edges.len()]; - let graph = SimpleGraph::new(num_vertices, edges); - Self { - graph, - edge_weights, - } - } -} - -impl<G: Graph, W: Clone + Default> HamiltonianCycle<G, W> { - /// Create a HamiltonianCycle problem from a graph with given edge weights. - pub fn from_graph(graph: G, edge_weights: Vec<W>) -> Self { - assert_eq!( - edge_weights.len(), - graph.num_edges(), - "edge_weights length must match num_edges" - ); - Self { - graph, - edge_weights, - } - } - - /// Create a HamiltonianCycle problem from a graph with unit weights. - pub fn from_graph_unit_weights(graph: G) -> Self - where - W: From<i32>, - { - let edge_weights = vec![W::from(1); graph.num_edges()]; - Self { - graph, - edge_weights, - } - } - - /// Get a reference to the underlying graph. - pub fn graph(&self) -> &G { - &self.graph - } - - /// Get the number of vertices. - pub fn num_vertices(&self) -> usize { - self.graph.num_vertices() - } - - /// Get the number of edges. - pub fn num_edges(&self) -> usize { - self.graph.num_edges() - } - - /// Get all edges with their weights. - pub fn edges(&self) -> Vec<(usize, usize, W)> { - self.graph - .edges() - .into_iter() - .zip(self.edge_weights.iter().cloned()) - .map(|((u, v), w)| (u, v, w)) - .collect() - } - - /// Set new weights for the problem. - pub fn set_weights(&mut self, weights: Vec<W>) { - assert_eq!(weights.len(), self.graph.num_edges()); - self.edge_weights = weights; - } - - /// Get the weights for the problem. - pub fn weights(&self) -> Vec<W> { - self.edge_weights.clone() - } - - /// Check if the problem has non-uniform weights. - pub fn is_weighted(&self) -> bool - where - W: PartialEq, - { - if self.edge_weights.is_empty() { - return false; - } - let first = &self.edge_weights[0]; - !self.edge_weights.iter().all(|w| w == first) - } - - /// Check if a configuration forms a valid Hamiltonian cycle. - fn is_valid_hamiltonian_cycle(&self, config: &[usize]) -> bool { - let n = self.graph.num_vertices(); - let edges = self.graph.edges(); - - // Count selected edges and check degree constraint - let mut degree = vec![0usize; n]; - let mut selected_count = 0; - let mut first_selected_vertex = None; - - for (idx, &sel) in config.iter().enumerate() { - if sel == 1 { - if let Some(&(u, v)) = edges.get(idx) { - degree[u] += 1; - degree[v] += 1; - selected_count += 1; - if first_selected_vertex.is_none() { - first_selected_vertex = Some(u); - } - } - } - } - - // Must select exactly n edges - if selected_count != n { - return false; - } - - // Every vertex must have degree exactly 2 - if degree.iter().any(|&d| d != 2) { - return false; - } - - // Check connectivity: BFS/DFS on selected edges must reach all vertices - let first = match first_selected_vertex { - Some(v) => v, - None => return false, - }; - - // Build adjacency list from selected edges - let mut adj: Vec<Vec<usize>> = vec![vec![]; n]; - for (idx, &sel) in config.iter().enumerate() { - if sel == 1 { - if let Some(&(u, v)) = edges.get(idx) { - adj[u].push(v); - adj[v].push(u); - } - } - } - - // BFS from first vertex - let mut visited = vec![false; n]; - let mut queue = std::collections::VecDeque::new(); - visited[first] = true; - queue.push_back(first); - let mut visit_count = 1; - - while let Some(node) = queue.pop_front() { - for &neighbor in &adj[node] { - if !visited[neighbor] { - visited[neighbor] = true; - visit_count += 1; - queue.push_back(neighbor); - } - } - } - - visit_count == n - } -} - -impl<G, W> Problem for HamiltonianCycle<G, W> -where - G: Graph, - W: Clone - + Default - + PartialOrd - + num_traits::Num - + num_traits::Zero - + std::ops::AddAssign - + 'static, -{ - const NAME: &'static str = "HamiltonianCycle"; - type Metric = SolutionSize<W>; - - fn variant() -> Vec<(&'static str, &'static str)> { - vec![ - ("graph", crate::variant::short_type_name::<G>()), - ("weight", crate::variant::short_type_name::<W>()), - ] - } - - fn dims(&self) -> Vec<usize> { - vec![2; self.graph.num_edges()] - } - - fn evaluate(&self, config: &[usize]) -> SolutionSize<W> { - if !self.is_valid_hamiltonian_cycle(config) { - return SolutionSize::Invalid; - } - let mut total = W::zero(); - for (idx, &selected) in config.iter().enumerate() { - if selected == 1 { - if let Some(w) = self.edge_weights.get(idx) { - total += w.clone(); - } - } - } - SolutionSize::Valid(total) - } -} - -impl<G, W> OptimizationProblem for HamiltonianCycle<G, W> -where - G: Graph, - W: Clone - + Default - + PartialOrd - + num_traits::Num - + num_traits::Zero - + std::ops::AddAssign - + 'static, -{ - type Value = W; - - fn direction(&self) -> Direction { - Direction::Minimize - } -} - -/// Check if a selection of edges forms a valid Hamiltonian cycle. -/// -/// # Arguments -/// * `num_vertices` - Total number of vertices -/// * `edges` - List of edges as (u, v) pairs -/// * `selected` - Boolean slice indicating which edges are selected -pub fn is_hamiltonian_cycle( - num_vertices: usize, - edges: &[(usize, usize)], - selected: &[bool], -) -> bool { - if selected.len() != edges.len() { - return false; - } - - let n = num_vertices; - let mut degree = vec![0usize; n]; - let mut selected_count = 0; - let mut first_vertex = None; - - for (idx, &sel) in selected.iter().enumerate() { - if sel { - let (u, v) = edges[idx]; - if u >= n || v >= n { - return false; - } - degree[u] += 1; - degree[v] += 1; - selected_count += 1; - if first_vertex.is_none() { - first_vertex = Some(u); - } - } - } - - if selected_count != n { - return false; - } - - if degree.iter().any(|&d| d != 2) { - return false; - } - - let first = match first_vertex { - Some(v) => v, - None => return false, - }; - - let mut adj: Vec<Vec<usize>> = vec![vec![]; n]; - for (idx, &sel) in selected.iter().enumerate() { - if sel { - let (u, v) = edges[idx]; - adj[u].push(v); - adj[v].push(u); - } - } - - let mut visited = vec![false; n]; - let mut queue = std::collections::VecDeque::new(); - visited[first] = true; - queue.push_back(first); - let mut visit_count = 1; - - while let Some(node) = queue.pop_front() { - for &neighbor in &adj[node] { - if !visited[neighbor] { - visited[neighbor] = true; - visit_count += 1; - queue.push_back(neighbor); - } - } - } - - visit_count == n -} - -#[cfg(test)] -#[path = "../../unit_tests/models/graph/hamiltonian_cycle.rs"] -mod tests; -``` - -**Step 2: Register in mod.rs** - -Add to `src/models/graph/mod.rs`: -- Add `mod hamiltonian_cycle;` line -- Add `pub use hamiltonian_cycle::{is_hamiltonian_cycle, HamiltonianCycle};` -- Add to module doc comment: `//! - [`HamiltonianCycle`]: Minimum weight Hamiltonian cycle` - -**Step 3: Run tests to verify they pass** - -Run: `cargo test hamiltonian_cycle -v` -Expected: All tests pass. - -**Step 4: Run full check** - -Run: `make test clippy` -Expected: All tests pass, no clippy warnings. - -**Step 5: Commit** - -```bash -git add src/models/graph/hamiltonian_cycle.rs src/models/graph/mod.rs src/unit_tests/models/graph/hamiltonian_cycle.rs -git commit -m "feat: add HamiltonianCycle model (#47)" -``` - ---- - -## Task 3: Add HamiltonianCycle to paper - -**Files:** -- Modify: `docs/paper/reductions.typ` - -**Step 1: Add display-name entry** - -Add to the `display-name` dictionary: -```typst -"HamiltonianCycle": [Hamiltonian Cycle], -``` - -**Step 2: Add problem-def entry** - -Add a `#problem-def` block after the existing graph problems (e.g., after `MaximumMatching`): - -```typst -#problem-def("HamiltonianCycle")[ - Given an undirected graph $G=(V,E)$ with edge weights $w: E -> RR$, find a cycle visiting every vertex exactly once that minimizes $sum_(e in C) w(e)$. -] -``` - -**Step 3: Regenerate schema** - -Run: `make export-schemas` - -**Step 4: Verify paper builds** - -Run: `make paper` (if Typst is available) or just verify no JSON errors. - -**Step 5: Commit** - -```bash -git add docs/paper/reductions.typ docs/src/reductions/problem_schemas.json -git commit -m "docs: add HamiltonianCycle definition to paper (#47)" -``` - ---- - -## Task 4: Final verification - -**Step 1: Run full test suite** - -Run: `make check` -Expected: fmt, clippy, and all tests pass. - -**Step 2: Check coverage (if applicable)** - -Run: `make coverage` -Expected: >95% coverage for new code. diff --git a/docs/plans/2026-02-13-natural-variant-reductions-design.md b/docs/plans/2026-02-13-natural-variant-reductions-design.md deleted file mode 100644 index d2a40c26d..000000000 --- a/docs/plans/2026-02-13-natural-variant-reductions-design.md +++ /dev/null @@ -1,125 +0,0 @@ -# Natural Variant Reduction Edges - -## Problem - -The reduction graph export (`reduction_graph.json`) does not include "natural" reductions between variant nodes of the same problem. For example, `MaximumIndependentSet/GridGraph` should naturally reduce to `MaximumIndependentSet/SimpleGraph` because a GridGraph is a SimpleGraph. These edges should be auto-generated based on type hierarchies, not manually coded as `ReduceTo` impls. - -## Design - -### 1. Fix Graph Type Hierarchy (`src/graph_types.rs`) - -**Bug fix**: Remove incorrect `UnitDiskGraph => PlanarGraph` relationship. - -**Corrected hierarchy** (with all transitive relationships for compile-time trait bounds): - -``` -HyperGraph (most general) -└── SimpleGraph - ├── PlanarGraph - ├── BipartiteGraph - └── UnitDiskGraph - └── GridGraph -``` - -Declarations: -```rust -declare_graph_subtype!(GridGraph => UnitDiskGraph); -declare_graph_subtype!(GridGraph => SimpleGraph); // transitive -declare_graph_subtype!(GridGraph => HyperGraph); // transitive -declare_graph_subtype!(UnitDiskGraph => SimpleGraph); -declare_graph_subtype!(UnitDiskGraph => HyperGraph); // transitive -declare_graph_subtype!(PlanarGraph => SimpleGraph); -declare_graph_subtype!(PlanarGraph => HyperGraph); // transitive -declare_graph_subtype!(BipartiteGraph => SimpleGraph); -declare_graph_subtype!(BipartiteGraph => HyperGraph); // transitive -declare_graph_subtype!(SimpleGraph => HyperGraph); -``` - -Add `GridGraph` and `HyperGraph` as `GraphMarker` types (marker structs + trait impls). - -### 2. Weight Type Hierarchy (new) - -Add `WeightSubtypeEntry` parallel to `GraphSubtypeEntry`: - -```rust -pub struct WeightSubtypeEntry { - pub subtype: &'static str, - pub supertype: &'static str, -} -inventory::collect!(WeightSubtypeEntry); - -macro_rules! declare_weight_subtype { - ($sub:expr => $sup:expr) => { - inventory::submit! { - WeightSubtypeEntry { - subtype: $sub, - supertype: $sup, - } - } - }; -} - -declare_weight_subtype!("Unweighted" => "i32"); -declare_weight_subtype!("Unweighted" => "f64"); // transitive -declare_weight_subtype!("i32" => "f64"); -``` - -`ReductionGraph` builds `weight_hierarchy` using the same transitive closure algorithm as `graph_hierarchy`. - -### 3. Concrete Variant Registration (new) - -Add `ConcreteVariantEntry` to register problem+variant combinations that exist as concrete types but have no explicit reduction rules: - -```rust -pub struct ConcreteVariantEntry { - pub name: &'static str, - pub variant: &'static [(&'static str, &'static str)], -} -inventory::collect!(ConcreteVariantEntry); -``` - -Register concrete variants in `register_types!` (or a new `register_variants!` section): - -```rust -// For each problem that supports non-SimpleGraph types: -submit_variant!("MaximumIndependentSet", &[("graph", "GridGraph"), ("weight", "Unweighted")]); -submit_variant!("MaximumIndependentSet", &[("graph", "UnitDiskGraph"), ("weight", "Unweighted")]); -submit_variant!("SpinGlass", &[("graph", "GridGraph"), ("weight", "f64")]); -// etc. -``` - -### 4. Auto-generate Natural Edges in `to_json()` - -In `ReductionGraph::to_json()`, after collecting all nodes from reduction entries and concrete variant entries, add: - -``` -For every pair of nodes (A, B) with the same problem name: - If A != B AND for EVERY variant field: - field=="graph": A.graph is_subtype_of B.graph (using graph_hierarchy) - field=="weight": A.weight is_subtype_of B.weight (using weight_hierarchy) - Then emit a natural edge A -> B with: - - overhead: identity mapping (each output field = same input field) - - doc_path: "" (or a special marker like "natural") -``` - -### 5. Variant Comparison Rule - -A variant A is "more restrictive" than B when ALL fields satisfy the subtype relationship: -- `graph`: checked against `graph_hierarchy` -- `weight`: checked against `weight_hierarchy` -- Other fields (e.g., `k`): must be equal (no hierarchy defined) - -### 6. Files Changed - -| File | Change | -|------|--------| -| `src/graph_types.rs` | Fix hierarchy, add GridGraph/HyperGraph markers, add WeightSubtypeEntry | -| `src/rules/graph.rs` | Build weight_hierarchy, generate natural edges in to_json(), collect ConcreteVariantEntry nodes | -| `src/rules/registry.rs` | Add ConcreteVariantEntry struct | -| `examples/export_graph.rs` | No changes needed (uses ReductionGraph::to_json()) | - -### 7. Non-Goals - -- No new `ReduceTo` impls for natural reductions (these are visualization-only edges in the JSON) -- No changes to runtime path finding (it already uses `rule_applicable` with graph hierarchy) -- No changes to the paper/Typst documentation initially diff --git a/docs/plans/2026-02-13-natural-variant-reductions-impl.md b/docs/plans/2026-02-13-natural-variant-reductions-impl.md deleted file mode 100644 index fc9d885e9..000000000 --- a/docs/plans/2026-02-13-natural-variant-reductions-impl.md +++ /dev/null @@ -1,617 +0,0 @@ -# Natural Variant Reduction Edges — Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Auto-generate natural reduction edges in the JSON export between variant nodes of the same problem type when all variant fields of the source are transitively more restrictive than the target. - -**Architecture:** Fix the graph type hierarchy (add GridGraph, HyperGraph; remove wrong UnitDiskGraph=>PlanarGraph). Add a parallel weight type hierarchy. Register concrete variant nodes. In `to_json()`, detect transitive reducibility between same-name nodes and emit natural edges. - -**Tech Stack:** Rust, inventory crate for compile-time collection, serde_json for export. - ---- - -### Task 1: Fix Graph Type Hierarchy - -**Files:** -- Modify: `src/graph_types.rs` -- Modify: `src/unit_tests/graph_types.rs` - -**Step 1: Write failing tests for the corrected hierarchy** - -Add to `src/unit_tests/graph_types.rs`: - -```rust -#[test] -fn test_gridgraph_subtypes() { - fn assert_subtype<A: GraphSubtype<B>, B: GraphMarker>() {} - - assert_subtype::<GridGraph, UnitDiskGraph>(); - assert_subtype::<GridGraph, SimpleGraph>(); - assert_subtype::<GridGraph, HyperGraph>(); -} - -#[test] -fn test_hypergraph_subtypes() { - fn assert_subtype<A: GraphSubtype<B>, B: GraphMarker>() {} - - assert_subtype::<SimpleGraph, HyperGraph>(); - assert_subtype::<PlanarGraph, HyperGraph>(); - assert_subtype::<UnitDiskGraph, HyperGraph>(); - assert_subtype::<BipartiteGraph, HyperGraph>(); - assert_subtype::<GridGraph, HyperGraph>(); -} - -#[test] -fn test_gridgraph_entries_registered() { - let entries: Vec<_> = inventory::iter::<GraphSubtypeEntry>().collect(); - assert!(entries - .iter() - .any(|e| e.subtype == "GridGraph" && e.supertype == "UnitDiskGraph")); -} - -#[test] -fn test_hypergraph_entries_registered() { - let entries: Vec<_> = inventory::iter::<GraphSubtypeEntry>().collect(); - assert!(entries - .iter() - .any(|e| e.subtype == "SimpleGraph" && e.supertype == "HyperGraph")); -} -``` - -Also update existing `test_declared_subtypes` to remove the `UnitDiskGraph, PlanarGraph` assertion and add new ones. Update `test_unit_disk_to_planar_registered` to assert it is NOT registered. - -**Step 2: Run tests to verify they fail** - -Run: `cargo test --lib graph_types` -Expected: FAIL — `GridGraph` and `HyperGraph` not found as marker types. - -**Step 3: Implement the hierarchy changes** - -In `src/graph_types.rs`: - -1. Add `GridGraph` and `HyperGraph` marker types: -```rust -/// Grid graph - vertices on a grid, edges to neighbors. -#[derive(Debug, Clone, Copy, Default)] -pub struct GridGraph; - -impl GraphMarker for GridGraph {} - -/// Hypergraph - most general graph type. Edges can connect any number of vertices. -#[derive(Debug, Clone, Copy, Default)] -pub struct HyperGraph; - -impl GraphMarker for HyperGraph {} -``` - -2. Replace the hierarchy declarations: -```rust -// Corrected hierarchy: -// HyperGraph (most general) -// └── SimpleGraph -// ├── PlanarGraph -// ├── BipartiteGraph -// └── UnitDiskGraph -// └── GridGraph -declare_graph_subtype!(GridGraph => UnitDiskGraph); -declare_graph_subtype!(GridGraph => SimpleGraph); // transitive -declare_graph_subtype!(GridGraph => HyperGraph); // transitive -declare_graph_subtype!(UnitDiskGraph => SimpleGraph); -declare_graph_subtype!(UnitDiskGraph => HyperGraph); // transitive -declare_graph_subtype!(PlanarGraph => SimpleGraph); -declare_graph_subtype!(PlanarGraph => HyperGraph); // transitive -declare_graph_subtype!(BipartiteGraph => SimpleGraph); -declare_graph_subtype!(BipartiteGraph => HyperGraph); // transitive -declare_graph_subtype!(SimpleGraph => HyperGraph); -``` - -**Step 4: Run tests to verify they pass** - -Run: `cargo test --lib graph_types` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/graph_types.rs src/unit_tests/graph_types.rs -git commit -m "fix: correct graph type hierarchy, add GridGraph and HyperGraph markers" -``` - ---- - -### Task 2: Add Weight Type Hierarchy - -**Files:** -- Modify: `src/graph_types.rs` -- Modify: `src/unit_tests/graph_types.rs` -- Modify: `src/rules/graph.rs` (build weight_hierarchy) -- Modify: `src/unit_tests/rules/graph.rs` - -**Step 1: Write failing tests** - -In `src/unit_tests/graph_types.rs`: -```rust -#[test] -fn test_weight_subtype_entries_registered() { - let entries: Vec<_> = inventory::iter::<WeightSubtypeEntry>().collect(); - assert!(entries - .iter() - .any(|e| e.subtype == "Unweighted" && e.supertype == "i32")); - assert!(entries - .iter() - .any(|e| e.subtype == "i32" && e.supertype == "f64")); - assert!(entries - .iter() - .any(|e| e.subtype == "Unweighted" && e.supertype == "f64")); -} -``` - -In `src/unit_tests/rules/graph.rs`: -```rust -#[test] -fn test_weight_hierarchy_built() { - let graph = ReductionGraph::new(); - let hierarchy = graph.weight_hierarchy(); - assert!( - hierarchy - .get("Unweighted") - .map(|s| s.contains("i32")) - .unwrap_or(false), - "Unweighted should have i32 as supertype" - ); - assert!( - hierarchy - .get("i32") - .map(|s| s.contains("f64")) - .unwrap_or(false), - "i32 should have f64 as supertype" - ); - assert!( - hierarchy - .get("Unweighted") - .map(|s| s.contains("f64")) - .unwrap_or(false), - "Unweighted should transitively have f64 as supertype" - ); -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test --lib weight_subtype weight_hierarchy` -Expected: FAIL — `WeightSubtypeEntry` does not exist. - -**Step 3: Implement weight hierarchy** - -In `src/graph_types.rs`, add: -```rust -/// Runtime registration of weight subtype relationships. -pub struct WeightSubtypeEntry { - pub subtype: &'static str, - pub supertype: &'static str, -} - -inventory::collect!(WeightSubtypeEntry); - -/// Macro to declare weight subtype relationships (runtime only, no compile-time trait). -#[macro_export] -macro_rules! declare_weight_subtype { - ($sub:expr => $sup:expr) => { - ::inventory::submit! { - $crate::graph_types::WeightSubtypeEntry { - subtype: $sub, - supertype: $sup, - } - } - }; -} - -declare_weight_subtype!("Unweighted" => "i32"); -declare_weight_subtype!("Unweighted" => "f64"); // transitive -declare_weight_subtype!("i32" => "f64"); -``` - -In `src/rules/graph.rs`, add `weight_hierarchy` field to `ReductionGraph` and build it in `new()`: -```rust -// In ReductionGraph struct: -weight_hierarchy: HashMap<&'static str, HashSet<&'static str>>, - -// In new(): -let weight_hierarchy = Self::build_weight_hierarchy(); - -// New method: -fn build_weight_hierarchy() -> HashMap<&'static str, HashSet<&'static str>> { - let mut supertypes: HashMap<&'static str, HashSet<&'static str>> = HashMap::new(); - for entry in inventory::iter::<WeightSubtypeEntry> { - supertypes.entry(entry.subtype).or_default().insert(entry.supertype); - } - // Same transitive closure as build_graph_hierarchy - loop { - let mut changed = false; - let types: Vec<_> = supertypes.keys().copied().collect(); - for sub in &types { - let current: Vec<_> = supertypes.get(sub).map(|s| s.iter().copied().collect()).unwrap_or_default(); - for sup in current { - if let Some(sup_supers) = supertypes.get(sup).cloned() { - for ss in sup_supers { - if supertypes.entry(sub).or_default().insert(ss) { - changed = true; - } - } - } - } - } - if !changed { break; } - } - supertypes -} - -// Public accessor: -pub fn weight_hierarchy(&self) -> &HashMap<&'static str, HashSet<&'static str>> { - &self.weight_hierarchy -} - -// Weight subtype check: -pub fn is_weight_subtype(&self, sub: &str, sup: &str) -> bool { - sub == sup - || self.weight_hierarchy.get(sub).map(|s| s.contains(sup)).unwrap_or(false) -} -``` - -**Step 4: Run tests to verify they pass** - -Run: `cargo test --lib weight_subtype weight_hierarchy` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/graph_types.rs src/unit_tests/graph_types.rs src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: add weight type hierarchy (Unweighted => i32 => f64)" -``` - ---- - -### Task 3: Add Concrete Variant Registration - -**Files:** -- Modify: `src/rules/registry.rs` -- Modify: `src/rules/graph.rs` (register_types and to_json) -- Modify: `src/unit_tests/rules/graph.rs` - -**Step 1: Write failing test** - -In `src/unit_tests/rules/graph.rs`: -```rust -#[test] -fn test_concrete_variant_nodes_in_json() { - let graph = ReductionGraph::new(); - let json = graph.to_json(); - - // GridGraph variants should appear as nodes - let gridgraph_node = json.nodes.iter().any(|n| { - n.name == "MaximumIndependentSet" - && n.variant.get("graph") == Some(&"GridGraph".to_string()) - }); - assert!(gridgraph_node, "MIS/GridGraph node should exist"); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test --lib test_concrete_variant_nodes` -Expected: FAIL — no GridGraph variant node exists. - -**Step 3: Implement ConcreteVariantEntry and registrations** - -In `src/rules/registry.rs`, add: -```rust -/// A registered concrete problem variant (for JSON export nodes). -/// Variants registered here appear as nodes even without explicit reduction rules. -pub struct ConcreteVariantEntry { - pub name: &'static str, - pub variant: &'static [(&'static str, &'static str)], -} - -inventory::collect!(ConcreteVariantEntry); -``` - -In `src/rules/graph.rs`, in `register_types()`, add variant registrations after the existing `register!` block. Use `inventory::submit!` for each concrete variant that should appear: - -```rust -fn register_variants() { - // These are registered via inventory::submit! at module level, not inside a function. - // See the submit! blocks below register_types. -} -``` - -Actually, add the variant submissions at module level in `src/rules/graph.rs` (outside functions): - -```rust -// Register concrete variants for graph problems that support non-SimpleGraph types. -// These generate nodes in the JSON export. -inventory::submit! { ConcreteVariantEntry { name: "MaximumIndependentSet", variant: &[("graph", "GridGraph"), ("weight", "Unweighted")] } } -inventory::submit! { ConcreteVariantEntry { name: "MaximumIndependentSet", variant: &[("graph", "UnitDiskGraph"), ("weight", "Unweighted")] } } -inventory::submit! { ConcreteVariantEntry { name: "MaxCut", variant: &[("graph", "GridGraph"), ("weight", "Unweighted")] } } -inventory::submit! { ConcreteVariantEntry { name: "SpinGlass", variant: &[("graph", "GridGraph"), ("weight", "f64")] } } -``` - -In `to_json()`, collect nodes from `ConcreteVariantEntry` in addition to `ReductionEntry`: - -```rust -// After collecting from ReductionEntry, also collect from ConcreteVariantEntry -for entry in inventory::iter::<ConcreteVariantEntry> { - node_set.insert(( - entry.name.to_string(), - Self::variant_to_map(entry.variant), - )); -} -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test --lib test_concrete_variant_nodes` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/rules/registry.rs src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: add ConcreteVariantEntry for non-SimpleGraph variant nodes" -``` - ---- - -### Task 4: Auto-Generate Natural Edges in `to_json()` - -**Files:** -- Modify: `src/rules/graph.rs` (to_json method) -- Modify: `src/unit_tests/rules/graph.rs` - -**Step 1: Write failing tests for natural edges** - -In `src/unit_tests/rules/graph.rs`: -```rust -#[test] -fn test_natural_edge_graph_relaxation() { - let graph = ReductionGraph::new(); - let json = graph.to_json(); - - // MIS/GridGraph -> MIS/SimpleGraph should exist (graph type relaxation) - let has_edge = json.edges.iter().any(|e| { - e.source.name == "MaximumIndependentSet" - && e.target.name == "MaximumIndependentSet" - && e.source.variant.get("graph") == Some(&"GridGraph".to_string()) - && e.target.variant.get("graph") == Some(&"SimpleGraph".to_string()) - && e.source.variant.get("weight") == e.target.variant.get("weight") - }); - assert!(has_edge, "Natural edge MIS/GridGraph -> MIS/SimpleGraph should exist"); -} - -#[test] -fn test_natural_edge_weight_promotion() { - let graph = ReductionGraph::new(); - let json = graph.to_json(); - - // MIS{SimpleGraph, Unweighted} -> MIS{SimpleGraph, i32} should exist - let has_edge = json.edges.iter().any(|e| { - e.source.name == "MaximumIndependentSet" - && e.target.name == "MaximumIndependentSet" - && e.source.variant.get("graph") == Some(&"SimpleGraph".to_string()) - && e.target.variant.get("graph") == Some(&"SimpleGraph".to_string()) - && e.source.variant.get("weight") == Some(&"Unweighted".to_string()) - && e.target.variant.get("weight") == Some(&"i32".to_string()) - }); - assert!(has_edge, "Natural edge MIS/Unweighted -> MIS/i32 should exist"); -} - -#[test] -fn test_no_natural_edge_wrong_direction() { - let graph = ReductionGraph::new(); - let json = graph.to_json(); - - // MIS/SimpleGraph -> MIS/GridGraph should NOT exist (wrong direction) - let has_edge = json.edges.iter().any(|e| { - e.source.name == "MaximumIndependentSet" - && e.target.name == "MaximumIndependentSet" - && e.source.variant.get("graph") == Some(&"SimpleGraph".to_string()) - && e.target.variant.get("graph") == Some(&"GridGraph".to_string()) - }); - assert!(!has_edge, "Should NOT have MIS/SimpleGraph -> MIS/GridGraph"); -} - -#[test] -fn test_no_natural_self_edge() { - let graph = ReductionGraph::new(); - let json = graph.to_json(); - - // No self-edges (same node to same node) - for edge in &json.edges { - assert!( - edge.source != edge.target, - "Should not have self-edge: {} {:?}", - edge.source.name, - edge.source.variant - ); - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test --lib test_natural_edge` -Expected: FAIL — no natural edges exist yet. - -**Step 3: Implement natural edge generation** - -In `src/rules/graph.rs`, add a method to check if a variant is transitively reducible to another: - -```rust -/// Check if variant A is strictly more restrictive than variant B (same problem name). -/// Returns true if every field of A is a subtype of (or equal to) the corresponding field in B, -/// and at least one field is strictly more restrictive. -fn is_variant_reducible( - &self, - a: &std::collections::BTreeMap<String, String>, - b: &std::collections::BTreeMap<String, String>, -) -> bool { - if a == b { - return false; // No self-reduction - } - - let mut all_compatible = true; - let mut any_strict = false; - - // Check all fields in both variants - let all_keys: std::collections::BTreeSet<_> = a.keys().chain(b.keys()).collect(); - - for key in all_keys { - let a_val = a.get(key.as_str()).map(|s| s.as_str()).unwrap_or(""); - let b_val = b.get(key.as_str()).map(|s| s.as_str()).unwrap_or(""); - - if a_val == b_val { - continue; // Equal on this field - } - - // Check subtype relationship based on field type - let is_sub = match key.as_str() { - "graph" => self.is_graph_subtype(a_val, b_val), - "weight" => self.is_weight_subtype(a_val, b_val), - _ => false, // Unknown fields must be equal - }; - - if is_sub { - any_strict = true; - } else { - all_compatible = false; - break; - } - } - - all_compatible && any_strict -} -``` - -In `to_json()`, after building `edges`, add natural edge generation: - -```rust -// Auto-generate natural edges between same-name variants -// Group nodes by name -let mut nodes_by_name: HashMap<&str, Vec<&std::collections::BTreeMap<String, String>>> = HashMap::new(); -for (name, variant) in &node_set { - if !variant.is_empty() { - nodes_by_name.entry(name.as_str()).or_default().push(variant); - } -} - -// For each pair of same-name nodes, check transitive reducibility -for (name, variants) in &nodes_by_name { - for a in variants { - for b in variants { - if self.is_variant_reducible(a, b) { - let src_ref = VariantRef { name: name.to_string(), variant: (*a).clone() }; - let dst_ref = VariantRef { name: name.to_string(), variant: (*b).clone() }; - let key = (src_ref.clone(), dst_ref.clone()); - if edge_set.insert(key) { - // Identity overhead: each output field = same input field, p(x) = x - let overhead: Vec<OverheadFieldJson> = a.iter() - .filter(|(k, _)| *k != "graph" && *k != "weight") - .map(|(k, _)| OverheadFieldJson { field: k.clone(), formula: k.clone() }) - .collect(); - // For graph problems, carry through standard size fields - let overhead = if overhead.is_empty() { - // Infer from existing edges of same problem - // Fallback: emit common fields as identity - vec![] // Will be populated from source node's known fields - } else { - overhead - }; - edges.push(EdgeJson { - source: src_ref, - target: dst_ref, - overhead, - doc_path: String::new(), // No module path — natural reduction - }); - } - } - } - } -} - -// Re-sort edges after adding natural ones -edges.sort_by(|a, b| { ... }); -``` - -**Step 4: Run tests to verify they pass** - -Run: `cargo test --lib test_natural_edge` -Expected: PASS - -**Step 5: Run full test suite** - -Run: `cargo test` -Expected: PASS (existing tests may need minor adjustments for changed node/edge counts) - -**Step 6: Commit** - -```bash -git add src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: auto-generate natural variant reduction edges in JSON export" -``` - ---- - -### Task 5: Update Existing Tests and Regenerate Graph - -**Files:** -- Modify: `src/unit_tests/rules/graph.rs` (fix any broken count assertions) -- Regenerate: `docs/book/reductions/reduction_graph.json` - -**Step 1: Run full test suite and fix any failures** - -Run: `cargo test` - -Likely fixes: -- `test_to_json`: node count assertion `json.nodes.len() >= 10` may need updating -- `test_to_json`: edge count assertion `json.edges.len() >= 10` may need updating -- `test_graph_hierarchy_built`: remove assertion about `UnitDiskGraph` having `PlanarGraph` as supertype (if present) -- `test_is_graph_subtype_direct`: remove `UnitDiskGraph, PlanarGraph` assertion -- `test_is_graph_subtype_transitive`: update comment and assertions -- `test_subtype_entries_registered`: count assertion `entries.len() >= 4` may need updating - -**Step 2: Regenerate reduction graph JSON** - -Run: `cargo run --example export_graph` - -**Step 3: Verify the generated JSON contains natural edges** - -Run: `cargo test` -Expected: PASS - -**Step 4: Commit** - -```bash -git add src/unit_tests/rules/graph.rs docs/book/reductions/reduction_graph.json -git commit -m "fix: update tests for corrected hierarchy, regenerate reduction graph" -``` - ---- - -### Task 6: Run clippy and full verification - -**Files:** None (verification only) - -**Step 1: Run clippy** - -Run: `cargo clippy -- -D warnings` -Expected: PASS - -**Step 2: Run full test suite** - -Run: `cargo test` -Expected: PASS - -**Step 3: Run fmt check** - -Run: `cargo fmt --check` -Expected: PASS (or fix formatting) diff --git a/docs/plans/2026-02-13-reduction-macro-redesign-design.md b/docs/plans/2026-02-13-reduction-macro-redesign-design.md deleted file mode 100644 index 467bb991c..000000000 --- a/docs/plans/2026-02-13-reduction-macro-redesign-design.md +++ /dev/null @@ -1,164 +0,0 @@ -# Reduction Macro Redesign: Dynamic Variant Extraction - -## Problem - -The `#[reduction]` proc macro infers variant information (graph type, weight type) from type parameters using heuristic pattern matching. This breaks when problem types don't fit the expected `Problem<G, W>` pattern: - -- `KColoring<K, G>` — const generic `K` parsed as type param, `SimpleGraph` misidentified as weight -- `CircuitSAT` — no type params, defaults work but are fragile -- `KSatisfiability<K>` — literal `3` works, generic `K` doesn't -- `QUBO<W>` — single-param, needs special-case detection - -The macro duplicates logic that `Problem::variant()` already handles correctly, leading to bugs and maintenance burden. - -## Solution: Dynamic Function Pointers - -Replace static variant inference with function pointers that call `Problem::variant()` at runtime. - -### Changes - -#### 1. `ReductionEntry` (src/rules/registry.rs) - -Replace static `&'static [(&'static str, &'static str)]` fields with `fn()` pointers: - -```rust -// Before: -pub struct ReductionEntry { - pub source_name: &'static str, - pub target_name: &'static str, - pub source_variant: &'static [(&'static str, &'static str)], - pub target_variant: &'static [(&'static str, &'static str)], - pub overhead_fn: fn() -> ReductionOverhead, - pub module_path: &'static str, -} - -// After: -pub struct ReductionEntry { - pub source_name: &'static str, - pub target_name: &'static str, - pub source_variant_fn: fn() -> Vec<(&'static str, &'static str)>, - pub target_variant_fn: fn() -> Vec<(&'static str, &'static str)>, - pub overhead_fn: fn() -> ReductionOverhead, - pub module_path: &'static str, -} -``` - -#### 2. Macro Simplification (problemreductions-macros/src/lib.rs) - -**Remove** (~120 lines): -- `extract_graph_type()` — graph type inference from first param -- `extract_weight_type()` — weight type inference from second param -- `is_weight_type()` — weight type name list -- `get_weight_name()` — type-to-name conversion -- `source_graph`, `target_graph`, `source_weighted`, `target_weighted` attributes from `ReductionAttrs` - -**Keep**: -- `extract_type_name()` — still needed for `source_name`/`target_name` -- `extract_target_from_trait()` — still needed to get target type from `ReduceTo<T>` -- `overhead` attribute parsing -- `ReductionAttrs` (with only `overhead` remaining) - -**Add**: -- Const generic detection: scan impl generics for `const K: usize` patterns -- Type substitution: replace const generic idents with `usize::MAX` in variant_fn calls - -**Generated code**: - -For `impl ReduceTo<QUBO<f64>> for MaximumIndependentSet<SimpleGraph, i32>`: -```rust -inventory::submit! { - crate::rules::registry::ReductionEntry { - source_name: "MaximumIndependentSet", - target_name: "QUBO", - source_variant_fn: || <MaximumIndependentSet<SimpleGraph, i32> as Problem>::variant(), - target_variant_fn: || <QUBO<f64> as Problem>::variant(), - overhead_fn: || { /* overhead */ }, - module_path: module_path!(), - } -} -``` - -For `impl<const K: usize> ReduceTo<QUBO<f64>> for KColoring<K, SimpleGraph>`: -```rust -// K is detected as const generic → substituted with usize::MAX -// const_usize_str::<{usize::MAX}>() returns "N" → variant becomes ("k", "N") -inventory::submit! { - crate::rules::registry::ReductionEntry { - source_name: "KColoring", - target_name: "QUBO", - source_variant_fn: || <KColoring<{usize::MAX}, SimpleGraph> as crate::traits::Problem>::variant(), - target_variant_fn: || <QUBO<f64> as crate::traits::Problem>::variant(), - overhead_fn: || { /* overhead */ }, - module_path: module_path!(), - } -} -``` - -For `impl<W: NumericSize> ReduceTo<MinimumVertexCover<SimpleGraph, W>> for MaximumIndependentSet<SimpleGraph, W>`: -```rust -// W is a type generic (not const) → use Unweighted as default representative -inventory::submit! { - crate::rules::registry::ReductionEntry { - source_name: "MaximumIndependentSet", - target_name: "MinimumVertexCover", - source_variant_fn: || <MaximumIndependentSet<SimpleGraph, Unweighted> as crate::traits::Problem>::variant(), - target_variant_fn: || <MinimumVertexCover<SimpleGraph, Unweighted> as crate::traits::Problem>::variant(), - overhead_fn: || { /* overhead */ }, - module_path: module_path!(), - } -} -``` - -#### 3. Manual `inventory::submit!` Updates - -Update all 3 manual registrations to use new field names: -- `coloring_ilp.rs` — may become auto-generated (macro can now handle `KColoring<K, G>`) -- `factoring_ilp.rs` — keep manual (complex overhead logic) -- `sat_ksat.rs` — may become auto-generated - -#### 4. Graph Builder Updates - -Update `export_graph` example and any code that reads `ReductionEntry` to call the fn pointers: -```rust -// Before: -let source_variant = entry.source_variant; - -// After: -let source_variant = (entry.source_variant_fn)(); -``` - -### Const Generic Substitution Rules - -When the macro encounters `impl<const K: usize>`, it: -1. Identifies `K` as a const generic parameter -2. In the generated variant_fn calls, replaces `K` with `usize::MAX` -3. `const_usize_str::<{usize::MAX}>()` → `"N"`, meaning "any K" - -When the impl uses a literal const (e.g., `KSatisfiability<2>`): -- No substitution needed — `2` is already concrete -- `const_usize_str::<2>()` → `"2"`, preserving the specific value - -### Type Generic Substitution Rules - -When the macro encounters `impl<W: SomeBound>`, it: -1. Identifies `W` as a type generic parameter -2. In the generated variant_fn calls, replaces `W` with `Unweighted` -3. `Unweighted::variant_name()` or `short_type_name::<Unweighted>()` → `"Unweighted"` - -This produces the "base" variant for generic reductions. - -### What Doesn't Change - -- `Problem::variant()` signature and implementations -- `ReduceTo` and `ReductionResult` traits -- Existing reduction rule logic (only the generated registration code changes) -- `variant()` implementations on all problem types -- `const_usize_str()` utility - -### Benefits - -- **Zero inference bugs** — variant info comes from the authoritative `Problem::variant()` implementation -- **Simpler macro** — ~120 lines of inference logic removed -- **No new traits** — no `VariantName` trait, no changes to `Problem` -- **Handles all type patterns** — `<G, W>`, `<K, G>`, `<W>`, no params, all work -- **Const generics handled automatically** — `usize::MAX` sentinel produces `"N"` diff --git a/docs/plans/2026-02-13-travelingsalesman-ilp.md b/docs/plans/2026-02-13-travelingsalesman-ilp.md deleted file mode 100644 index aa30fa7d9..000000000 --- a/docs/plans/2026-02-13-travelingsalesman-ilp.md +++ /dev/null @@ -1,696 +0,0 @@ -# TravelingSalesman → ILP Reduction Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Implement a reduction from TravelingSalesman to ILP using position-based variables with McCormick linearization. - -**Architecture:** Introduce binary variables x_{v,k} (vertex v at position k) and auxiliary variables y_{v,w,k} for linearizing products. Constraints enforce a valid permutation (assignment), prohibit non-edge consecutive positions, and McCormick constraints linearize the objective. Solution extraction reads the tour permutation from x variables and maps back to edge selection. - -**Tech Stack:** Rust, `#[cfg(feature = "ilp")]` gated, `ILPSolver` for solving, `BruteForce` for cross-validation. - ---- - -## Background - -The issue says "HamiltonianCycle" but the model was renamed to `TravelingSalesman` during development. The source problem is `TravelingSalesman<SimpleGraph, i32>`. - -**Source model** (`src/models/graph/traveling_salesman.rs`): -- Variables: one binary per edge (0 = not in cycle, 1 = in cycle) -- Metric: `SolutionSize<W>` (minimize total edge weight) -- Key methods: `num_vertices()`, `num_edges()`, `edges() -> Vec<(usize, usize, W)>`, `graph() -> &G` - -**Target model** (`src/models/optimization/ilp.rs`): -- `ILP::new(num_vars, bounds, constraints, objective, sense)` -- `VarBounds::binary()`, `LinearConstraint::le/ge/eq(terms, rhs)` - -**Reference reduction with similar structure:** `src/rules/coloring_ilp.rs` (vertex × color variables, assignment constraints, non-trivial extraction) - -## ILP Formulation - -Given graph G=(V,E) with n=|V|, m=|E|, edge weights w. - -**Main variables:** x_{v,k} ∈ {0,1} for v ∈ V, k ∈ {0,...,n-1}. Meaning: vertex v is at position k in the tour. Index: `v * n + k`. Total: n². - -**Auxiliary variables:** For each undirected edge (u,v) ∈ E (with u < v) and each position k ∈ {0,...,n-1}, introduce two auxiliary variables for the two directions: -- y_{u,v,k} linearizes x_{u,k} · x_{v,(k+1) mod n} (edge traversed u→v at position k) -- y_{v,u,k} linearizes x_{v,k} · x_{u,(k+1) mod n} (edge traversed v→u at position k) - -Index auxiliary as: n² + edge_idx * 2n + 2k + direction (0 for u→v, 1 for v→u). Total auxiliary: 2mn. - -**Constraints:** -1. Each vertex has exactly one position: Σ_k x_{v,k} = 1 for all v. (n constraints) -2. Each position has exactly one vertex: Σ_v x_{v,k} = 1 for all k. (n constraints) -3. Non-edge consecutive prohibition: For each ordered pair (v,w) where {v,w} ∉ E (and v ≠ w), for each k: x_{v,k} + x_{w,(k+1) mod n} ≤ 1. Count: n · (n(n-1) - 2m) constraints. -4. McCormick linearization (3 constraints per auxiliary variable): - - y ≤ x_{v,k} - - y ≤ x_{w,(k+1) mod n} - - y ≥ x_{v,k} + x_{w,(k+1) mod n} - 1 - Total: 3 · 2mn = 6mn constraints. - -**Objective:** Minimize Σ_{(u,v)∈E} w(u,v) · Σ_k (y_{u,v,k} + y_{v,u,k}) - -**Solution extraction:** -1. Read x_{v,k} from ILP solution: for each position k, find vertex v where x_{v,k} = 1 → get tour permutation π -2. Convert tour to edge selection: for each consecutive pair (π(k), π((k+1) mod n)), find the edge index in the source graph and set it to 1 - -**Size overhead:** -- num_vars: n² + 2mn -- num_constraints: 2n + n(n(n-1) - 2m) + 6mn - ---- - -### Task 1: Implement TravelingSalesman → ILP reduction - -**Files:** -- Create: `src/rules/travelingsalesman_ilp.rs` -- Modify: `src/rules/mod.rs` - -**Step 1: Create the reduction file** - -Create `src/rules/travelingsalesman_ilp.rs`: - -```rust -//! Reduction from TravelingSalesman to ILP (Integer Linear Programming). -//! -//! Uses position-based variables x_{v,k} with McCormick linearization. -//! - Variables: x_{v,k} for vertex v at position k (binary), plus auxiliary y variables -//! - Constraints: assignment, non-edge consecutive, McCormick -//! - Objective: minimize total edge weight of the tour - -use crate::models::graph::TravelingSalesman; -use crate::models::optimization::{LinearConstraint, ObjectiveSense, VarBounds, ILP}; -use crate::poly; -use crate::rules::registry::ReductionOverhead; -use crate::rules::traits::{ReduceTo, ReductionResult}; -use crate::topology::{Graph, SimpleGraph}; - -/// Result of reducing TravelingSalesman to ILP. -#[derive(Debug, Clone)] -pub struct ReductionTSPToILP { - target: ILP, - /// Number of vertices in the source graph. - num_vertices: usize, - /// Edges of the source graph (for solution extraction). - source_edges: Vec<(usize, usize)>, -} - -impl ReductionTSPToILP { - /// Variable index for x_{v,k}: vertex v at position k. - fn x_index(&self, v: usize, k: usize) -> usize { - v * self.num_vertices + k - } -} - -impl ReductionResult for ReductionTSPToILP { - type Source = TravelingSalesman<SimpleGraph, i32>; - type Target = ILP; - - fn target_problem(&self) -> &ILP { - &self.target - } - - /// Extract solution: read tour permutation from x variables, - /// then map to edge selection for the source problem. - fn extract_solution(&self, target_solution: &[usize]) -> Vec<usize> { - let n = self.num_vertices; - - // Read tour: for each position k, find vertex v with x_{v,k} = 1 - let mut tour = vec![0usize; n]; - for k in 0..n { - for v in 0..n { - if target_solution[self.x_index(v, k)] == 1 { - tour[k] = v; - break; - } - } - } - - // Map tour to edge selection - let mut edge_selection = vec![0usize; self.source_edges.len()]; - for k in 0..n { - let u = tour[k]; - let v = tour[(k + 1) % n]; - // Find the edge index for (u, v) or (v, u) - for (idx, &(a, b)) in self.source_edges.iter().enumerate() { - if (a == u && b == v) || (a == v && b == u) { - edge_selection[idx] = 1; - break; - } - } - } - - edge_selection - } -} - -#[reduction( - overhead = { - ReductionOverhead::new(vec![ - ("num_vars", poly!(num_vertices ^ 2) + poly!(num_vertices ^ 2 * num_edges)), - ("num_constraints", poly!(num_vertices) + poly!(num_vertices ^ 3) + poly!(num_vertices * num_edges)), - ]) - } -)] -impl ReduceTo<ILP> for TravelingSalesman<SimpleGraph, i32> { - type Result = ReductionTSPToILP; - - fn reduce_to(&self) -> Self::Result { - let n = self.num_vertices(); - let graph = self.graph(); - let edges_with_weights = self.edges(); - let source_edges: Vec<(usize, usize)> = edges_with_weights.iter().map(|&(u, v, _)| (u, v)).collect(); - let edge_weights: Vec<f64> = edges_with_weights.iter().map(|&(_, _, w)| w as f64).collect(); - let m = source_edges.len(); - - // Variable layout: - // [0, n²): x_{v,k} = vertex v at position k - // [n², n² + 2mn): auxiliary y variables for McCormick linearization - // For edge_idx e and position k: - // y_{forward} at n² + e * 2n + 2k (x_{u,k} * x_{v,(k+1)%n}) - // y_{reverse} at n² + e * 2n + 2k + 1 (x_{v,k} * x_{u,(k+1)%n}) - let num_x = n * n; - let num_y = 2 * m * n; - let num_vars = num_x + num_y; - - let x_idx = |v: usize, k: usize| -> usize { v * n + k }; - let y_idx = |edge: usize, k: usize, dir: usize| -> usize { num_x + edge * 2 * n + 2 * k + dir }; - - let bounds = vec![VarBounds::binary(); num_vars]; - let mut constraints = Vec::new(); - - // Constraint 1: Each vertex has exactly one position - for v in 0..n { - let terms: Vec<(usize, f64)> = (0..n).map(|k| (x_idx(v, k), 1.0)).collect(); - constraints.push(LinearConstraint::eq(terms, 1.0)); - } - - // Constraint 2: Each position has exactly one vertex - for k in 0..n { - let terms: Vec<(usize, f64)> = (0..n).map(|v| (x_idx(v, k), 1.0)).collect(); - constraints.push(LinearConstraint::eq(terms, 1.0)); - } - - // Constraint 3: Non-edge consecutive prohibition - // For each ordered pair (v, w) where {v, w} ∉ E and v ≠ w: - // x_{v,k} + x_{w,(k+1) mod n} <= 1 for all k - for v in 0..n { - for w in 0..n { - if v == w { - continue; - } - if graph.has_edge(v, w) { - continue; - } - for k in 0..n { - constraints.push(LinearConstraint::le( - vec![(x_idx(v, k), 1.0), (x_idx(w, (k + 1) % n), 1.0)], - 1.0, - )); - } - } - } - - // Constraint 4: McCormick linearization for auxiliary variables - // For each edge (u, v) at index e: - // Forward (dir=0): y = x_{u,k} * x_{v,(k+1)%n} - // Reverse (dir=1): y = x_{v,k} * x_{u,(k+1)%n} - for (e, &(u, v)) in source_edges.iter().enumerate() { - for k in 0..n { - let k_next = (k + 1) % n; - - // Forward: y_{e,k,0} = x_{u,k} * x_{v,k_next} - let y_fwd = y_idx(e, k, 0); - let xu = x_idx(u, k); - let xv_next = x_idx(v, k_next); - constraints.push(LinearConstraint::le(vec![(y_fwd, 1.0), (xu, -1.0)], 0.0)); - constraints.push(LinearConstraint::le(vec![(y_fwd, 1.0), (xv_next, -1.0)], 0.0)); - constraints.push(LinearConstraint::ge( - vec![(y_fwd, 1.0), (xu, -1.0), (xv_next, -1.0)], - -1.0, - )); - - // Reverse: y_{e,k,1} = x_{v,k} * x_{u,k_next} - let y_rev = y_idx(e, k, 1); - let xv = x_idx(v, k); - let xu_next = x_idx(u, k_next); - constraints.push(LinearConstraint::le(vec![(y_rev, 1.0), (xv, -1.0)], 0.0)); - constraints.push(LinearConstraint::le(vec![(y_rev, 1.0), (xu_next, -1.0)], 0.0)); - constraints.push(LinearConstraint::ge( - vec![(y_rev, 1.0), (xv, -1.0), (xu_next, -1.0)], - -1.0, - )); - } - } - - // Objective: minimize Σ_{e=(u,v)} w_e * Σ_k (y_{e,k,0} + y_{e,k,1}) - let mut objective: Vec<(usize, f64)> = Vec::new(); - for (e, &w) in edge_weights.iter().enumerate() { - for k in 0..n { - objective.push((y_idx(e, k, 0), w)); - objective.push((y_idx(e, k, 1), w)); - } - } - - let target = ILP::new(num_vars, bounds, constraints, objective, ObjectiveSense::Minimize); - - ReductionTSPToILP { - target, - num_vertices: n, - source_edges, - } - } -} - -#[cfg(test)] -#[path = "../unit_tests/rules/travelingsalesman_ilp.rs"] -mod tests; -``` - -**Step 2: Register in `src/rules/mod.rs`** - -Add after the existing ILP reduction registrations (after `mod minimumvertexcover_ilp;`): - -```rust -#[cfg(feature = "ilp")] -mod travelingsalesman_ilp; -``` - -And after the existing `pub use minimumvertexcover_ilp::...` line: - -```rust -#[cfg(feature = "ilp")] -pub use travelingsalesman_ilp::ReductionTSPToILP; -``` - -**Step 3: Run build to verify compilation** - -Run: `cargo build --features ilp` -Expected: Compiles successfully - -**Step 4: Commit** - -```bash -git add src/rules/travelingsalesman_ilp.rs src/rules/mod.rs -git commit -m "feat: add TravelingSalesman to ILP reduction (#52)" -``` - ---- - -### Task 2: Write unit tests for the reduction - -**Files:** -- Create: `src/unit_tests/rules/travelingsalesman_ilp.rs` - -**Step 1: Create the unit test file** - -Create `src/unit_tests/rules/travelingsalesman_ilp.rs`: - -```rust -use super::*; -use crate::solvers::{BruteForce, ILPSolver}; -use crate::traits::Problem; -use crate::types::SolutionSize; - -#[test] -fn test_reduction_creates_valid_ilp_c4() { - // C4 cycle: 4 vertices, 4 edges. Unique Hamiltonian cycle (the cycle itself). - let problem = TravelingSalesman::<SimpleGraph, i32>::unweighted( - 4, - vec![(0, 1), (1, 2), (2, 3), (3, 0)], - ); - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - - // n=4, m=4: num_vars = 16 + 2*4*4 = 48 - assert_eq!(ilp.num_vars, 48); - assert_eq!(ilp.sense, ObjectiveSense::Minimize); - - // All variables should be binary - for bound in &ilp.bounds { - assert_eq!(*bound, VarBounds::binary()); - } -} - -#[test] -fn test_reduction_c4_closed_loop() { - // C4 cycle with unit weights: optimal tour cost = 4 - let problem = TravelingSalesman::<SimpleGraph, i32>::unweighted( - 4, - vec![(0, 1), (1, 2), (2, 3), (3, 0)], - ); - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - - let ilp_solver = ILPSolver::new(); - let ilp_solution = ilp_solver.solve(ilp).expect("ILP should be solvable"); - let extracted = reduction.extract_solution(&ilp_solution); - - // Verify extracted solution is valid on source problem - let metric = problem.evaluate(&extracted); - assert!(metric.is_valid(), "Extracted solution must be valid"); - assert_eq!(metric, SolutionSize::Valid(4)); -} - -#[test] -fn test_reduction_k4_weighted_closed_loop() { - // K4 weighted: find minimum weight Hamiltonian cycle - let problem = TravelingSalesman::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - - // Solve via ILP reduction - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - let ilp_solver = ILPSolver::new(); - let ilp_solution = ilp_solver.solve(ilp).expect("ILP should be solvable"); - let extracted = reduction.extract_solution(&ilp_solution); - - // Solve via brute force for cross-check - let bf = BruteForce::new(); - let bf_solutions = bf.find_all_best(&problem); - let bf_metric = problem.evaluate(&bf_solutions[0]); - let ilp_metric = problem.evaluate(&extracted); - - assert!(ilp_metric.is_valid()); - assert_eq!(ilp_metric, bf_metric, "ILP and brute force must agree on optimal cost"); -} - -#[test] -fn test_reduction_c5_unweighted_closed_loop() { - // C5 cycle with unit weights - let problem = TravelingSalesman::<SimpleGraph, i32>::unweighted( - 5, - vec![(0, 1), (1, 2), (2, 3), (3, 4), (4, 0)], - ); - - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - let ilp_solver = ILPSolver::new(); - let ilp_solution = ilp_solver.solve(ilp).expect("ILP should be solvable"); - let extracted = reduction.extract_solution(&ilp_solution); - - let metric = problem.evaluate(&extracted); - assert!(metric.is_valid()); - assert_eq!(metric, SolutionSize::Valid(5)); -} - -#[test] -fn test_no_hamiltonian_cycle_infeasible() { - // Path graph 0-1-2-3: no Hamiltonian cycle exists - let problem = TravelingSalesman::<SimpleGraph, i32>::unweighted( - 4, - vec![(0, 1), (1, 2), (2, 3)], - ); - - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - let ilp_solver = ILPSolver::new(); - let result = ilp_solver.solve(ilp); - - assert!(result.is_none(), "Path graph should have no Hamiltonian cycle (infeasible ILP)"); -} - -#[test] -fn test_solution_extraction_structure() { - // C4 cycle: verify extraction produces correct edge selection format - let problem = TravelingSalesman::<SimpleGraph, i32>::unweighted( - 4, - vec![(0, 1), (1, 2), (2, 3), (3, 0)], - ); - let reduction: ReductionTSPToILP = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - - let ilp_solver = ILPSolver::new(); - let ilp_solution = ilp_solver.solve(ilp).expect("ILP should be solvable"); - let extracted = reduction.extract_solution(&ilp_solution); - - // Should have one value per edge - assert_eq!(extracted.len(), 4); - // All edges should be selected (C4 has unique cycle = all edges) - assert_eq!(extracted.iter().sum::<usize>(), 4); -} - -#[test] -fn test_solve_reduced() { - // Test via ILPSolver::solve_reduced - let problem = TravelingSalesman::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - - let ilp_solver = ILPSolver::new(); - let solution = ilp_solver.solve_reduced(&problem).expect("solve_reduced should work"); - - let metric = problem.evaluate(&solution); - assert!(metric.is_valid()); - - // Cross-check with brute force - let bf = BruteForce::new(); - let bf_solutions = bf.find_all_best(&problem); - assert_eq!(metric, problem.evaluate(&bf_solutions[0])); -} -``` - -**Step 2: Run tests** - -Run: `cargo test --features ilp travelingsalesman_ilp -- --nocapture` -Expected: All 7 tests pass - -**Step 3: Commit** - -```bash -git add src/unit_tests/rules/travelingsalesman_ilp.rs -git commit -m "test: add unit tests for TravelingSalesman to ILP reduction (#52)" -``` - ---- - -### Task 3: Write example program - -**Files:** -- Create: `examples/reduction_travelingsalesman_to_ilp.rs` -- Modify: `tests/suites/examples.rs` - -**Step 1: Create example file** - -Create `examples/reduction_travelingsalesman_to_ilp.rs`: - -```rust -// # Traveling Salesman to ILP Reduction -// -// ## Mathematical Formulation -// Variables: x_{v,k} in {0,1} for vertex v and position k; -// auxiliary y variables for McCormick linearization of products. -// Constraints: assignment, non-edge consecutive prohibition, McCormick. -// Objective: minimize total edge weight of the tour. -// -// ## This Example -// - Instance: K4 complete graph with weights -// - Source: TravelingSalesman with 4 vertices, 6 edges -// - Target: ILP with position-based binary variables -// -// ## Output -// Exports `docs/paper/examples/travelingsalesman_to_ilp.json` and `travelingsalesman_to_ilp.result.json`. - -use problemreductions::export::*; -use problemreductions::prelude::*; -use problemreductions::topology::SimpleGraph; - -pub fn run() { - // 1. Create TSP instance: K4 with weights - let problem = TravelingSalesman::<SimpleGraph, i32>::new( - 4, - vec![ - (0, 1, 10), (0, 2, 15), (0, 3, 20), - (1, 2, 35), (1, 3, 25), (2, 3, 30), - ], - ); - - // 2. Reduce to ILP - let reduction = ReduceTo::<ILP>::reduce_to(&problem); - let ilp = reduction.target_problem(); - - // 3. Print transformation - println!("\n=== Problem Transformation ==="); - println!( - "Source: TravelingSalesman with {} variables ({} edges)", - problem.num_variables(), - problem.num_edges() - ); - println!( - "Target: ILP with {} variables, {} constraints", - ilp.num_vars, - ilp.constraints.len() - ); - - // 4. Solve target ILP - let ilp_solver = ILPSolver::new(); - let ilp_solution = ilp_solver.solve(ilp).expect("ILP should be solvable"); - - // 5. Extract source solution - let tsp_solution = reduction.extract_solution(&ilp_solution); - println!("\n=== Solution ==="); - println!("Edge selection: {:?}", tsp_solution); - - // 6. Verify - let metric = problem.evaluate(&tsp_solution); - println!("Tour cost: {:?}", metric); - assert!(metric.is_valid()); - - // Cross-check with brute force - let bf = BruteForce::new(); - let bf_solutions = bf.find_all_best(&problem); - let bf_metric = problem.evaluate(&bf_solutions[0]); - assert_eq!(metric, bf_metric, "ILP must match brute force optimum"); - println!("Brute force confirms optimality"); - - // 7. Collect solutions and export JSON - let mut solutions = Vec::new(); - solutions.push(SolutionPair { - source_config: tsp_solution.clone(), - target_config: ilp_solution, - }); - - let overhead = lookup_overhead_or_empty("TravelingSalesman", "ILP"); - let edges: Vec<(usize, usize)> = problem.edges().iter().map(|&(u, v, _)| (u, v)).collect(); - - let data = ReductionData { - source: ProblemSide { - problem: TravelingSalesman::<SimpleGraph, i32>::NAME.to_string(), - variant: variant_to_map(TravelingSalesman::<SimpleGraph, i32>::variant()), - instance: serde_json::json!({ - "num_vertices": problem.num_vertices(), - "num_edges": problem.num_edges(), - "edges": edges, - }), - }, - target: ProblemSide { - problem: ILP::NAME.to_string(), - variant: variant_to_map(ILP::variant()), - instance: serde_json::json!({ - "num_vars": ilp.num_vars, - "num_constraints": ilp.constraints.len(), - }), - }, - overhead: overhead_to_json(&overhead), - }; - - let results = ResultData { solutions }; - let name = "travelingsalesman_to_ilp"; - write_example(name, &data, &results); -} - -fn main() { - run() -} -``` - -**Step 2: Register in `tests/suites/examples.rs`** - -Add after existing example registrations: - -```rust -example_test!(reduction_travelingsalesman_to_ilp); -``` - -And the corresponding test function: - -```rust -example_fn!( - test_travelingsalesman_to_ilp, - reduction_travelingsalesman_to_ilp -); -``` - -**Step 3: Run example test** - -Run: `cargo test --features ilp test_travelingsalesman_to_ilp -- --nocapture` -Expected: PASS - -**Step 4: Commit** - -```bash -git add examples/reduction_travelingsalesman_to_ilp.rs tests/suites/examples.rs -git commit -m "feat: add TravelingSalesman to ILP example (#52)" -``` - ---- - -### Task 4: Document in paper - -**Files:** -- Modify: `docs/paper/reductions.typ` - -**Step 1: Add reduction-rule entry** - -Add the following `reduction-rule` entry in `docs/paper/reductions.typ` in the appropriate location (near other ILP reductions): - -```typst -#reduction-rule("TravelingSalesman", "ILP", - example: true, - example-caption: [Weighted $K_4$: the optimal tour $0 arrow 1 arrow 3 arrow 2 arrow 0$ with cost 80 is found by position-based ILP.], -)[ - The traveling salesman problem reduces to binary ILP with $n^2 + 2 m n$ variables via position-based encoding with McCormick linearization. -][ - _Construction._ For graph $G = (V, E)$ with $n = |V|$ and $m = |E|$: - - _Variables:_ Binary $x_(v,k) in {0, 1}$ for each vertex $v in V$ and position $k in {0, ..., n-1}$. Interpretation: $x_(v,k) = 1$ iff vertex $v$ is at position $k$ in the tour. - - _Auxiliary variables:_ For each edge $(u,v) in E$ and position $k$, introduce $y_(u,v,k)$ and $y_(v,u,k)$ to linearize the products $x_(u,k) dot x_(v,(k+1) mod n)$ and $x_(v,k) dot x_(u,(k+1) mod n)$ respectively. - - _Constraints:_ (1) Each vertex has exactly one position: $sum_(k=0)^(n-1) x_(v,k) = 1$ for all $v in V$. (2) Each position has exactly one vertex: $sum_(v in V) x_(v,k) = 1$ for all $k$. (3) Non-edge consecutive prohibition: if ${v,w} in.not E$, then $x_(v,k) + x_(w,(k+1) mod n) <= 1$ for all $k$. (4) McCormick: $y <= x_(v,k)$, $y <= x_(w,(k+1) mod n)$, $y >= x_(v,k) + x_(w,(k+1) mod n) - 1$. - - _Objective:_ Minimize $sum_((u,v) in E) w(u,v) dot sum_k (y_(u,v,k) + y_(v,u,k))$. - - _Solution extraction._ For each position $k$, find vertex $v$ with $x_(v,k) = 1$ to recover the tour permutation; then select edges between consecutive positions. -] -``` - -**Step 2: Commit** - -```bash -git add docs/paper/reductions.typ -git commit -m "docs: add TravelingSalesman to ILP reduction rule in paper (#52)" -``` - ---- - -### Task 5: Regenerate reduction graph and verify - -**Step 1: Regenerate the reduction graph** - -Run: `cargo run --features ilp --example export_graph` -Expected: Updates `docs/src/reductions/reduction_graph.json` with TravelingSalesman → ILP edge - -**Step 2: Run full test suite** - -Run: `make test clippy` -Expected: All tests pass, no clippy warnings - -**Step 3: Run coverage check** - -Run: `make coverage` -Expected: Coverage >95% for new code - -**Step 4: Commit any generated files** - -```bash -git add docs/src/reductions/reduction_graph.json -git commit -m "chore: regenerate reduction graph with TravelingSalesman to ILP edge (#52)" -``` - ---- - -## Notes - -- The `poly!()` macro expressions for overhead are approximate upper bounds. Verify the exact polynomial forms compile correctly; adjust if the macro doesn't support the needed expressions. -- The `ILPSolver::solve()` returns `Option<Vec<usize>>` — `None` means infeasible, which is the expected result for graphs without Hamiltonian cycles. -- For small test instances (n ≤ 5), the ILP solver should complete quickly. Avoid K5 or larger complete graphs in tests as the ILP grows rapidly. -- The `#[reduction(...)]` proc macro must be on the `impl ReduceTo<ILP>` block. Check that `poly!` supports the `^` operator or use multiplication: `poly!(num_vertices * num_vertices)`. diff --git a/docs/plans/2026-02-14-type-system-cleanup-design.md b/docs/plans/2026-02-14-type-system-cleanup-design.md deleted file mode 100644 index 6d52ee8d4..000000000 --- a/docs/plans/2026-02-14-type-system-cleanup-design.md +++ /dev/null @@ -1,131 +0,0 @@ -# Type System Cleanup Design - -## Problem - -The weight and trait system has several mathematical inconsistencies: - -1. **Weight dual role**: The type parameter `W` serves as both the per-element weight type and the accumulation/metric type. This prevents using a unit-weight type (`One`) because `One + One` can't produce `2` within the same type. - -2. **Dead abstractions**: `Unweighted(usize)` is never used as a type parameter. The `Weights` trait is implemented but never used outside its own tests. `NumericWeight` and `NumericSize` are nearly identical traits. - -3. **Missing satisfaction trait**: Satisfaction problems (SAT, CircuitSAT, KColoring, Factoring) use `Metric = bool` but have no shared trait. The `BruteForce::find_satisfying()` method uses `Problem<Metric = bool>` inline. - -## Design - -### 1. `WeightElement` trait + `One` type - -Introduce a trait that maps weight element types to their accumulation type: - -```rust -/// Maps a weight element to its sum/metric type. -pub trait WeightElement: Clone + Default + 'static { - /// The numeric type used for sums and comparisons. - type Sum: NumericSize; - /// Convert this weight element to the sum type. - fn to_sum(&self) -> Self::Sum; -} -``` - -Implementations: - -```rust -/// The constant 1. Unit weight for unweighted problems. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default, Serialize, Deserialize)] -pub struct One; - -impl WeightElement for One { - type Sum = i32; - fn to_sum(&self) -> i32 { 1 } -} - -impl WeightElement for i32 { - type Sum = i32; - fn to_sum(&self) -> i32 { *self } -} - -impl WeightElement for f64 { - type Sum = f64; - fn to_sum(&self) -> f64 { *self } -} -``` - -**Impact on problems:** - -Before: -```rust -impl<G, W> Problem for MaximumIndependentSet<G, W> -where W: Clone + Default + PartialOrd + Num + Zero + AddAssign + 'static -{ - type Metric = SolutionSize<W>; - fn evaluate(&self, config: &[usize]) -> SolutionSize<W> { - let mut total = W::zero(); - for (i, &sel) in config.iter().enumerate() { - if sel == 1 { total += self.weights[i].clone(); } - } - SolutionSize::Valid(total) - } -} - -impl<G, W> OptimizationProblem for MaximumIndependentSet<G, W> { - type Value = W; -} -``` - -After: -```rust -impl<G, W: WeightElement> Problem for MaximumIndependentSet<G, W> -where W::Sum: PartialOrd -{ - type Metric = SolutionSize<W::Sum>; - fn evaluate(&self, config: &[usize]) -> SolutionSize<W::Sum> { - let mut total = W::Sum::zero(); - for (i, &sel) in config.iter().enumerate() { - if sel == 1 { total += self.weights[i].to_sum(); } - } - SolutionSize::Valid(total) - } -} - -impl<G, W: WeightElement> OptimizationProblem for MaximumIndependentSet<G, W> { - type Value = W::Sum; -} -``` - -**Variant output:** `variant()` uses `short_type_name::<W>()` which returns `"One"`, `"i32"`, or `"f64"`. The variant label changes from `"Unweighted"` to `"One"`. - -### 2. `SatisfactionProblem` marker trait - -```rust -/// Marker trait for satisfaction (decision) problems. -pub trait SatisfactionProblem: Problem<Metric = bool> {} -``` - -Implemented by: `Satisfiability`, `KSatisfiability`, `CircuitSAT`, `KColoring`, `Factoring`. - -No new methods. Makes the problem category explicit in the type system. `BruteForce::find_satisfying()` can use `P: SatisfactionProblem` as its bound. - -### 3. Merge `NumericWeight` / `NumericSize` - -Delete `NumericWeight`. Keep `NumericSize` as the sole numeric bound trait: - -```rust -pub trait NumericSize: - Clone + Default + PartialOrd + Num + Zero + Bounded + AddAssign + 'static -{} -``` - -This is the bound on `WeightElement::Sum`. The extra `Bounded` requirement (vs the old `NumericWeight`) is needed for solver penalty calculations and is satisfied by `i32` and `f64`. - -### Removals - -- `Unweighted` struct (replaced by `One`) -- `Weights` trait (unused, subsumed by `WeightElement`) -- `NumericWeight` trait (merged into `NumericSize`) - -### Reduction impact - -Concrete `ReduceTo` impls change `Unweighted` references to `One`. The `ConcreteVariantEntry` registrations in `variants.rs` change `"Unweighted"` to `"One"`. The natural edge system (weight subtype hierarchy) adds `One` as a subtype of `i32`. - -### Variant impact - -The `variant()` output for unweighted problems changes from `("weight", "Unweighted")` to `("weight", "One")`. The reduction graph JSON, paper, and JavaScript visualization update accordingly. diff --git a/docs/plans/2026-02-14-type-system-cleanup-impl.md b/docs/plans/2026-02-14-type-system-cleanup-impl.md deleted file mode 100644 index 9795908ff..000000000 --- a/docs/plans/2026-02-14-type-system-cleanup-impl.md +++ /dev/null @@ -1,368 +0,0 @@ -# Type System Cleanup Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Clean up the weight type system by introducing `WeightElement` trait and `One` type, add `SatisfactionProblem` marker trait, and delete dead abstractions. - -**Architecture:** Three independent changes: (1) `WeightElement` trait decouples weight element type from accumulation type, enabling `One` as a unit weight; (2) `SatisfactionProblem` marker trait for `Metric = bool` problems; (3) merge `NumericWeight` into `NumericSize`. - -**Tech Stack:** Rust, inventory crate for registry - -**Design doc:** `docs/plans/2026-02-14-type-system-cleanup-design.md` - ---- - -### Task 1: Add `WeightElement` trait and `One` type to `types.rs` - -**Files:** -- Modify: `src/types.rs` -- Test: `src/unit_tests/types.rs` - -**Step 1: Add `WeightElement` trait and implementations after `NumericSize`** - -Add after the `NumericSize` blanket impl (after line 51): - -```rust -/// Maps a weight element to its sum/metric type. -/// -/// This decouples the per-element weight type from the accumulation type. -/// For concrete weights (`i32`, `f64`), `Sum` is the same type. -/// For the unit weight `One`, `Sum = i32`. -pub trait WeightElement: Clone + Default + 'static { - /// The numeric type used for sums and comparisons. - type Sum: NumericSize; - /// Convert this weight element to the sum type. - fn to_sum(&self) -> Self::Sum; -} - -impl WeightElement for i32 { - type Sum = i32; - fn to_sum(&self) -> i32 { - *self - } -} - -impl WeightElement for f64 { - type Sum = f64; - fn to_sum(&self) -> f64 { - *self - } -} -``` - -**Step 2: Replace `Unweighted` with `One`** - -Replace the `Unweighted` struct, its methods, Display impl, and Weights impl with: - -```rust -/// The constant 1. Unit weight for unweighted problems. -/// -/// When used as the weight type parameter `W`, indicates that all weights -/// are uniformly 1. `One::to_sum()` returns `1i32`. -#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash, Default, Serialize, Deserialize)] -pub struct One; - -impl WeightElement for One { - type Sum = i32; - fn to_sum(&self) -> i32 { - 1 - } -} - -impl std::fmt::Display for One { - fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { - write!(f, "One") - } -} -``` - -**Step 3: Delete dead code** - -- Delete `NumericWeight` trait and blanket impl (lines 11-26) -- Delete `Weights` trait (lines 53-67) -- Delete `Weights for Unweighted` impl (lines 93-102) -- Delete `Weights for Vec<i32>` impl (lines 104-113) -- Delete `Weights for Vec<f64>` impl (lines 115-124) - -**Step 4: Update `src/unit_tests/types.rs`** - -- Replace `test_unweighted` to test `One` instead -- Replace `test_unweighted_weights_trait` to test `WeightElement for One` -- Add tests for `WeightElement for i32` and `WeightElement for f64` - -**Step 5: Run test to verify** - -Run: `cargo test --lib types::tests` - -**Step 6: Commit** - -``` -feat: add WeightElement trait and One type, remove Unweighted/Weights/NumericWeight -``` - ---- - -### Task 2: Update `lib.rs` exports - -**Files:** -- Modify: `src/lib.rs` - -**Step 1: Update prelude and crate-level re-exports** - -In the `pub mod prelude` block (line 102), replace: -```rust -Direction, NumericSize, NumericWeight, ProblemSize, SolutionSize, Unweighted, Weights, -``` -with: -```rust -Direction, NumericSize, One, ProblemSize, SolutionSize, WeightElement, -``` - -**Step 2: Run build to check** - -Run: `cargo check --all-features` - -**Step 3: Commit** - -``` -refactor: update lib.rs exports for WeightElement/One -``` - ---- - -### Task 3: Update graph problem `Problem` and `OptimizationProblem` impls - -**Files (8 graph problems):** -- Modify: `src/models/graph/maximum_independent_set.rs` -- Modify: `src/models/graph/maximum_clique.rs` -- Modify: `src/models/graph/minimum_vertex_cover.rs` -- Modify: `src/models/graph/minimum_dominating_set.rs` -- Modify: `src/models/graph/maximal_is.rs` -- Modify: `src/models/graph/maximum_matching.rs` -- Modify: `src/models/graph/traveling_salesman.rs` -- Modify: `src/models/graph/max_cut.rs` - -For each file, apply the same pattern: - -**Step 1: Update `Problem` impl** - -Change trait bounds from: -```rust -impl<G, W> Problem for ProblemType<G, W> -where - G: Graph, - W: Clone + Default + PartialOrd + Num + Zero + AddAssign + 'static, -``` -to: -```rust -impl<G, W> Problem for ProblemType<G, W> -where - G: Graph, - W: WeightElement, -``` - -Change `type Metric = SolutionSize<W>` to `type Metric = SolutionSize<W::Sum>`. - -Change `evaluate()` body: replace `W::zero()` with `W::Sum::zero()`, replace `self.weights[i].clone()` with `self.weights[i].to_sum()` (in the accumulation `total +=` line). - -**Step 2: Update `OptimizationProblem` impl** - -Same bound changes. Change `type Value = W` to `type Value = W::Sum`. - -**Step 3: Update constructors and helper methods** - -In `new()` constructors that create default weights: `vec![W::from(1); n]` stays as-is since `One::default()` doesn't produce 1 — but actually these constructors are bounded on `W: From<i32>`, and `One` doesn't implement `From<i32>`. Instead, `new()` should use `W::default()` for `One` (but default of `One` is `One`, which is correct). Check each constructor individually — most use `W::from(1)` which needs `From<i32>` bound. Since `One` is constructed as the default and `from(1)` makes no sense for `One`, the `new()` constructor that creates unit weights should be specialized or use a `WeightElement`-specific helper. - -**Alternative:** Add `From<i32> for One`: -```rust -impl From<i32> for One { - fn from(_: i32) -> Self { One } -} -``` -This allows `W::from(1)` to work for `One` — it ignores the value and returns `One`. This is mathematically sound: promoting any integer to the `One` type gives `One`. - -Add this to `types.rs` in Task 1. - -**Step 4: Update `ReductionResult` impls in the same files** - -`ReductionResult` impls with generic `W` bounds need the same bound change from the long trait list to `W: WeightElement`. - -**Step 5: Run tests** - -Run: `cargo test --lib models::graph` - -**Step 6: Commit** - -``` -refactor: update graph problem impls to use WeightElement -``` - ---- - -### Task 4: Update set and optimization problem impls - -**Files (4 problems):** -- Modify: `src/models/set/maximum_set_packing.rs` -- Modify: `src/models/set/minimum_set_covering.rs` -- Modify: `src/models/optimization/qubo.rs` -- Modify: `src/models/optimization/spin_glass.rs` - -Same pattern as Task 3: update `Problem` bounds, `Metric`, `Value`, and `evaluate()` body. - -For `QUBO<W>` and `SpinGlass<G, W>`, the `W` parameter is already the numeric type (not a weight element in the vertex sense), so `WeightElement for f64` with `Sum = f64` should work directly. Verify that `W::zero()` still works via `NumericSize` bound on `W::Sum`. - -**Step 1: Apply same changes as Task 3** - -**Step 2: Run tests** - -Run: `cargo test --lib models::set models::optimization` - -**Step 3: Commit** - -``` -refactor: update set and optimization problem impls to use WeightElement -``` - ---- - -### Task 5: Update reduction rule files - -**Files (~20 reduction files):** -- All files in `src/rules/` that have generic `W` bounds on `ReductionResult` impls - -The concrete `ReduceTo` impls (from our previous work) don't need changes since they use `i32`/`f64` directly. But the generic `ReductionResult` impls need bounds updated. - -**Step 1: For each reduction file with generic `ReductionResult` impls** - -Replace: -```rust -where W: Clone + Default + PartialOrd + Num + Zero + Bounded + AddAssign + 'static -``` -or similar long bound lists with: -```rust -where W: WeightElement -``` - -Add `use crate::types::WeightElement;` to imports if not already present. Remove unused `num_traits` imports. - -**Step 2: Run tests** - -Run: `cargo test --all-features` - -**Step 3: Commit** - -``` -refactor: update reduction rule bounds to use WeightElement -``` - ---- - -### Task 6: Update variant metadata — `"Unweighted"` to `"One"` - -**Files:** -- Modify: `src/rules/variants.rs` — replace all `"Unweighted"` with `"One"` -- Modify: `src/graph_types.rs` — replace `"Unweighted"` in weight subtype declarations -- Modify: `src/rules/registry.rs` — replace `"Unweighted"` in weight checking (if any) -- Modify: `docs/src/reductions/reduction_graph.json` — regenerated -- Modify: test files that assert on `"Unweighted"` string - -**Step 1: Replace `"Unweighted"` with `"One"` in source files** - -In `variants.rs`, `graph_types.rs`, and `registry.rs`. - -**Step 2: Update test assertions** - -In `unit_tests/rules/graph.rs`, `unit_tests/rules/registry.rs`, `unit_tests/graph_types.rs` — replace all `"Unweighted"` assertions with `"One"`. - -**Step 3: Regenerate reduction graph JSON** - -Run: `make rust-export` - -**Step 4: Run tests** - -Run: `cargo test --all-features` - -**Step 5: Commit** - -``` -refactor: rename Unweighted to One in variant metadata -``` - ---- - -### Task 7: Add `SatisfactionProblem` marker trait - -**Files:** -- Modify: `src/traits.rs` -- Modify: `src/models/satisfiability/sat.rs` -- Modify: `src/models/satisfiability/ksat.rs` -- Modify: `src/models/specialized/circuit.rs` -- Modify: `src/models/graph/kcoloring.rs` -- Modify: `src/lib.rs` (re-export) - -**Step 1: Add trait to `src/traits.rs`** - -After the `OptimizationProblem` trait: -```rust -/// Marker trait for satisfaction (decision) problems. -/// -/// Satisfaction problems evaluate configurations to `bool`: -/// `true` if the configuration satisfies all constraints, `false` otherwise. -pub trait SatisfactionProblem: Problem<Metric = bool> {} -``` - -**Step 2: Implement for each satisfaction problem** - -In each file, add after the `Problem` impl: -```rust -impl SatisfactionProblem for Satisfiability {} -impl<const K: usize> SatisfactionProblem for KSatisfiability<K> {} -impl SatisfactionProblem for CircuitSAT {} -impl<const K: usize, G: Graph> SatisfactionProblem for KColoring<K, G> {} -``` - -**Step 3: Add re-export in `lib.rs`** - -Add `SatisfactionProblem` to the traits re-export. - -**Step 4: Optionally update solver bounds** - -In `src/solvers/brute_force.rs` and `src/solvers/mod.rs`, change `P: Problem<Metric = bool>` to `P: SatisfactionProblem`. This is optional — the existing bound still works. - -**Step 5: Run tests** - -Run: `cargo test --all-features` - -**Step 6: Commit** - -``` -feat: add SatisfactionProblem marker trait -``` - ---- - -### Task 8: Final verification and cleanup - -**Step 1: Run full test suite** - -Run: `make test clippy` - -**Step 2: Check for any remaining `Unweighted` or `NumericWeight` references** - -Run: `rg "Unweighted|NumericWeight" src/` - -Any remaining references should be in comments/docs only — update those too. - -**Step 3: Update paper if needed** - -Check `docs/paper/reductions.typ` for `Unweighted` references. - -**Step 4: Run doc build** - -Run: `make doc` - -**Step 5: Final commit** - -``` -chore: cleanup remaining Unweighted/NumericWeight references -``` diff --git a/docs/plans/2026-02-14-variant-aware-paths-design.md b/docs/plans/2026-02-14-variant-aware-paths-design.md deleted file mode 100644 index 452a1b81f..000000000 --- a/docs/plans/2026-02-14-variant-aware-paths-design.md +++ /dev/null @@ -1,204 +0,0 @@ -# Variant-Aware Reduction Paths - -**Goal:** Make reduction paths variant-level so that (a) variant-specific reductions are disambiguated (issue 2) and (b) natural cast steps are computed automatically from subtype hierarchies (issue 5). - -## Background - -The runtime `ReductionGraph` uses name-only nodes. `ReductionPath` is `Vec<&'static str>` — it carries no variant information. This causes two problems: - -1. **Overhead lookup ambiguity (issue 2):** `lookup_overhead("KSatisfiability", "QUBO")` returns the first hit from inventory. KSatisfiability<2>→QUBO and KSatisfiability<3>→QUBO have different overheads, but the caller can't distinguish them. - -2. **Natural edge inconsistency (issue 5):** The JSON export infers 8 natural edges (e.g., MIS GridGraph→SimpleGraph) from subtype hierarchies, but only 1 has a backing `ReduceTo` impl. Users see edges in documentation that aren't executable. - -## Design - -### 1. `ResolvedPath` Data Model - -A variant-level path where each node carries `(name, variant)` and each edge is typed: - -```rust -/// A node in a variant-level reduction path. -#[derive(Debug, Clone, Serialize)] -pub struct ReductionStep { - /// Problem name (e.g., "MaximumIndependentSet"). - pub name: String, - /// Variant at this point (e.g., {"graph": "GridGraph", "weight": "i32"}). - pub variant: BTreeMap<String, String>, -} - -/// The kind of transition between adjacent steps. -#[derive(Debug, Clone, Serialize)] -pub enum EdgeKind { - /// A registered reduction (backed by a ReduceTo impl). - Reduction { - /// Overhead from the matching ReductionEntry. - overhead: ReductionOverhead, - }, - /// A natural cast via subtype relaxation. Identity overhead. - NaturalCast, -} - -/// A fully resolved reduction path with variant information at each node. -#[derive(Debug, Clone, Serialize)] -pub struct ResolvedPath { - /// Sequence of (name, variant) nodes. - pub steps: Vec<ReductionStep>, - /// Edge kinds between adjacent steps. Length = steps.len() - 1. - pub edges: Vec<EdgeKind>, -} -``` - -Example — resolving `MIS(GridGraph, i32) → QUBO(f64)` through name-path `["MIS", "QUBO"]`: - -``` -steps: - [0] MIS {graph: "GridGraph", weight: "i32"} ← source - [1] MIS {graph: "SimpleGraph", weight: "i32"} ← natural cast - [2] QUBO {weight: "f64"} ← reduction target - -edges: - [0] NaturalCast ← GridGraph <: SimpleGraph - [1] Reduction { overhead: ... } ← MIS→QUBO rule -``` - -### 2. Resolution Algorithm - -```rust -impl ReductionGraph { - pub fn resolve_path( - &self, - path: &ReductionPath, - source_variant: &BTreeMap<String, String>, - target_variant: &BTreeMap<String, String>, - ) -> Option<ResolvedPath> { ... } -} -``` - -Algorithm: - -``` -current_variant = source_variant -steps = [ Step(path[0], current_variant) ] -edges = [] - -for each edge (src_name → dst_name) in the name-level path: - - 1. FIND CANDIDATES - Collect all ReductionEntry where - entry.source_name == src_name AND entry.target_name == dst_name - - 2. FILTER COMPATIBLE - Keep entries where current_variant is reducible to entry.source_variant - (current is equal-or-more-specific on every variant axis) - - 3. PICK MOST SPECIFIC - Among compatible entries, pick the one whose source_variant is the - tightest supertype of current_variant. - If none compatible → return None. - - 4. INSERT NATURAL CAST (if needed) - If current_variant ≠ best_rule.source_variant: - steps.push( Step(src_name, best_rule.source_variant) ) - edges.push( NaturalCast ) - - 5. ADVANCE - current_variant = best_rule.target_variant - steps.push( Step(dst_name, current_variant) ) - edges.push( Reduction { overhead: best_rule.overhead() } ) - -// Trailing natural cast if final variant differs from target -if current_variant ≠ target_variant - AND is_variant_reducible(current_variant, target_variant): - steps.push( Step(last_name, target_variant) ) - edges.push( NaturalCast ) - -return ResolvedPath { steps, edges } -``` - -### 3. KSat Disambiguation Example - -Resolving `KSat(k=3) → QUBO` via name-path `["KSatisfiability", "QUBO"]`: - -``` -FIND CANDIDATES: - - KSat<2>→QUBO (source_variant: {k:"2"}, overhead: num_vars) - - KSat<3>→QUBO (source_variant: {k:"3"}, overhead: num_vars + num_clauses) - -FILTER COMPATIBLE with current k=3: - - KSat<2>: k=3 reducible to k=2? No (3 is not a subtype of 2) - - KSat<3>: k=3 == k=3? Yes ✓ - -PICK: KSat<3>→QUBO with correct overhead. -``` - -Overhead ambiguity is resolved by construction — the resolver picks the exact matching entry. - -### 4. Natural Edges Become Implicit - -With `resolve_path`, natural casts are **computed from subtype hierarchies**, not registered as `ReduceTo` impls. - -**Removed:** -- `impl_natural_reduction!` macro invocations (the one in `natural.rs` and any future ones) -- Natural edges no longer need `ReductionEntry` registration via inventory - -**Kept:** -- `GraphSubtypeEntry` / `WeightSubtypeEntry` — source of truth for subtype relationships -- Inference logic in `to_json()` — unchanged, still produces natural edges in JSON export -- `GraphCast` trait — still needed for actual execution by callers - -**Callers execute natural steps** using `GraphCast::cast_graph()` (or equivalent weight cast) directly, guided by the `EdgeKind::NaturalCast` marker in the resolved path. No `ReduceTo` dispatch needed. - -### 5. `lookup_overhead` Deprecated - -`lookup_overhead(source_name, target_name)` is replaced by per-step overhead in `ResolvedPath`: - -```rust -impl ResolvedPath { - /// Total overhead for the entire path (composed across all steps). - pub fn total_overhead(&self) -> ReductionOverhead { ... } - - /// Number of reduction steps (excludes natural casts). - pub fn num_reductions(&self) -> usize { ... } - - /// Number of natural cast steps. - pub fn num_casts(&self) -> usize { ... } -} -``` - -Examples migrate from `lookup_overhead("A", "B")` to using the resolved path's overhead. - -### 6. Backward Compatibility - -| API | Change | -|-----|--------| -| `ReductionPath` | Unchanged — still returned by `find_paths`, `find_cheapest_path` | -| `find_paths`, `find_paths_by_name` | Unchanged | -| `find_cheapest_path` | Unchanged (name-level planning) | -| `has_direct_reduction` | Unchanged | -| `resolve_path` | **New** — lifts name-level path to variant-level | -| `ResolvedPath` | **New** | -| `lookup_overhead` | **Deprecated** — kept for one release, then removed | -| `lookup_overhead_or_empty` | **Deprecated** | -| `impl_natural_reduction!` | **Removed** after migration | - -Existing code using `find_paths` + `lookup_overhead` continues working. New code should use `find_paths` + `resolve_path` for variant-correct results. - -### 7. Files Changed - -| File | Change | -|------|--------| -| `src/rules/graph.rs` | Add `ResolvedPath`, `ReductionStep`, `EdgeKind`, `resolve_path()` method | -| `src/export.rs` | Deprecate `lookup_overhead`, `lookup_overhead_or_empty` | -| `src/rules/natural.rs` | Remove `impl_natural_reduction!` invocation | -| `src/rules/mod.rs` | Keep `impl_natural_reduction!` macro (optional convenience), remove from prelude | -| `examples/reduction_ksatisfiability_to_qubo.rs` | Migrate from `lookup_overhead` to `resolve_path` | -| `examples/*.rs` | Migrate remaining examples (can be incremental) | -| `src/unit_tests/rules/graph.rs` | Add tests for `resolve_path` | -| `src/unit_tests/rules/natural.rs` | Update or remove natural reduction tests | - -### 8. Non-Goals - -- Runtime graph does not become variant-level (stays name-only for path discovery) -- No execution engine — `ResolvedPath` is a plan; callers dispatch `ReduceTo` and `GraphCast` themselves -- No changes to `to_json()` natural edge inference (it already works correctly) -- No changes to `#[reduction]` macro diff --git a/docs/plans/2026-02-14-variant-aware-paths-impl.md b/docs/plans/2026-02-14-variant-aware-paths-impl.md deleted file mode 100644 index ce6beba50..000000000 --- a/docs/plans/2026-02-14-variant-aware-paths-impl.md +++ /dev/null @@ -1,747 +0,0 @@ -# Variant-Aware Reduction Paths — Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add `resolve_path()` to lift name-level reduction paths into variant-level paths with natural cast steps, fixing overhead disambiguation (issue 2) and natural edge inconsistency (issue 5). - -**Architecture:** `ResolvedPath` is a new type layered on top of the existing `ReductionPath`. The resolver walks a name-level path, threads variant state through each edge, picks the most-specific matching `ReductionEntry`, and inserts `NaturalCast` steps where the caller's variant is more specific than what the rule expects. No changes to the name-level graph or path-finding algorithms. - -**Tech Stack:** Rust, `inventory` crate (existing), `petgraph` (existing), `serde` (existing), `BTreeMap` for variant representation. - ---- - -### Task 1: Add `ResolvedPath` data types - -**Files:** -- Modify: `src/rules/graph.rs` (after `ReductionPath` impl block, ~line 132) - -**Step 1: Write the failing test** - -Add to `src/unit_tests/rules/graph.rs`: - -```rust -#[test] -fn test_resolved_path_basic_structure() { - use crate::rules::graph::{ResolvedPath, ReductionStep, EdgeKind}; - use std::collections::BTreeMap; - - let steps = vec![ - ReductionStep { - name: "A".to_string(), - variant: BTreeMap::from([("graph".to_string(), "SimpleGraph".to_string())]), - }, - ReductionStep { - name: "B".to_string(), - variant: BTreeMap::from([("weight".to_string(), "f64".to_string())]), - }, - ]; - let edges = vec![EdgeKind::Reduction { - overhead: Default::default(), - }]; - let path = ResolvedPath { - steps: steps.clone(), - edges, - }; - - assert_eq!(path.len(), 1); - assert_eq!(path.num_reductions(), 1); - assert_eq!(path.num_casts(), 0); - assert_eq!(path.steps[0].name, "A"); - assert_eq!(path.steps[1].name, "B"); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test test_resolved_path_basic_structure -- --no-run 2>&1` -Expected: Compilation error — `ResolvedPath`, `ReductionStep`, `EdgeKind` not defined. - -**Step 3: Write the types** - -Add to `src/rules/graph.rs` after the `ReductionPath` impl block (after line 132): - -```rust -/// A node in a variant-level reduction path. -#[derive(Debug, Clone, Serialize)] -pub struct ReductionStep { - /// Problem name (e.g., "MaximumIndependentSet"). - pub name: String, - /// Variant at this point (e.g., {"graph": "GridGraph", "weight": "i32"}). - pub variant: std::collections::BTreeMap<String, String>, -} - -/// The kind of transition between adjacent steps in a resolved path. -#[derive(Debug, Clone, Serialize)] -pub enum EdgeKind { - /// A registered reduction (backed by a ReduceTo impl). - Reduction { - /// Overhead from the matching ReductionEntry. - overhead: ReductionOverhead, - }, - /// A natural cast via subtype relaxation. Identity overhead. - NaturalCast, -} - -/// A fully resolved reduction path with variant information at each node. -/// -/// Created by [`ReductionGraph::resolve_path`] from a name-level [`ReductionPath`]. -/// Each adjacent pair of steps is connected by an [`EdgeKind`]: either a registered -/// reduction or a natural cast (subtype relaxation with identity overhead). -#[derive(Debug, Clone, Serialize)] -pub struct ResolvedPath { - /// Sequence of (name, variant) nodes. - pub steps: Vec<ReductionStep>, - /// Edge kinds between adjacent steps. Length = steps.len() - 1. - pub edges: Vec<EdgeKind>, -} - -impl ResolvedPath { - /// Number of edges (reductions + casts) in the path. - pub fn len(&self) -> usize { - self.edges.len() - } - - /// Whether the path is empty. - pub fn is_empty(&self) -> bool { - self.edges.is_empty() - } - - /// Number of registered reduction steps (excludes natural casts). - pub fn num_reductions(&self) -> usize { - self.edges - .iter() - .filter(|e| matches!(e, EdgeKind::Reduction { .. })) - .count() - } - - /// Number of natural cast steps. - pub fn num_casts(&self) -> usize { - self.edges - .iter() - .filter(|e| matches!(e, EdgeKind::NaturalCast)) - .count() - } -} -``` - -**Step 4: Run test to verify it passes** - -Run: `cargo test test_resolved_path_basic_structure` -Expected: PASS - -**Step 5: Commit** - -```bash -git add src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: add ResolvedPath, ReductionStep, EdgeKind types" -``` - ---- - -### Task 2: Add helper to find matching ReductionEntry candidates - -The resolver needs to iterate `inventory::iter::<ReductionEntry>` filtered by name pair, check variant compatibility, and pick the most specific match. Add this as a private helper on `ReductionGraph`. - -**Files:** -- Modify: `src/rules/graph.rs` (inside the second `impl ReductionGraph` block that contains `is_variant_reducible`, after line 618) - -**Step 1: Write the failing test** - -Add to `src/unit_tests/rules/graph.rs`: - -```rust -#[test] -fn test_find_matching_entry_ksat_k3() { - let graph = ReductionGraph::new(); - let variant_k3: std::collections::BTreeMap<String, String> = - [("k".to_string(), "3".to_string())].into(); - - let entry = graph.find_best_entry("KSatisfiability", "QUBO", &variant_k3); - assert!(entry.is_some()); - let (source_var, _target_var, overhead) = entry.unwrap(); - // K=3 overhead has num_clauses term; K=2 does not - assert!(overhead - .output_size - .iter() - .any(|(field, _)| *field == "num_vars")); - // K=3 overhead: poly!(num_vars) + poly!(num_clauses) → two terms total - let num_vars_poly = &overhead - .output_size - .iter() - .find(|(f, _)| *f == "num_vars") - .unwrap() - .1; - assert!( - num_vars_poly.terms.len() >= 2, - "K=3 overhead should have num_vars + num_clauses" - ); -} - -#[test] -fn test_find_matching_entry_ksat_k2() { - let graph = ReductionGraph::new(); - let variant_k2: std::collections::BTreeMap<String, String> = - [("k".to_string(), "2".to_string())].into(); - - let entry = graph.find_best_entry("KSatisfiability", "QUBO", &variant_k2); - assert!(entry.is_some()); - let (_source_var, _target_var, overhead) = entry.unwrap(); - // K=2 overhead: just poly!(num_vars) → one term - let num_vars_poly = &overhead - .output_size - .iter() - .find(|(f, _)| *f == "num_vars") - .unwrap() - .1; - assert_eq!( - num_vars_poly.terms.len(), - 1, - "K=2 overhead should have only num_vars" - ); -} - -#[test] -fn test_find_matching_entry_no_match() { - let graph = ReductionGraph::new(); - let variant: std::collections::BTreeMap<String, String> = - [("k".to_string(), "99".to_string())].into(); - - // k=99 is not a subtype of k=2 or k=3 - let entry = graph.find_best_entry("KSatisfiability", "QUBO", &variant); - assert!(entry.is_none()); -} -``` - -**Step 2: Run test to verify it fails** - -Run: `cargo test test_find_matching_entry -- --no-run 2>&1` -Expected: Compilation error — `find_best_entry` method not found. - -**Step 3: Implement `find_best_entry`** - -Add to `src/rules/graph.rs` in the `impl ReductionGraph` block that contains `is_variant_reducible` (after `is_variant_reducible` at ~line 618): - -```rust - /// Find the best matching `ReductionEntry` for a (source_name, target_name) pair - /// given the caller's current source variant. - /// - /// "Best" means: compatible (current variant is reducible to the entry's source variant) - /// and most specific (tightest fit among all compatible entries). - /// - /// Returns `(entry_source_variant, entry_target_variant, overhead)` or `None`. - pub fn find_best_entry( - &self, - source_name: &str, - target_name: &str, - current_variant: &std::collections::BTreeMap<String, String>, - ) -> Option<( - std::collections::BTreeMap<String, String>, - std::collections::BTreeMap<String, String>, - ReductionOverhead, - )> { - use crate::rules::registry::ReductionEntry; - - let mut best: Option<( - std::collections::BTreeMap<String, String>, - std::collections::BTreeMap<String, String>, - ReductionOverhead, - )> = None; - - for entry in inventory::iter::<ReductionEntry> { - if entry.source_name != source_name || entry.target_name != target_name { - continue; - } - - let entry_source = Self::variant_to_map(&entry.source_variant()); - let entry_target = Self::variant_to_map(&entry.target_variant()); - - // Check: current_variant is reducible to entry's source variant - // (current is equal-or-more-specific on every axis) - if current_variant != &entry_source - && !self.is_variant_reducible(current_variant, &entry_source) - { - continue; - } - - // Pick the most specific: if we already have a best, prefer the one - // whose source_variant is more specific (tighter fit) - let dominated = if let Some((ref best_source, _, _)) = best { - // New entry is more specific than current best? - self.is_variant_reducible(&entry_source, best_source) - || entry_source == *current_variant - } else { - true - }; - - if dominated { - best = Some((entry_source, entry_target, entry.overhead())); - } - } - - best - } -``` - -**Step 4: Run tests to verify they pass** - -Run: `cargo test test_find_matching_entry` -Expected: All 3 tests PASS. - -**Step 5: Commit** - -```bash -git add src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: add find_best_entry for variant-aware ReductionEntry lookup" -``` - ---- - -### Task 3: Implement `resolve_path` - -**Files:** -- Modify: `src/rules/graph.rs` (add method to `ReductionGraph`, near `find_best_entry`) - -**Step 1: Write the failing tests** - -Add to `src/unit_tests/rules/graph.rs`: - -```rust -#[test] -fn test_resolve_path_direct_same_variant() { - use std::collections::BTreeMap; - let graph = ReductionGraph::new(); - - // MIS(SimpleGraph, i32) → VC(SimpleGraph, i32) — no cast needed - let name_path = graph - .find_shortest_path::< - MaximumIndependentSet<SimpleGraph, i32>, - MinimumVertexCover<SimpleGraph, i32>, - >() - .unwrap(); - - let source_variant = BTreeMap::from([ - ("graph".to_string(), "SimpleGraph".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - let target_variant = BTreeMap::from([ - ("graph".to_string(), "SimpleGraph".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - - let resolved = graph - .resolve_path(&name_path, &source_variant, &target_variant) - .unwrap(); - - assert_eq!(resolved.num_reductions(), 1); - assert_eq!(resolved.num_casts(), 0); - assert_eq!(resolved.steps.len(), 2); - assert_eq!(resolved.steps[0].name, "MaximumIndependentSet"); - assert_eq!(resolved.steps[1].name, "MinimumVertexCover"); -} - -#[test] -fn test_resolve_path_with_natural_cast() { - use std::collections::BTreeMap; - use crate::topology::GridGraph; - let graph = ReductionGraph::new(); - - // MIS(GridGraph) → VC(SimpleGraph) — needs a natural cast MIS(GridGraph)→MIS(SimpleGraph) - let name_path = graph - .find_shortest_path::< - MaximumIndependentSet<GridGraph, i32>, - MinimumVertexCover<SimpleGraph, i32>, - >() - .unwrap(); - - let source_variant = BTreeMap::from([ - ("graph".to_string(), "GridGraph".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - let target_variant = BTreeMap::from([ - ("graph".to_string(), "SimpleGraph".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - - let resolved = graph - .resolve_path(&name_path, &source_variant, &target_variant) - .unwrap(); - - // Should be: MIS(GridGraph) --NaturalCast--> MIS(SimpleGraph) --Reduction--> VC(SimpleGraph) - assert_eq!(resolved.num_reductions(), 1); - assert_eq!(resolved.num_casts(), 1); - assert_eq!(resolved.steps.len(), 3); - assert_eq!(resolved.steps[0].name, "MaximumIndependentSet"); - assert_eq!( - resolved.steps[0].variant.get("graph").unwrap(), - "GridGraph" - ); - assert_eq!(resolved.steps[1].name, "MaximumIndependentSet"); - assert_eq!( - resolved.steps[1].variant.get("graph").unwrap(), - "SimpleGraph" - ); - assert_eq!(resolved.steps[2].name, "MinimumVertexCover"); - assert!(matches!(resolved.edges[0], EdgeKind::NaturalCast)); - assert!(matches!(resolved.edges[1], EdgeKind::Reduction { .. })); -} - -#[test] -fn test_resolve_path_ksat_disambiguates() { - use std::collections::BTreeMap; - use crate::rules::graph::EdgeKind; - let graph = ReductionGraph::new(); - - let name_path = graph - .find_shortest_path_by_name("KSatisfiability", "QUBO") - .unwrap(); - - // Resolve with k=3 - let source_k3 = BTreeMap::from([("k".to_string(), "3".to_string())]); - let target = BTreeMap::from([("weight".to_string(), "f64".to_string())]); - - let resolved_k3 = graph - .resolve_path(&name_path, &source_k3, &target) - .unwrap(); - assert_eq!(resolved_k3.num_reductions(), 1); - - // Extract overhead from the reduction edge - let overhead_k3 = match &resolved_k3.edges.last().unwrap() { - EdgeKind::Reduction { overhead } => overhead, - _ => panic!("last edge should be Reduction"), - }; - // K=3 overhead has 2 terms in num_vars polynomial - let num_vars_poly_k3 = &overhead_k3 - .output_size - .iter() - .find(|(f, _)| *f == "num_vars") - .unwrap() - .1; - assert!(num_vars_poly_k3.terms.len() >= 2); - - // Resolve with k=2 - let source_k2 = BTreeMap::from([("k".to_string(), "2".to_string())]); - let resolved_k2 = graph - .resolve_path(&name_path, &source_k2, &target) - .unwrap(); - let overhead_k2 = match &resolved_k2.edges.last().unwrap() { - EdgeKind::Reduction { overhead } => overhead, - _ => panic!("last edge should be Reduction"), - }; - let num_vars_poly_k2 = &overhead_k2 - .output_size - .iter() - .find(|(f, _)| *f == "num_vars") - .unwrap() - .1; - assert_eq!(num_vars_poly_k2.terms.len(), 1); -} - -#[test] -fn test_resolve_path_incompatible_returns_none() { - use std::collections::BTreeMap; - let graph = ReductionGraph::new(); - - let name_path = graph - .find_shortest_path_by_name("KSatisfiability", "QUBO") - .unwrap(); - - // k=99 matches neither k=2 nor k=3 - let source = BTreeMap::from([("k".to_string(), "99".to_string())]); - let target = BTreeMap::from([("weight".to_string(), "f64".to_string())]); - - let resolved = graph.resolve_path(&name_path, &source, &target); - assert!(resolved.is_none()); -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test test_resolve_path -- --no-run 2>&1` -Expected: Compilation error — `resolve_path` method not found. - -**Step 3: Implement `resolve_path`** - -Add to `src/rules/graph.rs` in the same `impl ReductionGraph` block, after `find_best_entry`: - -```rust - /// Resolve a name-level [`ReductionPath`] into a variant-level [`ResolvedPath`]. - /// - /// Walks the name-level path, threading variant state through each edge. - /// For each step, picks the most-specific compatible `ReductionEntry` and - /// inserts `NaturalCast` steps where the caller's variant is more specific - /// than the rule's expected source variant. - /// - /// Returns `None` if no compatible reduction entry exists for any step. - pub fn resolve_path( - &self, - path: &ReductionPath, - source_variant: &std::collections::BTreeMap<String, String>, - target_variant: &std::collections::BTreeMap<String, String>, - ) -> Option<ResolvedPath> { - if path.type_names.len() < 2 { - return None; - } - - let mut current_variant = source_variant.clone(); - let mut steps = vec![ReductionStep { - name: path.type_names[0].to_string(), - variant: current_variant.clone(), - }]; - let mut edges = Vec::new(); - - for i in 0..path.type_names.len() - 1 { - let src_name = path.type_names[i]; - let dst_name = path.type_names[i + 1]; - - let (entry_source, entry_target, overhead) = - self.find_best_entry(src_name, dst_name, ¤t_variant)?; - - // Insert natural cast if current variant differs from entry's source - if current_variant != entry_source { - steps.push(ReductionStep { - name: src_name.to_string(), - variant: entry_source, - }); - edges.push(EdgeKind::NaturalCast); - } - - // Advance through the reduction - current_variant = entry_target; - steps.push(ReductionStep { - name: dst_name.to_string(), - variant: current_variant.clone(), - }); - edges.push(EdgeKind::Reduction { overhead }); - } - - // Trailing natural cast if final variant differs from requested target - if current_variant != *target_variant - && self.is_variant_reducible(¤t_variant, target_variant) - { - let last_name = path.type_names.last().unwrap(); - steps.push(ReductionStep { - name: last_name.to_string(), - variant: target_variant.clone(), - }); - edges.push(EdgeKind::NaturalCast); - } - - Some(ResolvedPath { steps, edges }) - } -``` - -**Step 4: Run tests to verify they pass** - -Run: `cargo test test_resolve_path` -Expected: All 4 tests PASS. - -**Step 5: Commit** - -```bash -git add src/rules/graph.rs src/unit_tests/rules/graph.rs -git commit -m "feat: add resolve_path for variant-level reduction paths" -``` - ---- - -### Task 4: Deprecate `lookup_overhead` and migrate KSat example - -**Files:** -- Modify: `src/export.rs:91-98` (add deprecation) -- Modify: `src/export.rs:100-103` (add deprecation) -- Modify: `examples/reduction_ksatisfiability_to_qubo.rs:120-121` (migrate to resolve_path) - -**Step 1: Add deprecation annotations** - -In `src/export.rs`, add `#[deprecated]` to both functions: - -```rust -#[deprecated( - since = "0.2.0", - note = "Use ReductionGraph::resolve_path() for variant-aware overhead lookup" -)] -pub fn lookup_overhead(source_name: &str, target_name: &str) -> Option<ReductionOverhead> { - // ... unchanged body ... -} - -#[deprecated( - since = "0.2.0", - note = "Use ReductionGraph::resolve_path() for variant-aware overhead lookup" -)] -pub fn lookup_overhead_or_empty(source_name: &str, target_name: &str) -> ReductionOverhead { - lookup_overhead(source_name, target_name).unwrap_or_default() -} -``` - -**Step 2: Migrate the KSat example** - -In `examples/reduction_ksatisfiability_to_qubo.rs`, replace the `lookup_overhead` call (line 120-121) with `resolve_path`: - -```rust - // Resolve variant-aware overhead via resolve_path - let rg = problemreductions::rules::graph::ReductionGraph::new(); - let name_path = rg - .find_shortest_path_by_name("KSatisfiability", "QUBO") - .expect("KSatisfiability -> QUBO path not found"); - let source_variant = variant_to_map(KSatisfiability::<3>::variant()) - .into_iter() - .map(|(k, v)| (k, v)) - .collect::<std::collections::BTreeMap<_, _>>(); - let target_variant = variant_to_map(QUBO::<f64>::variant()) - .into_iter() - .map(|(k, v)| (k, v)) - .collect::<std::collections::BTreeMap<_, _>>(); - let resolved = rg - .resolve_path(&name_path, &source_variant, &target_variant) - .expect("Failed to resolve KSatisfiability -> QUBO path"); - // Extract overhead from the reduction edge - let overhead = match resolved.edges.iter().find_map(|e| match e { - problemreductions::rules::graph::EdgeKind::Reduction { overhead } => Some(overhead), - _ => None, - }) { - Some(o) => o.clone(), - None => panic!("Resolved path has no reduction edge"), - }; -``` - -**Step 3: Verify the example still compiles and runs** - -Run: `cargo build --example reduction_ksatisfiability_to_qubo --features ilp` -Expected: Builds (deprecation warnings for other examples are OK). - -Run: `cargo run --example reduction_ksatisfiability_to_qubo --features ilp` -Expected: Runs successfully, produces JSON output. - -**Step 4: Commit** - -```bash -git add src/export.rs examples/reduction_ksatisfiability_to_qubo.rs -git commit -m "refactor: deprecate lookup_overhead, migrate KSat example to resolve_path" -``` - ---- - -### Task 5: Remove `impl_natural_reduction!` invocation from `natural.rs` - -Now that `resolve_path` inserts natural casts automatically, the explicit natural reduction registration is no longer needed for planning. Remove it and update the test. - -**Files:** -- Modify: `src/rules/natural.rs` (remove invocation) -- Modify: `src/unit_tests/rules/natural.rs` (update test to use resolve_path instead) - -**Step 1: Update the test to verify natural casts via resolve_path** - -Replace `src/unit_tests/rules/natural.rs` contents: - -```rust -use crate::models::graph::MaximumIndependentSet; -use crate::rules::graph::{EdgeKind, ReductionGraph}; -use crate::topology::{SimpleGraph, Triangular}; -use crate::traits::Problem; -use std::collections::BTreeMap; - -#[test] -fn test_natural_cast_triangular_to_simple_via_resolve() { - let graph = ReductionGraph::new(); - - // Find any path from MIS to itself (via VC round-trip) to test natural cast insertion - // Instead, directly test that resolve_path inserts a natural cast for MIS(Triangular)→VC(SimpleGraph) - let name_path = graph - .find_shortest_path::< - MaximumIndependentSet<Triangular, i32>, - crate::models::graph::MinimumVertexCover<SimpleGraph, i32>, - >() - .unwrap(); - - let source_variant = BTreeMap::from([ - ("graph".to_string(), "Triangular".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - let target_variant = BTreeMap::from([ - ("graph".to_string(), "SimpleGraph".to_string()), - ("weight".to_string(), "i32".to_string()), - ]); - - let resolved = graph - .resolve_path(&name_path, &source_variant, &target_variant) - .unwrap(); - - // Path should be: MIS(Triangular) --NaturalCast--> MIS(SimpleGraph) --Reduction--> VC(SimpleGraph) - assert_eq!(resolved.num_casts(), 1); - assert_eq!(resolved.num_reductions(), 1); - assert!(matches!(resolved.edges[0], EdgeKind::NaturalCast)); - assert!(matches!(resolved.edges[1], EdgeKind::Reduction { .. })); - assert_eq!( - resolved.steps[0].variant.get("graph").unwrap(), - "Triangular" - ); - assert_eq!( - resolved.steps[1].variant.get("graph").unwrap(), - "SimpleGraph" - ); -} -``` - -**Step 2: Remove the `impl_natural_reduction!` invocation** - -Update `src/rules/natural.rs` to: - -```rust -//! Natural-edge reductions via graph subtype relaxation. -//! -//! Natural reductions (e.g., a problem on `Triangular` solved as `SimpleGraph`) -//! are handled automatically by [`ReductionGraph::resolve_path`], which inserts -//! `NaturalCast` steps based on the registered graph/weight subtype hierarchies. -//! -//! No explicit `ReduceTo` impls are needed for natural edges — the resolver -//! computes them from `GraphSubtypeEntry` and `WeightSubtypeEntry` registrations. - -#[cfg(test)] -#[path = "../unit_tests/rules/natural.rs"] -mod tests; -``` - -**Step 3: Run the test** - -Run: `cargo test test_natural_cast_triangular_to_simple_via_resolve` -Expected: PASS - -**Step 4: Run full test suite to check nothing broke** - -Run: `make test clippy` -Expected: PASS (some deprecation warnings for examples still using `lookup_overhead` are OK) - -**Step 5: Commit** - -```bash -git add src/rules/natural.rs src/unit_tests/rules/natural.rs -git commit -m "refactor: remove explicit natural reduction, rely on resolve_path" -``` - ---- - -### Task 6: Run full verification - -**Files:** None (verification only) - -**Step 1: Run full check** - -Run: `make check` -Expected: fmt, clippy, and all tests pass. - -**Step 2: Run doc build** - -Run: `cargo doc --no-deps -p problemreductions` -Expected: No warnings. - -**Step 3: Run examples that use lookup_overhead (should still work via deprecation)** - -Run: `cargo build --examples --features ilp` -Expected: Builds with deprecation warnings but no errors. - -**Step 4: Commit any fixups if needed, then final commit message** - -```bash -git add -A -git commit -m "chore: final cleanup for variant-aware reduction paths" -``` diff --git a/docs/plans/2026-02-14-variant-system-redesign.md b/docs/plans/2026-02-14-variant-system-redesign.md deleted file mode 100644 index 204dbf3f1..000000000 --- a/docs/plans/2026-02-14-variant-system-redesign.md +++ /dev/null @@ -1,1160 +0,0 @@ -# Robust Variant System Redesign — Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. -> **Save location:** Copy this plan to `docs/plans/2026-02-14-variant-system-redesign.md` before starting implementation. - -**Goal:** Replace the ad-hoc variant system with a unified `VariantParam` trait where graph types, weight types, and K values all self-declare their category, hierarchy position, and parent cast — eliminating hardcoded key matching, dead code, and const generic special-casing. - -**Architecture:** Three new traits (`VariantParam`, `CastToParent`, `KValue`), two `macro_rules!` macros (`impl_variant_param!`, `variant_params!`), and two new graph types (`KingsSubgraph`, `TriangularSubgraph`). The `ReductionGraph` discovers the full hierarchy from `VariantTypeEntry` inventory registrations at runtime. - -**Tech Stack:** Rust, `inventory` crate (already a dependency), `macro_rules!` (no proc macro changes needed beyond cleanup) - ---- - -## Design Decisions - -| Decision | Choice | -|----------|--------| -| Macro approach | `macro_rules!` only (no proc macro) | -| VariantParam coupling | Standalone impls (NOT supertrait of Graph/WeightElement) | -| Type parameter renaming | Skip — keys come from `VariantParam::CATEGORY` | -| Variant keys | Keep current lowercase: `"graph"`, `"weight"`, `"k"` | -| Hierarchy shape | Single-parent tree (transitivity computed by walking chain) | -| GridGraph/Triangular | Internal-only (not in public variant hierarchy) | -| New public graph types | `KingsSubgraph`, `TriangularSubgraph` (non-generic, subtype UnitDiskGraph) | -| Runtime casts | Required — `CastToParent` trait declared alongside hierarchy | -| VALUE derivation | From `stringify!($ty)` — no explicit VALUE argument needed | -| K values | Replace `const K: usize` with `K: KValue` type parameter (K2, K3, KN) | - ---- - -## Task 1: Core Variant Infrastructure - -**Files:** -- Modify: `src/variant.rs` -- Test: `src/unit_tests/variant.rs` - -### Step 1: Write failing tests for VariantParam trait and macros - -Add to `src/unit_tests/variant.rs`: - -```rust -use crate::variant::{CastToParent, VariantParam, VariantTypeEntry}; - -// Test types for the new system -#[derive(Clone, Debug)] -struct TestRoot; -#[derive(Clone, Debug)] -struct TestChild; - -impl_variant_param!(TestRoot, "test_cat"); -impl_variant_param!(TestChild, "test_cat", parent: TestRoot, cast: |_| TestRoot); - -#[test] -fn test_variant_param_root() { - assert_eq!(TestRoot::CATEGORY, "test_cat"); - assert_eq!(TestRoot::VALUE, "TestRoot"); - assert_eq!(TestRoot::PARENT_VALUE, None); -} - -#[test] -fn test_variant_param_child() { - assert_eq!(TestChild::CATEGORY, "test_cat"); - assert_eq!(TestChild::VALUE, "TestChild"); - assert_eq!(TestChild::PARENT_VALUE, Some("TestRoot")); -} - -#[test] -fn test_cast_to_parent() { - let child = TestChild; - let _parent: TestRoot = child.cast_to_parent(); -} - -#[test] -fn test_variant_type_entry_registered() { - let entries: Vec<_> = inventory::iter::<VariantTypeEntry>() - .filter(|e| e.category == "test_cat") - .collect(); - assert!(entries.iter().any(|e| e.value == "TestRoot" && e.parent.is_none())); - assert!(entries.iter().any(|e| e.value == "TestChild" && e.parent == Some("TestRoot"))); -} - -#[derive(Clone, Debug)] -struct TestKRoot; -#[derive(Clone, Debug)] -struct TestKChild; - -impl_variant_param!(TestKRoot, "test_k", k: None); -impl_variant_param!(TestKChild, "test_k", parent: TestKRoot, cast: |_| TestKRoot, k: Some(3)); - -#[test] -fn test_kvalue_via_macro_root() { - assert_eq!(TestKRoot::CATEGORY, "test_k"); - assert_eq!(TestKRoot::VALUE, "TestKRoot"); - assert_eq!(TestKRoot::PARENT_VALUE, None); - assert_eq!(TestKRoot::K, None); -} - -#[test] -fn test_kvalue_via_macro_child() { - assert_eq!(TestKChild::CATEGORY, "test_k"); - assert_eq!(TestKChild::VALUE, "TestKChild"); - assert_eq!(TestKChild::PARENT_VALUE, Some("TestKRoot")); - assert_eq!(TestKChild::K, Some(3)); -} - -#[test] -fn test_variant_params_macro_empty() { - let v: Vec<(&str, &str)> = variant_params![]; - assert!(v.is_empty()); -} - -#[test] -fn test_variant_params_macro_single() { - fn check<T: VariantParam>() -> Vec<(&'static str, &'static str)> { - variant_params![T] - } - let v = check::<TestRoot>(); - assert_eq!(v, vec![("test_cat", "TestRoot")]); -} - -#[test] -fn test_variant_params_macro_multiple() { - fn check<A: VariantParam, B: VariantParam>() -> Vec<(&'static str, &'static str)> { - variant_params![A, B] - } - let v = check::<TestRoot, TestChild>(); - assert_eq!(v, vec![("test_cat", "TestRoot"), ("test_cat", "TestChild")]); -} -``` - -### Step 2: Run tests to verify they fail - -Run: `cargo test --lib variant -- --test-output` -Expected: Compilation errors — `VariantParam`, `CastToParent`, `VariantTypeEntry`, `impl_variant_param!`, `variant_params!` don't exist yet. - -### Step 3: Implement VariantParam, CastToParent, VariantTypeEntry, and macros - -Replace `src/variant.rs` contents with: - -```rust -//! Variant system for type-level problem parameterization. -//! -//! Types declare their variant category, value, and parent via `VariantParam`. -//! The `impl_variant_param!` macro registers types with both the trait and -//! the runtime `VariantTypeEntry` inventory. The `variant_params!` macro -//! composes `Problem::variant()` bodies from type parameter names. - -/// A type that participates in the variant system. -/// -/// Declares its category (e.g., `"graph"`), value (e.g., `"SimpleGraph"`), -/// and optional parent in the subtype hierarchy. -pub trait VariantParam: 'static { - /// Category name (e.g., `"graph"`, `"weight"`, `"k"`). - const CATEGORY: &'static str; - /// Type name within the category (e.g., `"SimpleGraph"`, `"i32"`). - const VALUE: &'static str; - /// Parent type name in the subtype hierarchy, or `None` for root types. - const PARENT_VALUE: Option<&'static str>; -} - -/// Types that can convert themselves to their parent in the variant hierarchy. -pub trait CastToParent: VariantParam { - /// The parent type. - type Parent: VariantParam; - /// Convert this value to its parent type. - fn cast_to_parent(&self) -> Self::Parent; -} - -/// Runtime-discoverable variant type registration. -/// -/// Built by `impl_variant_param!` macro, collected by `inventory`. -pub struct VariantTypeEntry { - pub category: &'static str, - pub value: &'static str, - pub parent: Option<&'static str>, -} - -inventory::collect!(VariantTypeEntry); - -/// Implement `VariantParam` (and optionally `CastToParent`) for a type, -/// and register a `VariantTypeEntry` with inventory. -/// -/// # Usage -/// -/// ```rust,ignore -/// // Root type (no parent): -/// impl_variant_param!(SimpleGraph, "graph"); -/// -/// // Type with parent — cast closure required: -/// impl_variant_param!(UnitDiskGraph, "graph", parent: SimpleGraph, -/// cast: |g| SimpleGraph::new(g.num_vertices(), g.edges())); -/// ``` -#[macro_export] -macro_rules! impl_variant_param { - // Root type (no parent, no cast) - ($ty:ty, $cat:expr) => { - impl $crate::variant::VariantParam for $ty { - const CATEGORY: &'static str = $cat; - const VALUE: &'static str = stringify!($ty); - const PARENT_VALUE: Option<&'static str> = None; - } - ::inventory::submit! { - $crate::variant::VariantTypeEntry { - category: $cat, - value: stringify!($ty), - parent: None, - } - } - }; - // Type with parent + cast closure - ($ty:ty, $cat:expr, parent: $parent:ty, cast: $cast:expr) => { - impl $crate::variant::VariantParam for $ty { - const CATEGORY: &'static str = $cat; - const VALUE: &'static str = stringify!($ty); - const PARENT_VALUE: Option<&'static str> = Some(stringify!($parent)); - } - impl $crate::variant::CastToParent for $ty { - type Parent = $parent; - fn cast_to_parent(&self) -> $parent { - let f: fn(&$ty) -> $parent = $cast; - f(self) - } - } - ::inventory::submit! { - $crate::variant::VariantTypeEntry { - category: $cat, - value: stringify!($ty), - parent: Some(stringify!($parent)), - } - } - }; - // KValue root type (no parent, with k value) - ($ty:ty, $cat:expr, k: $k:expr) => { - $crate::impl_variant_param!($ty, $cat); - impl $crate::variant::KValue for $ty { - const K: Option<usize> = $k; - } - }; - // KValue type with parent + cast + k value - ($ty:ty, $cat:expr, parent: $parent:ty, cast: $cast:expr, k: $k:expr) => { - $crate::impl_variant_param!($ty, $cat, parent: $parent, cast: $cast); - impl $crate::variant::KValue for $ty { - const K: Option<usize> = $k; - } - }; -} - -/// Compose a `Problem::variant()` body from type parameter names. -/// -/// All variant dimensions must be types implementing `VariantParam`. -/// -/// # Usage -/// -/// ```rust,ignore -/// variant_params![] // → vec![] -/// variant_params![G, W] // → vec![(G::CATEGORY, G::VALUE), ...] -/// ``` -#[macro_export] -macro_rules! variant_params { - () => { vec![] }; - ($($T:ident),+) => { - vec![$((<$T as $crate::variant::VariantParam>::CATEGORY, - <$T as $crate::variant::VariantParam>::VALUE)),+] - }; -} - -#[cfg(test)] -#[path = "unit_tests/variant.rs"] -mod tests; -``` - -### Step 4: Run tests to verify they pass - -Run: `cargo test --lib variant` -Expected: All new tests PASS. Old tests for `short_type_name` and `const_usize_str` still pass (we haven't removed them yet). - -### Step 5: Commit - -```bash -git add src/variant.rs src/unit_tests/variant.rs -git commit -m "feat: add VariantParam trait, CastToParent, impl_variant_param!, variant_params! macros" -``` - ---- - -## Task 2: KValue Types - -**Files:** -- Modify: `src/variant.rs` -- Test: `src/unit_tests/variant.rs` - -### Step 1: Write failing tests for KValue types - -Add to `src/unit_tests/variant.rs`: - -```rust -use crate::variant::{K2, K3, KN, KValue}; - -#[test] -fn test_kvalue_k2() { - assert_eq!(K2::CATEGORY, "k"); - assert_eq!(K2::VALUE, "K2"); - assert_eq!(K2::PARENT_VALUE, Some("K3")); - assert_eq!(K2::K, Some(2)); -} - -#[test] -fn test_kvalue_k3() { - assert_eq!(K3::CATEGORY, "k"); - assert_eq!(K3::VALUE, "K3"); - assert_eq!(K3::PARENT_VALUE, Some("KN")); - assert_eq!(K3::K, Some(3)); -} - -#[test] -fn test_kvalue_kn() { - assert_eq!(KN::CATEGORY, "k"); - assert_eq!(KN::VALUE, "KN"); - assert_eq!(KN::PARENT_VALUE, None); - assert_eq!(KN::K, None); -} - -#[test] -fn test_kvalue_cast_chain() { - let k2 = K2; - let k3: K3 = k2.cast_to_parent(); - let kn: KN = k3.cast_to_parent(); - assert_eq!(KN::K, None); - let _ = kn; // use it -} - -#[test] -fn test_kvalue_variant_entries() { - let entries: Vec<_> = inventory::iter::<VariantTypeEntry>() - .filter(|e| e.category == "k") - .collect(); - assert!(entries.iter().any(|e| e.value == "KN" && e.parent.is_none())); - assert!(entries.iter().any(|e| e.value == "K3" && e.parent == Some("KN"))); - assert!(entries.iter().any(|e| e.value == "K2" && e.parent == Some("K3"))); -} -``` - -### Step 2: Run tests to verify they fail - -Run: `cargo test --lib variant::tests::test_kvalue` -Expected: Compilation errors — `KValue`, `K2`, `K3`, `KN` don't exist yet. - -### Step 3: Implement KValue trait and types - -Add to `src/variant.rs` (before the `#[cfg(test)]` block): - -```rust -/// Trait for K-value types used in KSatisfiability and KColoring. -/// -/// Each type represents a specific K value (K2=2, K3=3, etc.) or -/// the generic case (KN = any K). Hierarchy: K2 < K3 < KN. -/// -/// Use `impl_variant_param!` with the `k:` argument to implement this trait: -/// ```rust,ignore -/// impl_variant_param!(K3, "k", parent: KN, cast: |_| KN, k: Some(3)); -/// ``` -pub trait KValue: VariantParam + Clone + 'static { - /// The concrete K value, or `None` for the generic case (KN). - const K: Option<usize>; -} - -/// K=2 (e.g., 2-SAT, 2-coloring). -#[derive(Clone, Copy, Debug, Default)] -pub struct K2; - -/// K=3 (e.g., 3-SAT, 3-coloring). -#[derive(Clone, Copy, Debug, Default)] -pub struct K3; - -/// Generic K (any value). Used for reductions that apply to all K. -#[derive(Clone, Copy, Debug, Default)] -pub struct KN; - -impl_variant_param!(KN, "k", k: None); -impl_variant_param!(K3, "k", parent: KN, cast: |_| KN, k: Some(3)); -impl_variant_param!(K2, "k", parent: K3, cast: |_| K3, k: Some(2)); -``` - -### Step 4: Run tests to verify they pass - -Run: `cargo test --lib variant` -Expected: All KValue tests PASS. - -### Step 5: Commit - -```bash -git add src/variant.rs src/unit_tests/variant.rs -git commit -m "feat: add KValue trait with K2, K3, KN types for type-level K values" -``` - ---- - -## Task 3: Register Graph Types with VariantParam - -**Files:** -- Modify: `src/topology/graph.rs` (SimpleGraph) -- Modify: `src/topology/unit_disk_graph.rs` (UnitDiskGraph) -- Modify: `src/topology/hypergraph.rs` (HyperGraph) -- Test: `src/unit_tests/variant.rs` - -### Step 1: Write failing tests - -Add to `src/unit_tests/variant.rs`: - -```rust -use crate::topology::{Graph, SimpleGraph, UnitDiskGraph}; -use crate::topology::HyperGraph; - -#[test] -fn test_simple_graph_variant_param() { - assert_eq!(SimpleGraph::CATEGORY, "graph"); - assert_eq!(SimpleGraph::VALUE, "SimpleGraph"); - assert_eq!(SimpleGraph::PARENT_VALUE, Some("HyperGraph")); -} - -#[test] -fn test_unit_disk_graph_variant_param() { - assert_eq!(UnitDiskGraph::CATEGORY, "graph"); - assert_eq!(UnitDiskGraph::VALUE, "UnitDiskGraph"); - assert_eq!(UnitDiskGraph::PARENT_VALUE, Some("SimpleGraph")); -} - -#[test] -fn test_hyper_graph_variant_param() { - assert_eq!(HyperGraph::CATEGORY, "graph"); - assert_eq!(HyperGraph::VALUE, "HyperGraph"); - assert_eq!(HyperGraph::PARENT_VALUE, None); -} - -#[test] -fn test_graph_variant_entries() { - let entries: Vec<_> = inventory::iter::<VariantTypeEntry>() - .filter(|e| e.category == "graph") - .collect(); - assert!(entries.iter().any(|e| e.value == "HyperGraph" && e.parent.is_none())); - assert!(entries.iter().any(|e| e.value == "SimpleGraph" && e.parent == Some("HyperGraph"))); - assert!(entries.iter().any(|e| e.value == "UnitDiskGraph" && e.parent == Some("SimpleGraph"))); -} - -#[test] -fn test_simple_graph_cast_to_parent() { - let sg = SimpleGraph::new(3, vec![(0, 1), (1, 2)]); - let hg: HyperGraph = sg.cast_to_parent(); - assert_eq!(hg.num_vertices(), 3); - assert_eq!(hg.num_edges(), 2); -} - -#[test] -fn test_udg_cast_to_parent() { - let udg = UnitDiskGraph::new(vec![(0.0, 0.0), (0.5, 0.0), (2.0, 0.0)], 1.0); - let sg: SimpleGraph = udg.cast_to_parent(); - assert_eq!(sg.num_vertices(), 3); - // Only the first two points are within distance 1.0 - assert!(sg.has_edge(0, 1)); - assert!(!sg.has_edge(0, 2)); -} -``` - -### Step 2: Run tests to verify they fail - -Run: `cargo test --lib variant::tests::test_simple_graph` -Expected: Compilation errors — graph types don't implement `VariantParam`. - -### Step 3: Add impl_variant_param! to each graph type file - -In `src/topology/hypergraph.rs`, add at the end (before any `#[cfg(test)]`): -```rust -use crate::impl_variant_param; -impl_variant_param!(HyperGraph, "graph"); -``` - -In `src/topology/graph.rs`, add after the `SimpleGraph` impl of `Graph`: -```rust -use crate::impl_variant_param; -impl_variant_param!(SimpleGraph, "graph", parent: HyperGraph, - cast: |g| HyperGraph::from_graph_edges(g.num_vertices(), g.edges())); -``` - -In `src/topology/unit_disk_graph.rs`, add after the `UnitDiskGraph` impl of `Graph`: -```rust -use crate::impl_variant_param; -impl_variant_param!(UnitDiskGraph, "graph", parent: SimpleGraph, - cast: |g| SimpleGraph::new(g.num_vertices(), g.edges())); -``` - -Note: `HyperGraph::from_graph_edges` may need to be implemented (or use existing constructor). Check HyperGraph API and adapt the cast closure. - -### Step 4: Run tests to verify they pass - -Run: `cargo test --lib variant` -Expected: All graph variant tests PASS. - -### Step 5: Commit - -```bash -git add src/topology/graph.rs src/topology/unit_disk_graph.rs src/topology/hypergraph.rs src/unit_tests/variant.rs -git commit -m "feat: register SimpleGraph, UnitDiskGraph, HyperGraph with VariantParam" -``` - ---- - -## Task 4: Register Weight Types with VariantParam - -**Files:** -- Modify: `src/types.rs` -- Test: `src/unit_tests/variant.rs` - -### Step 1: Write failing tests - -Add to `src/unit_tests/variant.rs`: - -```rust -use crate::types::One; - -#[test] -fn test_weight_f64_variant_param() { - assert_eq!(<f64 as VariantParam>::CATEGORY, "weight"); - assert_eq!(<f64 as VariantParam>::VALUE, "f64"); - assert_eq!(<f64 as VariantParam>::PARENT_VALUE, None); -} - -#[test] -fn test_weight_i32_variant_param() { - assert_eq!(<i32 as VariantParam>::CATEGORY, "weight"); - assert_eq!(<i32 as VariantParam>::VALUE, "i32"); - assert_eq!(<i32 as VariantParam>::PARENT_VALUE, Some("f64")); -} - -#[test] -fn test_weight_one_variant_param() { - assert_eq!(One::CATEGORY, "weight"); - assert_eq!(One::VALUE, "One"); - assert_eq!(One::PARENT_VALUE, Some("i32")); -} - -#[test] -fn test_weight_cast_chain() { - let one = One; - let i: i32 = one.cast_to_parent(); - assert_eq!(i, 1); - let f: f64 = i.cast_to_parent(); - assert_eq!(f, 1.0); -} - -#[test] -fn test_weight_variant_entries() { - let entries: Vec<_> = inventory::iter::<VariantTypeEntry>() - .filter(|e| e.category == "weight") - .collect(); - assert!(entries.iter().any(|e| e.value == "f64" && e.parent.is_none())); - assert!(entries.iter().any(|e| e.value == "i32" && e.parent == Some("f64"))); - assert!(entries.iter().any(|e| e.value == "One" && e.parent == Some("i32"))); -} -``` - -### Step 2: Implement in `src/types.rs` - -Add at end of `src/types.rs`: - -```rust -use crate::impl_variant_param; - -impl_variant_param!(f64, "weight"); -impl_variant_param!(i32, "weight", parent: f64, cast: |w| *w as f64); -impl_variant_param!(One, "weight", parent: i32, cast: |_| 1i32); -``` - -### Step 3: Run tests - -Run: `cargo test --lib variant` -Expected: All weight variant tests PASS. - -### Step 4: Commit - -```bash -git add src/types.rs src/unit_tests/variant.rs -git commit -m "feat: register One, i32, f64 with VariantParam" -``` - ---- - -## Task 5: Migrate KSatisfiability from const K to KValue - -**Files:** -- Modify: `src/models/satisfiability/ksat.rs` -- Modify: `src/unit_tests/models/satisfiability/ksat.rs` -- Modify: `src/rules/sat_ksat.rs` -- Modify: `src/rules/ksatisfiability_qubo.rs` -- Modify: `src/prelude.rs` (in `src/lib.rs`) -- Test: existing ksat tests - -### Step 1: Migrate KSatisfiability struct - -In `src/models/satisfiability/ksat.rs`, change: - -```rust -// Before: -pub struct KSatisfiability<const K: usize> { ... } - -// After: -use crate::variant::{KValue, VariantParam}; - -pub struct KSatisfiability<K: KValue> { - num_vars: usize, - clauses: Vec<CNFClause>, - _phantom: std::marker::PhantomData<K>, -} -``` - -Update all methods: replace `K` (const value) with `K::K.expect("KN cannot be instantiated")` or appropriate access. The `new()` method validates clause length using `K::K.unwrap()`. - -Update `Problem` impl: - -```rust -impl<K: KValue> Problem for KSatisfiability<K> { - const NAME: &'static str = "KSatisfiability"; - type Metric = bool; - - fn variant() -> Vec<(&'static str, &'static str)> { - variant_params![K] - } - - fn dims(&self) -> Vec<usize> { - vec![2; self.num_vars] - } - // ... evaluate unchanged (uses self.clauses) -} -``` - -### Step 2: Update all KSatisfiability usages - -Search-and-replace across the codebase: -- `KSatisfiability::<3>` → `KSatisfiability::<K3>` -- `KSatisfiability::<2>` → `KSatisfiability::<K2>` -- `KSatisfiability<3>` → `KSatisfiability<K3>` -- `KSatisfiability<2>` → `KSatisfiability<K2>` -- `const K: usize` in sat_ksat.rs reduction structs → `K: KValue` - -Add `use crate::variant::{K2, K3, KN};` where needed. - -### Step 3: Update reduction rules - -In `src/rules/sat_ksat.rs`: -- `ReductionSATToKSAT<const K: usize>` → `ReductionSATToKSAT<K: KValue>` -- `ReductionKSATToSAT<const K: usize>` → `ReductionKSATToSAT<K: KValue>` -- `impl_sat_to_ksat!(3)` → concrete impl for K3 - -In `src/rules/ksatisfiability_qubo.rs`: -- Update similarly - -### Step 4: Update prelude in `src/lib.rs` - -Add variant system exports to the prelude: -```rust -pub use crate::variant::{CastToParent, KValue, VariantParam, K2, K3, KN}; -``` -Note: `impl_variant_param!` and `variant_params!` are `#[macro_export]` so they're automatically available at the crate root (`problemreductions::impl_variant_param!`). - -### Step 5: Run tests - -Run: `cargo test --lib models::satisfiability::ksat && cargo test --lib rules::sat_ksat` -Expected: All existing ksat tests PASS with new type parameter syntax. - -### Step 6: Commit - -```bash -git add src/models/satisfiability/ksat.rs src/rules/sat_ksat.rs src/rules/ksatisfiability_qubo.rs src/lib.rs src/unit_tests/ -git commit -m "refactor: migrate KSatisfiability from const K to KValue type parameter" -``` - ---- - -## Task 6: Migrate KColoring from const K to KValue - -**Files:** -- Modify: `src/models/graph/kcoloring.rs` -- Modify: `src/unit_tests/models/graph/kcoloring.rs` -- Modify: `src/rules/sat_coloring.rs` -- Modify: `src/rules/coloring_qubo.rs` - -### Step 1: Migrate KColoring struct - -Same pattern as Task 5. Change `KColoring<const K: usize, G>` to `KColoring<K: KValue, G>`. - -```rust -impl<K: KValue, G> Problem for KColoring<K, G> -where G: Graph + VariantParam { - fn variant() -> Vec<(&'static str, &'static str)> { - variant_params![K, G] - } - fn dims(&self) -> Vec<usize> { - vec![K::K.expect("KN cannot be used as problem instance"); self.num_vertices()] - } -} -``` - -### Step 2: Update all KColoring usages - -- `KColoring::<3, SimpleGraph>` → `KColoring::<K3, SimpleGraph>` -- `KColoring::<2, SimpleGraph>` → `KColoring::<K2, SimpleGraph>` -- `KColoring::<4, SimpleGraph>` → add K4 type if needed, or use KN -- `KColoring::<1, SimpleGraph>` → add K1 type if needed - -Note: Tests use K=1, K=2, K=3, K=4. Add `K1` and `K4` to `src/variant.rs`: - -```rust -#[derive(Clone, Copy, Debug, Default)] -pub struct K1; -#[derive(Clone, Copy, Debug, Default)] -pub struct K4; - -impl_variant_param!(K1, "k", parent: K2, cast: |_| K2, k: Some(1)); -impl_variant_param!(K4, "k", parent: KN, cast: |_| KN, k: Some(4)); -``` - -Update hierarchy: K1 < K2 < K3 < K4 < KN. Adjust K3's parent to K4. - -### Step 3: Run tests - -Run: `cargo test --lib models::graph::kcoloring && cargo test --lib rules::sat_coloring` -Expected: All PASS. - -### Step 4: Commit - -```bash -git add src/models/graph/kcoloring.rs src/rules/sat_coloring.rs src/rules/coloring_qubo.rs src/variant.rs src/unit_tests/ -git commit -m "refactor: migrate KColoring from const K to KValue type parameter" -``` - ---- - -## Task 7: Apply variant_params! to All Problem Impls - -**Files:** -- Modify: 21 model files in `src/models/**/*.rs` -- Modify: 10 test Problem types in `src/unit_tests/` - -### Step 1: Add VariantParam bounds and variant_params! to graph+weight problems (9 files) - -For each of: `maximum_independent_set.rs`, `minimum_vertex_cover.rs`, `minimum_dominating_set.rs`, `maximum_clique.rs`, `maximum_matching.rs`, `max_cut.rs`, `maximal_is.rs`, `traveling_salesman.rs`, `spin_glass.rs`: - -```rust -// Before: -impl<G, W> Problem for MaximumIndependentSet<G, W> -where G: Graph, W: WeightElement { - fn variant() -> Vec<(&'static str, &'static str)> { - vec![("graph", G::NAME), ("weight", crate::variant::short_type_name::<W>())] - } -} - -// After: -use crate::variant::VariantParam; - -impl<G, W> Problem for MaximumIndependentSet<G, W> -where G: Graph + VariantParam, W: WeightElement + VariantParam { - fn variant() -> Vec<(&'static str, &'static str)> { - variant_params![G, W] - } -} -``` - -### Step 2: Weight-only problems (3 files) - -For `qubo.rs`, `maximum_set_packing.rs`, `minimum_set_covering.rs`: - -```rust -where W: WeightElement + VariantParam { - fn variant() -> ... { variant_params![W] } -} -``` - -### Step 3: No-generic problems (7 files) - -For `ilp.rs`, `sat.rs`, `circuit.rs`, `factoring.rs`, `biclique_cover.rs`, `bmf.rs`, `paintshop.rs`: - -```rust -fn variant() -> ... { variant_params![] } -``` - -### Step 4: Update test Problem types - -In `src/unit_tests/traits.rs`, `src/unit_tests/solvers/brute_force.rs`, `src/unit_tests/rules/traits.rs`: - -Add `impl VariantParam` for each test-only Problem type. Since they use hardcoded variants, keep them as direct impls: - -```rust -impl VariantParam for TestSatProblem { - const CATEGORY: &'static str = "test"; - const VALUE: &'static str = "TestSatProblem"; - const PARENT_VALUE: Option<&'static str> = None; -} -``` - -Or use `variant_params![]` if the test doesn't need specific variant values. - -### Step 5: Run all tests - -Run: `make test` -Expected: All tests PASS. - -### Step 6: Commit - -```bash -git add src/models/ src/unit_tests/ -git commit -m "refactor: apply variant_params! macro to all Problem implementations" -``` - ---- - -## Task 8: New Public Graph Types (KingsSubgraph, TriangularSubgraph) - -**Files:** -- Create: `src/topology/kings_subgraph.rs` -- Create: `src/topology/triangular_subgraph.rs` -- Create: `src/unit_tests/topology/kings_subgraph.rs` -- Create: `src/unit_tests/topology/triangular_subgraph.rs` -- Modify: `src/topology/mod.rs` - -### Step 1: Implement KingsSubgraph - -Create `src/topology/kings_subgraph.rs`: - -```rust -//! KingsSubgraph: a non-generic square-grid unit disk subgraph. - -use super::graph::{Graph, SimpleGraph}; -use crate::impl_variant_param; -use serde::{Deserialize, Serialize}; - -/// Non-generic graph for square-grid unit disk subgraphs. -/// -/// Stores node positions and precomputed edges. Weights are NOT stored -/// here — they belong to the Problem that uses this graph. -#[derive(Clone, Debug, Serialize, Deserialize)] -pub struct KingsSubgraph { - positions: Vec<(i32, i32)>, - size: (usize, usize), - radius: f64, - edges: Vec<(usize, usize)>, -} - -impl KingsSubgraph { - /// Create from node positions and radius, computing edges. - pub fn new(positions: Vec<(i32, i32)>, size: (usize, usize), radius: f64) -> Self { - let mut edges = Vec::new(); - for i in 0..positions.len() { - for j in (i + 1)..positions.len() { - let (r1, c1) = positions[i]; - let (r2, c2) = positions[j]; - let dist = (((r1 - r2) as f64).powi(2) + ((c1 - c2) as f64).powi(2)).sqrt(); - if dist < radius { - edges.push((i, j)); - } - } - } - Self { positions, size, radius, edges } - } - - pub fn positions(&self) -> &[(i32, i32)] { &self.positions } - pub fn grid_size(&self) -> (usize, usize) { self.size } - pub fn radius(&self) -> f64 { self.radius } -} - -impl Graph for KingsSubgraph { - const NAME: &'static str = "KingsSubgraph"; - - fn num_vertices(&self) -> usize { self.positions.len() } - fn num_edges(&self) -> usize { self.edges.len() } - fn edges(&self) -> Vec<(usize, usize)> { self.edges.clone() } - - fn has_edge(&self, u: usize, v: usize) -> bool { - let (a, b) = if u < v { (u, v) } else { (v, u) }; - self.edges.iter().any(|&(x, y)| x == a && y == b) - } - - fn neighbors(&self, v: usize) -> Vec<usize> { - self.edges.iter().filter_map(|&(a, b)| { - if a == v { Some(b) } else if b == v { Some(a) } else { None } - }).collect() - } -} - -impl_variant_param!(KingsSubgraph, "graph", parent: UnitDiskGraph, - cast: |g| { - use super::unit_disk_graph::UnitDiskGraph; - let positions: Vec<(f64, f64)> = g.positions().iter() - .map(|&(r, c)| (r as f64, c as f64)) - .collect(); - UnitDiskGraph::new(positions, g.radius()) - }); -``` - -### Step 2: Implement TriangularSubgraph - -Same pattern as KingsSubgraph but with triangular coordinate system. Create `src/topology/triangular_subgraph.rs`. - -### Step 3: Update topology module exports - -In `src/topology/mod.rs`: - -```rust -mod kings_subgraph; -mod triangular_subgraph; - -pub use kings_subgraph::KingsSubgraph; -pub use triangular_subgraph::TriangularSubgraph; -``` - -### Step 4: Write tests, run, commit - -Run: `cargo test --lib topology` -Expected: PASS. - -```bash -git add src/topology/kings_subgraph.rs src/topology/triangular_subgraph.rs src/topology/mod.rs src/unit_tests/topology/ -git commit -m "feat: add KingsSubgraph and TriangularSubgraph graph types" -``` - ---- - -## Task 9: Internal-ize GridGraph/Triangular and Restructure Reductions - -**Files:** -- Modify: `src/rules/maximumindependentset_gridgraph.rs` -- Modify: `src/rules/maximumindependentset_triangular.rs` -- Modify: `src/topology/mod.rs` -- Modify: `src/graph_types.rs` - -### Step 1: Remove `#[reduction]` from GridGraph/Triangular reduction files - -The reduction code stays but is no longer registered in the variant graph. Internal unitdiskmapping still uses GridGraph/Triangular. - -### Step 2: Add new registered reductions for KingsSubgraph/TriangularSubgraph - -Restructure reduction files to output `MIS<KingsSubgraph, i32>` and `MIS<TriangularSubgraph, i32>` instead, converting from internal GridGraph results. - -### Step 3: Make GridGraph/Triangular `pub(crate)` in topology - -Update `src/topology/mod.rs` to use `pub(crate) use` for GridGraph and Triangular. - -### Step 4: Remove GridGraph/Triangular from graph_types.rs markers - -Remove `declare_graph_subtype!` entries for GridGraph and Triangular. - -### Step 5: Run tests, commit - -Run: `make test clippy` - -```bash -git commit -m "refactor: internal-ize GridGraph/Triangular, add KingsSubgraph/TriangularSubgraph reductions" -``` - ---- - -## Task 10: Unify Hierarchy in ReductionGraph - -**Files:** -- Modify: `src/rules/graph.rs` -- Modify: `src/unit_tests/rules/graph.rs` - -### Step 1: Replace graph_hierarchy/weight_hierarchy with variant_hierarchy - -In `src/rules/graph.rs`, replace the two separate hierarchy fields with: - -```rust -variant_hierarchy: HashMap<String, HashMap<String, Option<String>>>, -``` - -Build from `VariantTypeEntry` inventory. - -### Step 2: Generalize is_variant_reducible() - -Replace the hardcoded match on key names with generic parent-chain walk: - -```rust -fn is_subtype_in(&self, category_types: &HashMap<String, Option<String>>, a: &str, b: &str) -> bool { - if a == b { return true; } - let mut current = a; - loop { - match category_types.get(current) { - Some(Some(parent)) => { - if parent == b { return true; } - current = parent; - } - _ => return false, - } - } -} -``` - -### Step 3: Remove old hierarchy code - -Delete `is_graph_subtype`, `is_weight_subtype`, `is_const_subtype`, `graph_hierarchy`, `weight_hierarchy`. - -### Step 4: Run tests, commit - -Run: `cargo test --lib rules::graph` - -```bash -git commit -m "refactor: unify variant hierarchy in ReductionGraph using VariantTypeEntry" -``` - ---- - -## Task 11: Remove Old Hierarchy System - -**Files:** -- Modify: `src/graph_types.rs` -- Modify: `src/unit_tests/graph_types.rs` - -### Step 1: Remove from graph_types.rs - -Delete: -- `GraphSubtypeEntry`, `WeightSubtypeEntry` structs + inventory collections -- `declare_graph_subtype!`, `declare_weight_subtype!` macros + all invocations -- `GraphSubtype<G>` trait -- `GraphMarker` trait (verify no other usages first) - -Keep: ZST marker structs (SimpleGraph, UnitDiskGraph, etc.) if used elsewhere, OR remove if superseded by topology types. - -### Step 2: Update tests - -Remove or rewrite `src/unit_tests/graph_types.rs` tests that reference deleted types. - -### Step 3: Run tests, commit - -Run: `make test clippy` - -```bash -git commit -m "refactor: remove old GraphSubtypeEntry/WeightSubtypeEntry hierarchy system" -``` - ---- - -## Task 12: Cleanup and Remove Deprecated Code - -**Files:** -- Modify: `src/variant.rs` — remove `const_usize_str`, `short_type_name` -- Modify: `problemreductions-macros/src/lib.rs` — remove const generic rewriting logic -- Modify: `src/rules/graph.rs` — remove `is_const_subtype` -- Modify: `src/unit_tests/variant.rs` — remove old tests for deleted functions - -### Step 1: Remove const_usize_str and short_type_name - -These are replaced by `KValue` types and `VariantParam::VALUE`. - -### Step 2: Remove const generic rewriting from proc macro - -In `problemreductions-macros/src/lib.rs`: -- Remove `collect_const_generic_names()` -- Remove `rewrite_const_generics()` -- Simplify `make_variant_fn_body()` — no more const generic handling needed since K is now a type param - -### Step 3: Run full test suite - -Run: `make test clippy` - -```bash -git commit -m "refactor: remove deprecated const_usize_str, short_type_name, const generic rewriting" -``` - ---- - -## Task 13: Update Examples and Downstream - -**Files:** -- Modify: `examples/reduction_*.rs` (files referencing KSatisfiability or KColoring) -- Modify: `src/lib.rs` (prelude updates) - -### Step 1: Update example files - -- `examples/reduction_ksatisfiability_to_qubo.rs` — `KSatisfiability::<3>` → `KSatisfiability::<K3>` -- `examples/reduction_kcoloring_to_qubo.rs` — `KColoring::<3, SimpleGraph>` → `KColoring::<K3, SimpleGraph>` -- `examples/reduction_kcoloring_to_ilp.rs` — similar -- `examples/reduction_satisfiability_to_ksatisfiability.rs` — `KSatisfiability<3>` → `KSatisfiability<K3>` -- `examples/reduction_satisfiability_to_kcoloring.rs` — `KColoring<3, SimpleGraph>` → `KColoring<K3, SimpleGraph>` - -### Step 2: Update integration tests - -- `tests/suites/integration.rs` — KColoring<3, ...> → KColoring<K3, ...> -- `tests/suites/reductions.rs` — similar - -### Step 3: Run full suite - -Run: `make test` - -```bash -git commit -m "refactor: update examples and integration tests for KValue type parameters" -``` - ---- - -## Task 14: Final Verification - -### Step 1: Format and lint - -```bash -make fmt -make clippy -``` - -### Step 2: Run full test suite - -```bash -make test -``` - -### Step 3: Check coverage - -```bash -make coverage # Must remain >95% -``` - -### Step 4: Regenerate artifacts - -```bash -make rust-export -make examples -make doc -``` - -### Step 5: Verify no regressions - -```bash -make compare # Compare exported JSON -``` - -### Step 6: Final commit if needed - -```bash -git commit -m "chore: regenerate artifacts after variant system redesign" -``` - ---- - -## Key Files Summary - -| File | Change | -|------|--------| -| `src/variant.rs` | VariantParam, CastToParent, VariantTypeEntry, impl_variant_param! (4 arms: root, parent, k-root, k-parent), variant_params!, KValue, K1-K4/KN | -| `src/topology/kings_subgraph.rs` | NEW | -| `src/topology/triangular_subgraph.rs` | NEW | -| `src/topology/graph.rs` | impl_variant_param! for SimpleGraph | -| `src/topology/unit_disk_graph.rs` | impl_variant_param! for UnitDiskGraph | -| `src/topology/hypergraph.rs` | impl_variant_param! for HyperGraph | -| `src/topology/mod.rs` | Export new types, pub(crate) old ones | -| `src/types.rs` | impl_variant_param! for One, i32, f64 | -| `src/graph_types.rs` | Remove old hierarchy system | -| `src/models/satisfiability/ksat.rs` | const K → K: KValue | -| `src/models/graph/kcoloring.rs` | const K → K: KValue | -| `src/models/**/*.rs` | + VariantParam bounds, variant_params! (21 files) | -| `src/rules/graph.rs` | Unified variant_hierarchy | -| `src/rules/maximumindependentset_gridgraph.rs` | Restructure for KingsSubgraph | -| `src/rules/maximumindependentset_triangular.rs` | Restructure for TriangularSubgraph | -| `problemreductions-macros/src/lib.rs` | Remove const generic rewriting | diff --git a/docs/plans/2026-02-16-problem-size-trait-impl.md b/docs/plans/2026-02-16-problem-size-trait-impl.md deleted file mode 100644 index 46db19273..000000000 --- a/docs/plans/2026-02-16-problem-size-trait-impl.md +++ /dev/null @@ -1,577 +0,0 @@ -# `problem_size()` Trait Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add `fn problem_size(&self) -> ProblemSize` to the `Problem` trait, implement it for all 21 problem types, validate overhead polynomial variable names in `find_cheapest_path`, and test. - -**Architecture:** Add a required `problem_size(&self)` method to `Problem` trait. Each impl returns a `ProblemSize` with named components matching the overhead polynomial variable names used by `#[reduction(overhead = ...)]` annotations. Add `input_variable_names()` to `ReductionOverhead` to extract referenced variable names from polynomials. In `find_cheapest_path`, validate that overhead polynomial input variables are a subset of `source.problem_size()` component names — catching naming mismatches at path-finding time. Empty `input_size` skips validation for backward compatibility. - -**Tech Stack:** Rust, no new dependencies. - ---- - -### Task 1: Add `problem_size()` to `Problem` trait - -**Files:** -- Modify: `src/traits.rs:7-25` - -**Step 1: Add the method to the trait** - -In `src/traits.rs`, add after the `variant()` method (line 24): - -```rust - /// Named size components for overhead estimation in reduction path-finding. - fn problem_size(&self) -> crate::types::ProblemSize; -``` - -**Step 2: Verify it fails to compile (all 21 impls are now broken)** - -Run: `cargo check 2>&1 | head -5` -Expected: errors about missing `problem_size` method in impls. - -**Step 3: Commit** - -```bash -git add src/traits.rs -git commit -m "feat: add problem_size() to Problem trait (compile-breaking)" -``` - ---- - -### Task 2: Implement `problem_size()` for graph problems (9 types) - -All graph problems follow the same pattern: `num_vertices` from `self.graph().num_vertices()`, `num_edges` from `self.graph().num_edges()`. KColoring adds `num_colors`. - -**Files:** -- Modify: `src/models/graph/maximum_independent_set.rs` (impl at ~line 93) -- Modify: `src/models/graph/maximum_clique.rs` (impl at ~line 93) -- Modify: `src/models/graph/minimum_vertex_cover.rs` (impl at ~line 88) -- Modify: `src/models/graph/minimum_dominating_set.rs` (impl at ~line 113) -- Modify: `src/models/graph/max_cut.rs` (impl at ~line 141) -- Modify: `src/models/graph/maximum_matching.rs` (impl at ~line 161) -- Modify: `src/models/graph/maximal_is.rs` (impl at ~line 127) -- Modify: `src/models/graph/kcoloring.rs` (impl at ~line 120) -- Modify: `src/models/graph/traveling_salesman.rs` (impl at ~line 124) - -**Step 1: Add `use crate::types::ProblemSize;` to each file** (if not already imported). - -**Step 2: Add `problem_size()` to each `impl Problem for` block** - -For MIS, MaxClique, MinVC, MinDS, MaximalIS, MaxCut, MaximumMatching, TravelingSalesman: -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_vertices", self.graph().num_vertices()), - ("num_edges", self.graph().num_edges()), - ]) - } -``` - -For KColoring (adds `num_colors`): -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_vertices", self.graph().num_vertices()), - ("num_edges", self.graph().num_edges()), - ("num_colors", self.num_colors()), - ]) - } -``` - -**Step 3: Verify graph models compile** - -Run: `cargo check 2>&1 | grep "error\[" | head -5` -Expected: remaining errors only from non-graph models. - -**Step 4: Commit** - -```bash -git add src/models/graph/ -git commit -m "feat: implement problem_size() for all graph problems" -``` - ---- - -### Task 3: Implement `problem_size()` for SAT problems (2 types) - -**Files:** -- Modify: `src/models/satisfiability/sat.rs` (impl at ~line 171) -- Modify: `src/models/satisfiability/ksat.rs` (impl at ~line 161) - -**Step 1: Add import and implement for Satisfiability** - -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_vars", self.num_vars()), - ("num_clauses", self.num_clauses()), - ("num_literals", self.num_literals()), - ]) - } -``` - -**Step 2: Implement for KSatisfiability** - -KSatisfiability does not have a `num_literals()` method. Compute inline: -```rust - fn problem_size(&self) -> ProblemSize { - let num_literals: usize = self.clauses().iter().map(|c| c.len()).sum(); - ProblemSize::new(vec![ - ("num_vars", self.num_vars()), - ("num_clauses", self.num_clauses()), - ("num_literals", num_literals), - ]) - } -``` - -**Step 3: Commit** - -```bash -git add src/models/satisfiability/ -git commit -m "feat: implement problem_size() for SAT problems" -``` - ---- - -### Task 4: Implement `problem_size()` for set problems (2 types) - -**Files:** -- Modify: `src/models/set/maximum_set_packing.rs` (impl at ~line 122) -- Modify: `src/models/set/minimum_set_covering.rs` (impl at ~line 132) - -**Step 1: Implement for MaximumSetPacking** - -MaximumSetPacking has `num_sets()` but no `universe_size()`. Compute from sets: -```rust - fn problem_size(&self) -> ProblemSize { - let universe_size = self.sets().iter() - .flat_map(|s| s.iter()) - .max() - .map_or(0, |&m| m + 1); - ProblemSize::new(vec![ - ("num_sets", self.num_sets()), - ("universe_size", universe_size), - ]) - } -``` - -**Step 2: Implement for MinimumSetCovering** - -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_sets", self.num_sets()), - ("universe_size", self.universe_size()), - ]) - } -``` - -**Step 3: Commit** - -```bash -git add src/models/set/ -git commit -m "feat: implement problem_size() for set problems" -``` - ---- - -### Task 5: Implement `problem_size()` for optimization problems (3 types) - -**Files:** -- Modify: `src/models/optimization/qubo.rs` (impl at ~line 146) -- Modify: `src/models/optimization/spin_glass.rs` (impl at ~line 198) -- Modify: `src/models/optimization/ilp.rs` (impl at ~line 330) - -**Step 1: Implement for QUBO** - -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![("num_vars", self.num_vars())]) - } -``` - -**Step 2: Implement for SpinGlass** - -SpinGlass has `num_spins()` (= `graph.num_vertices()`). For `num_interactions`, use `graph.num_edges()`: -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_spins", self.num_spins()), - ("num_interactions", self.graph().num_edges()), - ]) - } -``` - -**Step 3: Implement for ILP** - -ILP has `num_variables()` and `self.constraints` (pub field): -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_vars", self.num_variables()), - ("num_constraints", self.constraints.len()), - ]) - } -``` - -**Step 4: Commit** - -```bash -git add src/models/optimization/ -git commit -m "feat: implement problem_size() for optimization problems" -``` - ---- - -### Task 6: Implement `problem_size()` for specialized problems (5 types) - -**Files:** -- Modify: `src/models/specialized/factoring.rs` (impl at ~line 116) -- Modify: `src/models/specialized/circuit.rs` (impl at ~line 269) -- Modify: `src/models/specialized/paintshop.rs` (impl at ~line 163) -- Modify: `src/models/specialized/biclique_cover.rs` (impl at ~line 212) -- Modify: `src/models/specialized/bmf.rs` (impl at ~line 193) - -**Step 1: Implement for Factoring** - -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_bits_first", self.m()), - ("num_bits_second", self.n()), - ]) - } -``` - -**Step 2: Implement for CircuitSAT** - -CircuitSAT stores `variables: Vec<String>` and has `circuit.num_assignments()`: -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_variables", self.num_variables()), - ("num_assignments", self.circuit().num_assignments()), - ]) - } -``` - -Note: verify that `self.circuit()` and `self.num_variables()` exist; if `circuit` is private without accessor, use `self.variables.len()` for `num_variables`. - -**Step 3: Implement for PaintShop** - -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("num_cars", self.num_cars()), - ("num_sequence", self.sequence_len()), - ]) - } -``` - -**Step 4: Add accessors to BicliqueCover** - -BicliqueCover has private `left_size` and `right_size` fields. Add public accessors: -```rust - /// Get the left partition size. - pub fn left_size(&self) -> usize { - self.left_size - } - - /// Get the right partition size. - pub fn right_size(&self) -> usize { - self.right_size - } -``` - -Then implement: -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("left_size", self.left_size()), - ("right_size", self.right_size()), - ("num_edges", self.num_edges()), - ("rank", self.k()), - ]) - } -``` - -**Step 5: Implement for BMF** - -BMF has `rows()`, `cols()`, `rank()`: -```rust - fn problem_size(&self) -> ProblemSize { - ProblemSize::new(vec![ - ("m", self.rows()), - ("n", self.cols()), - ("rank", self.rank()), - ]) - } -``` - -**Step 6: Verify full project compiles** - -Run: `cargo check --all-features` -Expected: clean compile, no errors. - -**Step 7: Commit** - -```bash -git add src/models/specialized/ -git commit -m "feat: implement problem_size() for specialized problems" -``` - ---- - -### Task 7: Add overhead variable name validation in `find_cheapest_path` - -**Files:** -- Modify: `src/polynomial.rs` (add `variable_names()` to `Polynomial`) -- Modify: `src/rules/registry.rs` (add `input_variable_names()` to `ReductionOverhead`) -- Modify: `src/rules/graph.rs` (add validation in `find_cheapest_path`) - -**Step 1: Add `variable_names()` to `Polynomial`** - -In `src/polynomial.rs`, add to `impl Polynomial`: -```rust - /// Collect all variable names referenced by this polynomial. - pub fn variable_names(&self) -> HashSet<&'static str> { - self.terms.iter() - .flat_map(|m| m.variables.iter().map(|(name, _)| *name)) - .collect() - } -``` - -Add `use std::collections::HashSet;` at the top of the file. - -**Step 2: Add `input_variable_names()` to `ReductionOverhead`** - -In `src/rules/registry.rs`, add to `impl ReductionOverhead`: -```rust - /// Collect all input variable names referenced by the overhead polynomials. - pub fn input_variable_names(&self) -> HashSet<&'static str> { - self.output_size.iter() - .flat_map(|(_, poly)| poly.variable_names()) - .collect() - } -``` - -Add `use std::collections::HashSet;` at the top. - -**Step 3: Add validation in `find_cheapest_path`** - -In `src/rules/graph.rs`, at the start of `find_cheapest_path`, after looking up the source node, validate that the `input_size` component names cover all overhead polynomial variables on outgoing edges. Skip validation when `input_size` is empty (backward compatibility): - -```rust - // Validate: when input_size is non-empty, check outgoing edges - if !input_size.components.is_empty() { - let size_names: std::collections::HashSet<&str> = input_size - .components.iter().map(|(k, _)| k.as_str()).collect(); - for edge_ref in self.graph.edges(src) { - let missing: Vec<_> = edge_ref.weight().overhead - .input_variable_names() - .into_iter() - .filter(|name| !size_names.contains(name)) - .collect(); - if !missing.is_empty() { - let target_node = &self.nodes[self.graph[edge_ref.target()]]; - panic!( - "Overhead for {} -> {} references variables {:?} \ - not in source problem_size() components {:?}", - source, target_node.name, missing, size_names, - ); - } - } - } -``` - -**Step 4: Verify compiles** - -Run: `cargo check --all-features` - -**Step 5: Commit** - -```bash -git add src/polynomial.rs src/rules/registry.rs src/rules/graph.rs -git commit -m "feat: validate overhead variable names against problem_size in find_cheapest_path" -``` - ---- - -### Task 8: Add unit tests for `problem_size()` - -**Files:** -- Create: `src/unit_tests/problem_size.rs` -- Modify: parent test module to include it via `#[path]` attribute - -**Step 1: Write tests** - -Create `src/unit_tests/problem_size.rs` with one test per problem category verifying component names and values match expectations. Example structure: - -```rust -//! Tests for Problem::problem_size() implementations. - -use crate::models::graph::*; -use crate::models::optimization::*; -use crate::models::satisfiability::*; -use crate::models::set::*; -use crate::models::specialized::*; -use crate::topology::SimpleGraph; -use crate::traits::Problem; - -#[test] -fn test_problem_size_mis() { - let g = SimpleGraph::new(4, vec![(0, 1), (1, 2), (2, 3)]); - let mis = MaximumIndependentSet::<SimpleGraph, i32>::unweighted(g); - let size = mis.problem_size(); - assert_eq!(size.get("num_vertices"), Some(4)); - assert_eq!(size.get("num_edges"), Some(3)); -} - -#[test] -fn test_problem_size_sat() { - use crate::models::satisfiability::CNFClause; - let sat = Satisfiability::new(3, vec![ - CNFClause::new(vec![1, -2]), - CNFClause::new(vec![2, 3]), - ]); - let size = sat.problem_size(); - assert_eq!(size.get("num_vars"), Some(3)); - assert_eq!(size.get("num_clauses"), Some(2)); - assert_eq!(size.get("num_literals"), Some(4)); -} - -#[test] -fn test_problem_size_qubo() { - let qubo = QUBO::<f64>::new(vec![1.0, 2.0, 3.0], vec![]); - let size = qubo.problem_size(); - assert_eq!(size.get("num_vars"), Some(3)); -} - -#[test] -fn test_problem_size_spinglass() { - let sg = SpinGlass::<SimpleGraph, f64>::new( - 3, - vec![((0, 1), 1.0), ((1, 2), -1.0)], - vec![0.0, 0.5, -0.5], - ); - let size = sg.problem_size(); - assert_eq!(size.get("num_spins"), Some(3)); - assert_eq!(size.get("num_interactions"), Some(2)); -} - -#[test] -fn test_problem_size_factoring() { - let f = Factoring::new(2, 3, 6); - let size = f.problem_size(); - assert_eq!(size.get("num_bits_first"), Some(2)); - assert_eq!(size.get("num_bits_second"), Some(3)); -} - -#[test] -fn test_problem_size_bmf() { - let bmf = BMF::new(vec![vec![true, false], vec![false, true]], 2); - let size = bmf.problem_size(); - assert_eq!(size.get("m"), Some(2)); - assert_eq!(size.get("n"), Some(2)); - assert_eq!(size.get("rank"), Some(2)); -} -``` - -Add similar tests for: MaxCut, KColoring, MaximumSetPacking, MinimumSetCovering, ILP, CircuitSAT, PaintShop, BicliqueCover. Each test creates a small instance and asserts the expected component names and values. - -**Step 2: Wire the test module** - -Find where unit test modules are declared (likely in `src/lib.rs` via `#[path]` attribute) and add: -```rust -#[cfg(test)] -#[path = "unit_tests/problem_size.rs"] -mod problem_size_tests; -``` - -**Step 3: Run tests** - -Run: `cargo test --all-features problem_size` -Expected: all tests pass. - -**Step 4: Commit** - -```bash -git add src/unit_tests/problem_size.rs src/lib.rs -git commit -m "test: add problem_size() unit tests for all 21 problem types" -``` - ---- - -### Task 9: Add integration test for `find_cheapest_path` with `problem_size` - -**Files:** -- Modify: `src/unit_tests/rules/reduction_path_parity.rs` - -**Step 1: Add test using `find_cheapest_path` with real problem size** - -```rust -#[test] -fn test_find_cheapest_path_with_problem_size() { - let graph = ReductionGraph::new(); - let petersen = SimpleGraph::new(10, vec![ - (0,1),(0,4),(0,5),(1,2),(1,6),(2,3),(2,7), - (3,4),(3,8),(4,9),(5,7),(5,8),(6,8),(6,9),(7,9), - ]); - let source = MaxCut::<SimpleGraph, i32>::unweighted(petersen); - let src_var = ReductionGraph::variant_to_map(&MaxCut::<SimpleGraph, i32>::variant()); - let dst_var = ReductionGraph::variant_to_map(&SpinGlass::<SimpleGraph, f64>::variant()); - - // Use source.problem_size() instead of ProblemSize::new(vec![]) - let rpath = graph - .find_cheapest_path( - "MaxCut", &src_var, - "SpinGlass", &dst_var, - &source.problem_size(), - &MinimizeSteps, - ) - .expect("Should find path MaxCut -> SpinGlass"); - - assert!(!rpath.type_names().is_empty()); - - // Verify problem_size has expected components - let size = source.problem_size(); - assert_eq!(size.get("num_vertices"), Some(10)); - assert_eq!(size.get("num_edges"), Some(15)); -} -``` - -**Step 2: Run test** - -Run: `cargo test --all-features find_cheapest_path_with_problem_size` -Expected: pass (validation passes because MaxCut's components match overhead polynomial vars). - -**Step 3: Commit** - -```bash -git add src/unit_tests/ -git commit -m "test: add find_cheapest_path integration test with problem_size" -``` - ---- - -### Task 10: Run full test suite and clippy - -**Step 1: Run tests** - -Run: `make test` -Expected: all tests pass. - -**Step 2: Run clippy** - -Run: `make clippy` -Expected: no warnings. - -**Step 3: Run fmt** - -Run: `make fmt` - -**Step 4: Final commit if any formatting changes** - -```bash -git add -A && git commit -m "chore: format" -``` diff --git a/docs/plans/2026-02-16-reduce-exported-functions-design.md b/docs/plans/2026-02-16-reduce-exported-functions-design.md deleted file mode 100644 index 022ca1ada..000000000 --- a/docs/plans/2026-02-16-reduce-exported-functions-design.md +++ /dev/null @@ -1,167 +0,0 @@ -# Design: Reduce Exported Functions (#77) - -## Goal - -Ensure every exported item has clear meaning, is well-documented, and relates to the package's vision of NP-hard problem reductions. Items that are implementation details become `pub(crate)`. - -## Approach: Moderate (Approach B) - -v0.1.x allows breaking changes — items are simply made `pub(crate)` with no deprecation. - ---- - -## 1. Internalize Reduction Structs & Implementation Details - -### 1a. Reduction structs (`src/rules/mod.rs`) - -Remove all `pub use` for individual `ReductionXToY` structs. Users interact via the `ReduceTo` trait or `ReductionGraph` — they never need the named struct types. - -**Items to internalize (~30+):** -- `ReductionCircuitToSG`, `ReductionKColoringToQUBO`, `ReductionFactoringToCircuit` -- `Reduction3SATToQUBO`, `ReductionKSatToQUBO` -- `ReductionISSimpleToGrid`, `ReductionISUnitDiskToGrid` -- `ReductionISToSP`, `ReductionSPToIS`, `ReductionISToQUBO` -- `ReductionISSimpleToTriangular` -- `ReductionMatchingToSP`, `ReductionSPToQUBO` -- `ReductionVCToIS`, `ReductionISToVC`, `ReductionVCToSC`, `ReductionVCToQUBO` -- `ReductionSATToCircuit`, `ReductionSATToColoring` -- `ReductionKSATToSAT`, `ReductionSATToKSAT` -- `ReductionSATToIS`, `ReductionSATToDS` -- `ReductionMaxCutToSG`, `ReductionSGToMaxCut` -- `ReductionQUBOToSG`, `ReductionSGToQUBO` -- All ILP reduction structs (feature-gated) - -### 1b. Circuit gadgets - -Internalize: `and_gadget`, `or_gadget`, `not_gadget`, `xor_gadget`, `set0_gadget`, `set1_gadget`, `LogicGadget` - -### 1c. Helper types - -Internalize: `BoolVar` (from `sat_maximumindependentset`), `ReductionAutoCast` struct - -### 1d. Unitdiskmapping internals (`src/rules/unitdiskmapping/mod.rs`) - -Internalize: `CopyLine`, `CellState`, `MappingGrid`, `Pattern`, `PatternCell`, `apply_gadget`, `pattern_matches`, `unapply_gadget`, `Weightable`, `map_weights`, `trace_centers`, `create_copylines`, `remove_order`, `mis_overhead_copyline`, `copyline_weighted_locations_triangular`, `mis_overhead_copyline_triangular` - -Keep public: `ksg`, `triangular` submodules, `MappingResult`, `GridKind` - -### 1e. Other internals - -- `src/polynomial.rs` -> `pub(crate) mod polynomial` in lib.rs -- `src/truth_table.rs` -> `pub(crate) mod truth_table` in lib.rs -- `NodeJson`, `EdgeJson`, `ReductionGraphJson` -> `pub(crate)` in `src/rules/graph.rs` -- `ReductionEntry`, `ReductionOverhead` -> `pub(crate)` in `src/rules/registry.rs` (already internal to proc macro + graph) - -**Keep public in `src/rules/mod.rs`:** -- Traits: `ReduceTo`, `ReductionResult` -- Graph API: `ReductionGraph`, `ReductionPath`, `ExecutablePath`, `ChainedReduction`, `ReductionStep` -- Cost: `CustomCost`, `Minimize`, `MinimizeSteps`, `PathCostFn` - ---- - -## 2. Consolidate Validation Functions into Problem Methods - -Add `is_valid_solution(&self, config: &[usize]) -> bool` inherent methods to each problem type. The existing free functions become `pub(crate)`. - -| Free function | Problem type | New method | -|---|---|---| -| `is_independent_set` | `MaximumIndependentSet` | `is_valid_solution` | -| `is_clique` | `MaximumClique` | `is_valid_solution` | -| `is_vertex_cover` | `MinimumVertexCover` | `is_valid_solution` | -| `is_dominating_set` | `MinimumDominatingSet` | `is_valid_solution` | -| `is_matching` | `MaximumMatching` | `is_valid_solution` | -| `is_valid_coloring` | `KColoring` | `is_valid_solution` | -| `is_maximal_independent_set` | `MaximalIS` | `is_valid_solution` | -| `is_hamiltonian_cycle` | `TravelingSalesman` | `is_valid_solution` | -| `is_satisfying_assignment` | `Satisfiability` | `is_valid_solution` | -| `is_biclique_cover` | `BicliqueCover` | `is_valid_solution` | -| `is_circuit_satisfying` | `CircuitSAT` | `is_valid_solution` | -| `is_factoring` | `Factoring` | `is_valid_solution` | -| `is_set_packing` | `MaximumSetPacking` | `is_valid_solution` | -| `is_set_cover` | `MinimumSetCovering` | `is_valid_solution` | - -Value-returning functions also become methods: - -| Free function | Problem type | New method | -|---|---|---| -| `cut_size` | `MaxCut` | `cut_size(&self, config)` | -| `count_paint_switches` | `PaintShop` | `count_switches(&self, config)` | - -General matrix utilities become `pub(crate)`: -- `boolean_matrix_product` — not owned by BMF type, internal utility -- `matrix_hamming_distance` — same - ---- - -## 3. Config & Module Consolidation - -### 3a. Config utilities (`src/config.rs`) - -Make `pub(crate)`: `config_to_bits`, `bits_to_config` - -Keep public: `ConfigIterator`, `DimsIterator`, `index_to_config`, `config_to_index` - -### 3b. Remove `graph_types` module - -`src/graph_types.rs` is redundant with `src/topology/`. Delete and update all internal `use crate::graph_types::X` to `use crate::topology::X`. - -### 3c. Circuit support types - -- Keep public: `Circuit`, `BooleanExpr`, `BooleanOp` (needed to construct `CircuitSAT`) -- Make `pub(crate)`: `Assignment` (internal evaluation detail) - ---- - -## 4. Updated Prelude - -```rust -pub mod prelude { - // Problem types - pub use crate::models::graph::{ - KColoring, MaxCut, MaximalIS, MaximumClique, MaximumIndependentSet, - MaximumMatching, MinimumDominatingSet, MinimumVertexCover, TravelingSalesman, - }; - pub use crate::models::optimization::{SpinGlass, QUBO}; - pub use crate::models::satisfiability::{CNFClause, KSatisfiability, Satisfiability}; - pub use crate::models::set::{MaximumSetPacking, MinimumSetCovering}; - pub use crate::models::specialized::{BicliqueCover, BMF, CircuitSAT, Factoring, PaintShop}; - - // Core traits - pub use crate::traits::{OptimizationProblem, Problem, SatisfactionProblem}; - pub use crate::rules::{ReduceTo, ReductionResult}; - pub use crate::solvers::{BruteForce, Solver}; - - // Types - pub use crate::types::{Direction, One, SolutionSize, Unweighted}; - pub use crate::error::{ProblemError, Result}; -} -``` - -**Removed from prelude** (still accessible via full path): -- `config::*` — power-user utilities -- `registry::{ComplexityClass, ProblemInfo, ProblemMetadata}` — metadata introspection -- `variant::{CastToParent, KValue, VariantParam, K1..KN}` — type-level machinery -- `types::{NumericSize, ProblemSize, WeightElement}` — trait bounds users rarely write - ---- - -## 5. Final Public Module Summary - -| Module | Visibility | Key public items | -|--------|-----------|-----------------| -| `models` | `pub` | All problem types, `Circuit`, `BooleanExpr`, `BooleanOp` | -| `rules` | `pub` | `ReduceTo`, `ReductionResult`, `ReductionGraph`, path types, cost types, `unitdiskmapping::{ksg, triangular, MappingResult, GridKind}` | -| `solvers` | `pub` | `Solver`, `BruteForce`, `ILPSolver` (feature-gated) | -| `traits` | `pub` | `Problem`, `OptimizationProblem`, `SatisfactionProblem` | -| `types` | `pub` | `Direction`, `SolutionSize`, `One`/`Unweighted`, `NumericSize`, `WeightElement`, `ProblemSize` | -| `error` | `pub` | `ProblemError`, `Result` | -| `config` | `pub` | `ConfigIterator`, `DimsIterator`, `index_to_config`, `config_to_index` | -| `topology` | `pub` | Graph types, `Graph` trait, `small_graphs` | -| `export` | `pub` | JSON export types (used by examples) | -| `io` | `pub` | `read_problem`, `write_problem`, `to_json`, `from_json` | -| `registry` | `pub` | `ProblemInfo`, `ComplexityClass`, `ProblemMetadata`, `collect_schemas` | -| `variant` | `pub` | `VariantParam`, `CastToParent`, `KValue`, K markers | -| `testing` | `pub` | Test macros & helpers | -| `polynomial` | `pub(crate)` | Internal | -| `truth_table` | `pub(crate)` | Internal | -| `graph_types` | **deleted** | Consolidated into `topology` | diff --git a/docs/plans/2026-02-16-reduce-exported-functions-impl.md b/docs/plans/2026-02-16-reduce-exported-functions-impl.md deleted file mode 100644 index 233e23b6a..000000000 --- a/docs/plans/2026-02-16-reduce-exported-functions-impl.md +++ /dev/null @@ -1,642 +0,0 @@ -# Reduce Exported Functions Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Clean up the public API surface by internalizing implementation details, consolidating validation functions into problem methods, and removing redundant modules. - -**Architecture:** Five phases — (1) internalize reduction structs and implementation details in `src/rules/mod.rs`, (2) internalize support modules (`polynomial`, `truth_table`, `graph_types`), (3) add `is_valid_solution` methods and internalize free functions, (4) config consolidation, (5) update prelude. Each phase ends with `make test clippy`. - -**Tech Stack:** Rust, `pub(crate)` visibility, inherent methods. - -**Design doc:** `docs/plans/2026-02-16-reduce-exported-functions-design.md` - -**Constraints discovered during planning:** -- `ReductionEntry`, `ReductionOverhead`, `ReductionAutoCast` MUST stay `pub` — referenced by `#[reduction]` proc macro and `impl_variant_reduction!` exported macro -- Integration tests (`tests/`) are a separate crate — `pub(crate)` items inaccessible there -- Validation functions take `&[bool]` while `Problem` trait uses `&[usize]` — new methods bridge this -- PR #76 added `problem_size()` free function and `problem_size_names()`/`problem_size_values()` to `Problem` trait — `problem_size` should stay in the prelude -- Some models already have private `_config` helpers (e.g., `is_independent_set_config`) — `is_valid_solution` methods should delegate to these where available - ---- - -### Task 1: Internalize reduction structs in rules/mod.rs - -**Files:** -- Modify: `src/rules/mod.rs` - -**Step 1: Remove pub use for all individual ReductionXToY structs and gadgets** - -In `src/rules/mod.rs`, replace all the `pub use` lines for reduction structs (lines 61-87 and 139-159) with `pub(crate) use` equivalents. Keep the trait re-exports and graph API exports. - -The file should go from: - -```rust -pub use circuit_spinglass::{ - and_gadget, not_gadget, or_gadget, set0_gadget, set1_gadget, xor_gadget, LogicGadget, - ReductionCircuitToSG, -}; -pub use coloring_qubo::ReductionKColoringToQUBO; -pub use factoring_circuit::ReductionFactoringToCircuit; -// ... all the other pub use lines for reduction structs ... -pub use sat_maximumindependentset::{BoolVar, ReductionSATToIS}; -// ... -``` - -To: - -```rust -pub(crate) use circuit_spinglass::{ - and_gadget, not_gadget, or_gadget, set0_gadget, set1_gadget, xor_gadget, LogicGadget, - ReductionCircuitToSG, -}; -pub(crate) use coloring_qubo::ReductionKColoringToQUBO; -pub(crate) use factoring_circuit::ReductionFactoringToCircuit; -// ... change ALL reduction struct pub use lines to pub(crate) use ... -``` - -Lines to change (all `pub use` → `pub(crate) use`): -- Lines 61-64: circuit_spinglass (gadgets + ReductionCircuitToSG) -- Line 65: coloring_qubo -- Line 66: factoring_circuit -- Lines 67-70: graph JSON types only (NodeJson, EdgeJson, ReductionGraphJson — keep ReductionGraph, ReductionPath, ExecutablePath, ChainedReduction, ReductionStep as pub) -- Line 71: ksatisfiability_qubo -- Line 72: maximumindependentset_gridgraph -- Line 73: maximumindependentset_maximumsetpacking -- Line 74: maximumindependentset_qubo -- Line 75: maximumindependentset_triangular -- Line 76: maximummatching_maximumsetpacking -- Line 77: maximumsetpacking_qubo -- Lines 78-79: minimumvertexcover_maximumindependentset, minimumvertexcover_minimumsetcovering -- Line 80: minimumvertexcover_qubo -- Line 81: sat_circuitsat -- Line 82: sat_coloring -- Line 83: sat_ksat -- Line 84: sat_maximumindependentset (BoolVar + ReductionSATToIS) -- Line 85: sat_minimumdominatingset -- Lines 86-87: spinglass_maxcut, spinglass_qubo -- Lines 139-159: all ILP reduction structs - -**Keep as `pub use`** (do NOT change): -- Line 5: `pub use cost::{CustomCost, Minimize, MinimizeSteps, PathCostFn};` -- Line 6: `pub use registry::{ReductionEntry, ReductionOverhead};` -- Line 88: `pub use traits::{ReduceTo, ReductionAutoCast, ReductionResult};` - -**Split the graph re-export** (line 67-70) into two: -```rust -pub use graph::{ - ChainedReduction, ExecutablePath, ReductionGraph, ReductionPath, ReductionStep, -}; -pub(crate) use graph::{EdgeJson, NodeJson, ReductionGraphJson}; -``` - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS — all reduction structs are used internally via the trait system, not by name from external code. - -**Step 3: Commit** - -```bash -git add src/rules/mod.rs -git commit -m "refactor: internalize reduction structs and gadgets in rules module" -``` - ---- - -### Task 2: Internalize unitdiskmapping internals - -**Files:** -- Modify: `src/rules/unitdiskmapping/mod.rs` - -**Step 1: Change pub use to pub(crate) use for internal types** - -Replace: -```rust -pub use copyline::{create_copylines, mis_overhead_copyline, remove_order, CopyLine}; -pub use grid::{CellState, MappingGrid}; -pub use pathdecomposition::{pathwidth, Layout, PathDecompositionMethod}; -pub use traits::{apply_gadget, pattern_matches, unapply_gadget, Pattern, PatternCell}; -pub use copyline::{copyline_weighted_locations_triangular, mis_overhead_copyline_triangular}; -pub use weighted::{map_weights, trace_centers, Weightable}; -``` - -With: -```rust -pub(crate) use copyline::{create_copylines, mis_overhead_copyline, remove_order, CopyLine}; -pub(crate) use grid::{CellState, MappingGrid}; -pub(crate) use pathdecomposition::{pathwidth, Layout, PathDecompositionMethod}; -pub(crate) use traits::{apply_gadget, pattern_matches, unapply_gadget, Pattern, PatternCell}; -pub(crate) use copyline::{copyline_weighted_locations_triangular, mis_overhead_copyline_triangular}; -pub(crate) use weighted::{map_weights, trace_centers, Weightable}; -``` - -Keep as `pub`: -```rust -pub mod ksg; -pub mod triangular; -pub use ksg::{GridKind, MappingResult}; -``` - -Also change `pub mod alpha_tensor` and `pub mod pathdecomposition` to `pub(crate) mod` if they only contain internal types. - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS - -**Step 3: Commit** - -```bash -git add src/rules/unitdiskmapping/mod.rs -git commit -m "refactor: internalize unitdiskmapping implementation details" -``` - ---- - -### Task 3: Internalize polynomial, truth_table modules and delete graph_types - -**Files:** -- Modify: `src/lib.rs` -- Delete: `src/graph_types.rs` - -**Step 1: Change module visibility in lib.rs** - -In `src/lib.rs`, change: -```rust -pub mod polynomial; -``` -to: -```rust -pub(crate) mod polynomial; -``` - -Change: -```rust -pub mod truth_table; -``` -to: -```rust -pub(crate) mod truth_table; -``` - -Change: -```rust -pub mod graph_types; -``` -to: delete this line entirely. Then delete `src/graph_types.rs`. - -Since `graph_types` has zero internal imports, no other files need updating. - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS — no code imports from these modules externally. - -**Step 3: Commit** - -```bash -git add src/lib.rs -git rm src/graph_types.rs -git commit -m "refactor: internalize polynomial/truth_table, delete unused graph_types module" -``` - ---- - -### Task 4: Add is_valid_solution methods to graph problems - -**Files:** -- Modify: `src/models/graph/maximum_independent_set.rs` -- Modify: `src/models/graph/maximum_clique.rs` -- Modify: `src/models/graph/minimum_vertex_cover.rs` -- Modify: `src/models/graph/minimum_dominating_set.rs` -- Modify: `src/models/graph/maximum_matching.rs` -- Modify: `src/models/graph/kcoloring.rs` -- Modify: `src/models/graph/maximal_is.rs` -- Modify: `src/models/graph/traveling_salesman.rs` -- Modify: `src/models/graph/max_cut.rs` - -**Step 1: Add is_valid_solution to each graph problem struct** - -For each problem type, add an inherent method that converts `&[usize]` config to `&[bool]` and delegates to the existing validation function. Add these methods inside the existing `impl` block for each type. - -**MaximumIndependentSet** (`src/models/graph/maximum_independent_set.rs`): -```rust -/// Check if a configuration is a valid independent set. -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - // Delegate to existing private helper that already takes &[usize] - is_independent_set_config(self.graph(), config) -} -``` -Note: This file already has a private `is_independent_set_config` helper taking `&[usize]`. Delegate to it directly instead of converting to `&[bool]`. Check other model files for similar `_config` helpers and use the same pattern. - -**MaximumClique** (`src/models/graph/maximum_clique.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_clique(self.graph(), &selected) -} -``` - -**MinimumVertexCover** (`src/models/graph/minimum_vertex_cover.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_vertex_cover(self.graph(), &selected) -} -``` - -**MinimumDominatingSet** (`src/models/graph/minimum_dominating_set.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_dominating_set(self.graph(), &selected) -} -``` - -**MaximumMatching** (`src/models/graph/maximum_matching.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_matching(self.graph(), &selected) -} -``` - -**KColoring** (`src/models/graph/kcoloring.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - is_valid_coloring(self.graph(), config, self.num_colors()) -} -``` -Note: `is_valid_coloring` already takes `&[usize]`, no conversion needed. - -**MaximalIS** (`src/models/graph/maximal_is.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_maximal_independent_set(self.graph(), &selected) -} -``` - -**TravelingSalesman** (`src/models/graph/traveling_salesman.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_hamiltonian_cycle(self.graph(), &selected) -} -``` - -**MaxCut** (`src/models/graph/max_cut.rs`): -```rust -/// Compute the cut size for a given partition configuration. -pub fn cut_size(&self, config: &[usize]) -> <W as WeightElement>::Sum { - let partition: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - cut_size(self.graph(), self.weights(), &partition) -} -``` - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS — new methods added, nothing removed yet. - -**Step 3: Commit** - -```bash -git add src/models/graph/ -git commit -m "feat: add is_valid_solution methods to graph problem types" -``` - ---- - -### Task 5: Add is_valid_solution methods to set and specialized problems - -**Files:** -- Modify: `src/models/set/maximum_set_packing.rs` -- Modify: `src/models/set/minimum_set_covering.rs` -- Modify: `src/models/specialized/biclique_cover.rs` -- Modify: `src/models/specialized/circuit.rs` -- Modify: `src/models/specialized/factoring.rs` -- Modify: `src/models/specialized/paintshop.rs` -- Modify: `src/models/specialized/bmf.rs` - -**Step 1: Add methods** - -**MaximumSetPacking** (`src/models/set/maximum_set_packing.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_set_packing(self.sets(), &selected) -} -``` - -**MinimumSetCovering** (`src/models/set/minimum_set_covering.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - let selected: Vec<bool> = config.iter().map(|&v| v != 0).collect(); - is_set_cover(self.universe_size(), self.sets(), &selected) -} -``` - -**BicliqueCover** (`src/models/specialized/biclique_cover.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - // Delegate to existing is_biclique_cover with problem's data - // Implementation depends on how config maps to biclique selection - // Check the existing evaluate() method for the mapping - is_biclique_cover(self.edges(), self.left_bicliques(config), self.right_bicliques(config)) -} -``` -Note: Inspect the actual struct fields and `evaluate()` to determine correct delegation. The method body may need adjustment. - -**CircuitSAT** (`src/models/specialized/circuit.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - // Convert config to assignments HashMap and delegate - // Check evaluate() for how config maps to variable assignments - self.evaluate(config) -} -``` -Note: For satisfaction problems where `Metric = bool`, `is_valid_solution` can just delegate to `evaluate`. - -**Factoring** (`src/models/specialized/factoring.rs`): -```rust -pub fn is_valid_solution(&self, config: &[usize]) -> bool { - self.evaluate(config) -} -``` - -**PaintShop** (`src/models/specialized/paintshop.rs`): -```rust -/// Count the number of paint switches for a given configuration. -pub fn count_switches(&self, config: &[usize]) -> usize { - count_paint_switches(config) -} -``` - -**BMF** (`src/models/specialized/bmf.rs`) — no `is_valid_solution` needed (BMF is optimization). The matrix utility functions just become `pub(crate)`. - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS - -**Step 3: Commit** - -```bash -git add src/models/set/ src/models/specialized/ -git commit -m "feat: add is_valid_solution/count_switches methods to set and specialized problems" -``` - ---- - -### Task 6: Internalize validation free functions - -**Files:** -- Modify: `src/models/graph/mod.rs` -- Modify: `src/models/graph/maximum_independent_set.rs` (change `pub fn` to `pub(crate) fn`) -- Modify: `src/models/graph/maximum_clique.rs` -- Modify: `src/models/graph/minimum_vertex_cover.rs` -- Modify: `src/models/graph/minimum_dominating_set.rs` -- Modify: `src/models/graph/maximum_matching.rs` -- Modify: `src/models/graph/kcoloring.rs` -- Modify: `src/models/graph/maximal_is.rs` -- Modify: `src/models/graph/traveling_salesman.rs` -- Modify: `src/models/graph/max_cut.rs` -- Modify: `src/models/set/maximum_set_packing.rs` -- Modify: `src/models/set/minimum_set_covering.rs` -- Modify: `src/models/set/mod.rs` -- Modify: `src/models/specialized/mod.rs` -- Modify: `src/models/specialized/biclique_cover.rs` -- Modify: `src/models/specialized/bmf.rs` -- Modify: `src/models/specialized/circuit.rs` -- Modify: `src/models/specialized/factoring.rs` -- Modify: `src/models/specialized/paintshop.rs` -- Modify: `src/models/mod.rs` - -**Step 1: Change function visibility** - -In each model file, change the validation free function from `pub fn` to `pub(crate) fn`: -- `is_independent_set` → `pub(crate) fn is_independent_set` -- `is_clique` → `pub(crate) fn is_clique` -- `is_vertex_cover` → `pub(crate) fn is_vertex_cover` -- `is_dominating_set` → `pub(crate) fn is_dominating_set` -- `is_matching` → `pub(crate) fn is_matching` -- `is_valid_coloring` → `pub(crate) fn is_valid_coloring` -- `is_maximal_independent_set` → `pub(crate) fn is_maximal_independent_set` -- `is_hamiltonian_cycle` → `pub(crate) fn is_hamiltonian_cycle` -- `cut_size` → `pub(crate) fn cut_size` -- `is_set_packing` → `pub(crate) fn is_set_packing` -- `is_set_cover` → `pub(crate) fn is_set_cover` -- `is_biclique_cover` → `pub(crate) fn is_biclique_cover` -- `is_circuit_satisfying` → `pub(crate) fn is_circuit_satisfying` -- `is_factoring` → `pub(crate) fn is_factoring` -- `count_paint_switches` → `pub(crate) fn count_paint_switches` -- `boolean_matrix_product` → `pub(crate) fn boolean_matrix_product` -- `matrix_hamming_distance` → `pub(crate) fn matrix_hamming_distance` - -**Step 2: Remove free functions from mod.rs re-exports** - -In `src/models/graph/mod.rs`, change: -```rust -pub use kcoloring::{is_valid_coloring, KColoring}; -pub use max_cut::{cut_size, MaxCut}; -pub use maximal_is::{is_maximal_independent_set, MaximalIS}; -pub use maximum_clique::{is_clique, MaximumClique}; -pub use maximum_independent_set::{is_independent_set, MaximumIndependentSet}; -pub use maximum_matching::{is_matching, MaximumMatching}; -pub use minimum_dominating_set::{is_dominating_set, MinimumDominatingSet}; -pub use minimum_vertex_cover::{is_vertex_cover, MinimumVertexCover}; -pub use traveling_salesman::{is_hamiltonian_cycle, TravelingSalesman}; -``` -to: -```rust -pub use kcoloring::KColoring; -pub use max_cut::MaxCut; -pub use maximal_is::MaximalIS; -pub use maximum_clique::MaximumClique; -pub use maximum_independent_set::MaximumIndependentSet; -pub use maximum_matching::MaximumMatching; -pub use minimum_dominating_set::MinimumDominatingSet; -pub use minimum_vertex_cover::MinimumVertexCover; -pub use traveling_salesman::TravelingSalesman; -``` - -In `src/models/set/mod.rs`, remove validation functions from re-exports (keep only the problem types). - -In `src/models/specialized/mod.rs`, change: -```rust -pub use biclique_cover::{is_biclique_cover, BicliqueCover}; -pub use bmf::{boolean_matrix_product, matrix_hamming_distance, BMF}; -pub use circuit::{is_circuit_satisfying, Assignment, BooleanExpr, BooleanOp, Circuit, CircuitSAT}; -pub use factoring::{is_factoring, Factoring}; -pub use paintshop::{count_paint_switches, PaintShop}; -``` -to: -```rust -pub use biclique_cover::BicliqueCover; -pub use bmf::BMF; -pub use circuit::{BooleanExpr, BooleanOp, Circuit, CircuitSAT}; -pub use factoring::Factoring; -pub use paintshop::PaintShop; -``` -Note: `Assignment` also becomes `pub(crate)`. - -In `src/models/mod.rs`, remove re-exported validation functions — only keep problem type re-exports. - -**Step 3: Update any integration test call sites** - -Check `tests/suites/integration.rs` — the audit found it uses `problem.evaluate(sol)` not free functions, so no changes needed there. - -Check `tests/suites/` for any other files using the free functions via `use problemreductions::models::graph::*` glob import. With `pub(crate)`, those functions simply won't be in scope — only the problem types will. This should be transparent unless tests call the functions by name. - -**Step 4: Run tests** - -Run: `make test clippy` -Expected: PASS — all internal unit tests use `pub(crate)` items fine. Integration tests don't call these functions directly. - -**Step 5: Commit** - -```bash -git add src/models/ -git commit -m "refactor: internalize validation free functions, keep as problem methods" -``` - ---- - -### Task 7: Internalize config utility functions - -**Files:** -- Modify: `src/config.rs` - -**Step 1: Change visibility** - -In `src/config.rs`, change: -```rust -pub fn config_to_bits(config: &[usize]) -> Vec<bool> { -``` -to: -```rust -pub(crate) fn config_to_bits(config: &[usize]) -> Vec<bool> { -``` - -And: -```rust -pub fn bits_to_config(bits: &[bool]) -> Vec<usize> { -``` -to: -```rust -pub(crate) fn bits_to_config(bits: &[bool]) -> Vec<usize> { -``` - -Keep `ConfigIterator`, `DimsIterator`, `index_to_config`, `config_to_index` as `pub`. - -**Step 2: Run tests** - -Run: `make test clippy` -Expected: PASS - -**Step 3: Commit** - -```bash -git add src/config.rs -git commit -m "refactor: internalize config_to_bits and bits_to_config" -``` - ---- - -### Task 8: Update prelude - -**Files:** -- Modify: `src/lib.rs` - -**Step 1: Slim down the prelude** - -Replace the prelude in `src/lib.rs` with: - -```rust -/// Prelude module for convenient imports. -pub mod prelude { - // Problem types - pub use crate::models::graph::{ - KColoring, MaxCut, MaximalIS, MaximumClique, MaximumIndependentSet, MaximumMatching, - MinimumDominatingSet, MinimumVertexCover, TravelingSalesman, - }; - pub use crate::models::optimization::{SpinGlass, QUBO}; - pub use crate::models::satisfiability::{CNFClause, KSatisfiability, Satisfiability}; - pub use crate::models::set::{MaximumSetPacking, MinimumSetCovering}; - pub use crate::models::specialized::{BicliqueCover, BMF, CircuitSAT, Factoring, PaintShop}; - - // Core traits - pub use crate::rules::{ReduceTo, ReductionResult}; - pub use crate::solvers::{BruteForce, Solver}; - pub use crate::traits::{problem_size, OptimizationProblem, Problem, SatisfactionProblem}; - - // Types - pub use crate::error::{ProblemError, Result}; - pub use crate::types::{Direction, One, ProblemSize, SolutionSize, Unweighted}; -} -``` - -Items removed from prelude (still accessible via full path): -- `config::{bits_to_config, config_to_bits, config_to_index, index_to_config, ConfigIterator}` -- `registry::{ComplexityClass, ProblemInfo, ProblemMetadata}` -- `variant::{CastToParent, KValue, VariantParam, K1, K2, K3, K4, K5, KN}` -- `types::{NumericSize, WeightElement}` -- `models::optimization::{Comparison, LinearConstraint, ObjectiveSense, VarBounds, ILP}` - -Items kept in prelude (from PR #76): -- `traits::problem_size` — core Problem API utility -- `types::ProblemSize` — return type of `problem_size()` - -**Step 2: Fix compilation errors in examples and tests** - -Many examples use `use problemreductions::prelude::*`. Items removed from the prelude may need explicit imports. Specifically: -- Examples that construct `ILP` instances need: `use problemreductions::models::optimization::{ILP, Comparison, LinearConstraint, ObjectiveSense, VarBounds};` -- Examples using `ReduceTo` — still in prelude, OK -- Examples using `ProblemSize` — add explicit import where needed -- Examples using `ComplexityClass` or `ProblemInfo` — add explicit import -- Examples using config functions — add `use problemreductions::config::*;` - -Search for compilation errors and fix each by adding the specific import. - -**Step 3: Run tests** - -Run: `make test clippy` -Expected: PASS after fixing imports - -**Step 4: Commit** - -```bash -git add src/lib.rs examples/ tests/ -git commit -m "refactor: slim down prelude to essential items" -``` - ---- - -### Task 9: Final verification - -**Step 1: Full test suite** - -Run: `make test clippy` -Expected: PASS - -**Step 2: Check doc build** - -Run: `cargo doc --no-deps 2>&1 | head -50` -Expected: No warnings about broken doc links - -**Step 3: Verify the public API looks clean** - -Run: `cargo doc --no-deps --open` and review the top-level documentation. Verify: -- No `ReductionXToY` structs visible in `rules` module docs -- No gadget functions visible -- No `NodeJson`/`EdgeJson` visible -- Problem types have `is_valid_solution` methods in their docs -- Prelude is clean and focused - -**Step 4: Commit any doc fixes** - -```bash -git add -A -git commit -m "docs: fix any doc link issues from API cleanup" -``` diff --git a/docs/plans/2026-02-18-cli-tool-design.md b/docs/plans/2026-02-18-cli-tool-design.md deleted file mode 100644 index b17f81aa8..000000000 --- a/docs/plans/2026-02-18-cli-tool-design.md +++ /dev/null @@ -1,152 +0,0 @@ -# CLI Tool Design: `pred` - -## Overview - -A command-line tool for researchers and students to explore NP-hard problem reductions and solve problem instances without writing Rust code. Implemented as a separate workspace crate (`problemreductions-cli`), binary name `pred`. - -## Audience - -Researchers and students studying NP-hard reductions who want to explore and visualize without writing any Rust code. - -## Command Structure - -Subcommand-based CLI with two top-level groups: `graph` (exploration) and solve/reduce/evaluate (computation). - -### Graph Exploration - -``` -pred graph list # List all registered problems -pred graph show <Problem> # Show problem details, variants, reductions -pred graph show MIS --variants # List all variants -pred graph path <Source> <Target> # Find cheapest reduction path -pred graph path MIS QUBO --cost minimize:num_vars # Custom cost function -pred graph export [--output path] # Export reduction_graph.json -``` - -### Computation - -``` -pred solve <input.json> --via <Target> # Reduce + solve + map solution back -pred solve --problem MIS --edges 0-1,1-2 --via QUBO # Inline input -pred reduce <input.json> --to <Target> # Reduce only, output target as JSON -pred evaluate <input.json> --config 1,0,1 # Evaluate a configuration -pred schema <Problem> # Show JSON schema for a problem type -``` - -### Global Flags - -- `--json` — structured JSON output, saved to file (default filename derived from command) -- `--output <path>` — custom output file path (used with `--json`) -- `--help` / `-h` — per-command help - -## Problem Name Resolution - -Case-insensitive matching with common aliases: - -| Input | Resolves to | -|-------|-------------| -| `MIS`, `mis` | `MaximumIndependentSet` | -| `MVC` | `MinimumVertexCover` | -| `SAT` | `Satisfiability` | -| `3SAT` | `KSatisfiability` (K=3) | -| `QUBO` | `QUBO` | -| `MaxCut` | `MaxCut` | - -Unambiguous prefix matching: `MaximumI` → `MaximumIndependentSet`, but `Maximum` is rejected (ambiguous). - -## Variant Syntax - -Slash-based positional notation after the problem name. Order follows `Problem::variant()` key order. Partial specification fills from the left; no skipping. - -``` -MIS → defaults (SimpleGraph, One) -MIS/UnitDiskGraph → UnitDiskGraph, default weight -MIS/SimpleGraph/f64 → must spell out graph to set weight -KColoring/K3 → SimpleGraph, K=3 -3SAT → alias for KSatisfiability/K3 -QUBO → no variants -``` - -## Input Formats - -### JSON Files - -Reuses the library's existing serde serialization: - -```json -{ - "problem": "MaximumIndependentSet", - "graph": {"edges": [[0,1], [1,2], [2,0]], "num_vertices": 3}, - "weights": [1, 1, 1] -} -``` - -### Inline Arguments - -For simple cases without a JSON file: - -``` -pred solve --problem MIS --edges 0-1,1-2,2-0 --weights 1,1,1 --via QUBO -``` - -## Output - -- **Human-readable (default):** plain text to stdout -- **`--json`:** structured JSON saved to file (default name derived from command, e.g., `pred_path_MIS_QUBO.json`) -- **`--json --output custom.json`:** custom output path -- **Errors:** always to stderr -- **Exit codes:** non-zero on any error - -## Architecture - -### Crate Layout - -Separate workspace crate: `problemreductions-cli/` - -``` -src/ - main.rs # Cli::parse(), dispatch to commands - cli.rs # Clap derive structs (Cli, Commands, GraphCommands) - commands/ - graph.rs # list, show, path, export - solve.rs # reduce + solve + extract solution - reduce.rs # reduce only, output target problem - evaluate.rs # evaluate a config - schema.rs # show JSON schema for a problem type - output.rs # OutputMode enum, write_json_file(), print_human() - problem_name.rs # Alias resolution + variant parsing (slash notation) -``` - -### Dependencies - -- `clap` (derive) — argument parsing -- `anyhow` — error handling -- `serde_json` — JSON I/O -- `problemreductions` — the library (all features) - -### Dynamic Dispatch - -- **Graph commands:** use `ReductionGraph` directly — already works with string names -- **Solve/reduce/evaluate:** dispatch table — a `match` over known problem names that constructs concrete types from JSON. ~20 match arms, one per problem type. - -### Error Handling - -`anyhow::Result` throughout, with `.context()` for actionable error messages. Non-zero exit code on any error. - -## V1 Scope - -### In scope - -- `pred graph list` -- `pred graph show <Problem>` (with `--variants`) -- `pred graph path <Source> <Target>` (with `--cost`) -- `pred graph export` -- `pred solve` (JSON file and inline input, brute-force solver) -- `pred reduce` (reduce only) -- `pred evaluate` -- `pred schema` -- `--json` output to file - -### Out of scope (v2+) - -See GitHub issue for future plans. diff --git a/docs/plans/2026-02-22-mcp-prompts-redesign-design.md b/docs/plans/2026-02-22-mcp-prompts-redesign-design.md deleted file mode 100644 index 8ef9a273b..000000000 --- a/docs/plans/2026-02-22-mcp-prompts-redesign-design.md +++ /dev/null @@ -1,134 +0,0 @@ -# MCP Prompts Redesign: Task-Oriented Prompts - -## Problem - -The current 3 MCP prompts (`analyze_problem`, `reduction_walkthrough`, `explore_graph`) are tool-centric — they list which tools to call rather than expressing what the user wants to accomplish. This makes them disconnected from how researchers, students, and LLM agents actually think about reductions. - -## Design - -Replace the 3 existing prompts with 7 task-oriented prompts. All prompt text frames requests as user questions. No tool names appear in prompt text — the LLM decides which tools to call. - -### Prompt Inventory - -| # | Name | Arguments | User question it answers | -|---|------|-----------|--------------------------| -| 1 | `what_is` | `problem` (req) | "What is MaxCut?" | -| 2 | `model_my_problem` | `description` (req) | "I have a scheduling problem — what maps to it?" | -| 3 | `compare` | `problem_a` (req), `problem_b` (req) | "How do MIS and Vertex Cover relate?" | -| 4 | `reduce` | `source` (req), `target` (req) | "Walk me through reducing MIS to QUBO" | -| 5 | `solve` | `problem_type` (req), `instance` (req) | "Solve this graph for maximum independent set" | -| 6 | `find_reduction` | `source` (req), `target` (req) | "What's the cheapest path from SAT to QUBO?" | -| 7 | `overview` | *(none)* | "Show me the full problem landscape" | - -### Prompt Texts - -#### 1. `what_is` - -**Description:** Explain a problem type: what it models, its variants, and how it connects to other problems - -``` -Explain the "{problem}" problem to me. - -What does it model in the real world? What are its variants (graph types, -weight types)? What other problems can it reduce to, and which problems -reduce to it? - -Give me a concise summary suitable for someone encountering this problem -for the first time, then show the technical details. -``` - -#### 2. `model_my_problem` - -**Description:** Map a real-world problem to the closest NP-hard problem type in the reduction graph - -``` -I have a real-world problem and I need help identifying which NP-hard -problem type it maps to. - -Here's my problem: "{description}" - -Look through the available problem types in the reduction graph and -identify which one(s) best model my problem. Explain why it's a good fit, -what the variables and constraints map to, and suggest how I could encode -my specific instance. -``` - -#### 3. `compare` - -**Description:** Compare two problem types: their relationship, differences, and reduction path between them - -``` -Compare "{problem_a}" and "{problem_b}". - -How are they related? Is there a direct reduction between them, or do they -connect through intermediate problems? What are the key differences in -what they model? If one can be reduced to the other, what is the overhead? -``` - -#### 4. `reduce` - -**Description:** Step-by-step reduction walkthrough: create an instance, reduce it, solve it, and map the solution back - -``` -Walk me through reducing a "{source}" instance to "{target}", step by step. - -1. Find the reduction path and explain the overhead. -2. Create a small, concrete example instance of "{source}". -3. Reduce it to "{target}" and show what the transformed instance looks like. -4. Solve the reduced instance. -5. Explain how the solution maps back to the original problem. - -Use a small example so I can follow each transformation by hand. -``` - -#### 5. `solve` - -**Description:** Create and solve a problem instance, showing the optimal solution - -``` -Create a {problem_type} instance with these parameters: {instance} - -Solve it and show me: -- The problem instance details (size, structure) -- The optimal solution and its objective value -- Why this solution is optimal (briefly) -``` - -#### 6. `find_reduction` - -**Description:** Find the best reduction path between two problems, with cost analysis - -``` -Find the best way to reduce "{source}" to "{target}". - -Show me the cheapest reduction path and explain the cost at each step. -Are there alternative paths? If so, compare them — which is better for -small instances vs. large instances? -``` - -#### 7. `overview` - -**Description:** Explore the full landscape of NP-hard problems and reductions in the graph - -``` -Give me an overview of the NP-hard problem reduction landscape. - -How many problem types are registered? What are the major categories -(graph, SAT, optimization)? Which problems are the most connected hubs? -Which problems can reach the most targets through reductions? - -Summarize the structure so I understand what's available and where to -start exploring. -``` - -## Scope - -- **Changed:** `problemreductions-cli/src/mcp/prompts.rs` (prompt definitions) -- **Changed:** `problemreductions-cli/src/mcp/tests.rs` (prompt tests) -- **Unchanged:** All 10 MCP tools, tool handlers, server infrastructure - -## Testing - -- Unit tests: verify `list_prompts` returns 7 prompts with correct names/arguments -- Unit tests: verify `get_prompt` returns correct message text for each prompt -- Integration test: call each prompt via JSON-RPC and verify response structure diff --git a/docs/plans/2026-02-22-mcp-prompts-redesign.md b/docs/plans/2026-02-22-mcp-prompts-redesign.md deleted file mode 100644 index 35f1571b0..000000000 --- a/docs/plans/2026-02-22-mcp-prompts-redesign.md +++ /dev/null @@ -1,411 +0,0 @@ -# MCP Prompts Redesign Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Replace 3 tool-centric MCP prompts with 7 task-oriented prompts that map to real user journeys. - -**Architecture:** Rewrite `prompts.rs` with new `list_prompts()` and `get_prompt()` functions. Update integration test to match new prompt set. No tool or server changes. - -**Tech Stack:** Rust, rmcp crate (MCP SDK), serde_json - ---- - -### Task 1: Rewrite `list_prompts()` with 7 new prompt definitions - -**Files:** -- Modify: `problemreductions-cli/src/mcp/prompts.rs:4-43` - -**Step 1: Replace `list_prompts()` body** - -Replace the entire `list_prompts()` function body with 7 new `Prompt::new(...)` entries: - -```rust -pub fn list_prompts() -> Vec<Prompt> { - vec![ - Prompt::new( - "what_is", - Some("Explain a problem type: what it models, its variants, and how it connects to other problems"), - Some(vec![PromptArgument { - name: "problem".into(), - title: None, - description: Some("Problem name or alias (e.g., MIS, QUBO, MaxCut)".into()), - required: Some(true), - }]), - ), - Prompt::new( - "model_my_problem", - Some("Map a real-world problem to the closest NP-hard problem type in the reduction graph"), - Some(vec![PromptArgument { - name: "description".into(), - title: None, - description: Some("Free-text description of your real-world problem".into()), - required: Some(true), - }]), - ), - Prompt::new( - "compare", - Some("Compare two problem types: their relationship, differences, and reduction path between them"), - Some(vec![ - PromptArgument { - name: "problem_a".into(), - title: None, - description: Some("First problem name or alias".into()), - required: Some(true), - }, - PromptArgument { - name: "problem_b".into(), - title: None, - description: Some("Second problem name or alias".into()), - required: Some(true), - }, - ]), - ), - Prompt::new( - "reduce", - Some("Step-by-step reduction walkthrough: create an instance, reduce it, solve it, and map the solution back"), - Some(vec![ - PromptArgument { - name: "source".into(), - title: None, - description: Some("Source problem name or alias".into()), - required: Some(true), - }, - PromptArgument { - name: "target".into(), - title: None, - description: Some("Target problem name or alias".into()), - required: Some(true), - }, - ]), - ), - Prompt::new( - "solve", - Some("Create and solve a problem instance, showing the optimal solution"), - Some(vec![ - PromptArgument { - name: "problem_type".into(), - title: None, - description: Some("Problem type (e.g., MIS, SAT, QUBO, MaxCut)".into()), - required: Some(true), - }, - PromptArgument { - name: "instance".into(), - title: None, - description: Some( - "Instance parameters (e.g., \"edges: 0-1,1-2\" or \"clauses: 1,2;-1,3\")".into(), - ), - required: Some(true), - }, - ]), - ), - Prompt::new( - "find_reduction", - Some("Find the best reduction path between two problems, with cost analysis"), - Some(vec![ - PromptArgument { - name: "source".into(), - title: None, - description: Some("Source problem name or alias".into()), - required: Some(true), - }, - PromptArgument { - name: "target".into(), - title: None, - description: Some("Target problem name or alias".into()), - required: Some(true), - }, - ]), - ), - Prompt::new( - "overview", - Some("Explore the full landscape of NP-hard problems and reductions in the graph"), - None, - ), - ] -} -``` - -**Step 2: Run `cargo check` to verify compilation** - -Run: `cargo check -p problemreductions-cli` -Expected: compiles without errors - -**Step 3: Commit** - -```bash -git add problemreductions-cli/src/mcp/prompts.rs -git commit -m "refactor(mcp): replace prompt definitions with 7 task-oriented prompts" -``` - ---- - -### Task 2: Rewrite `get_prompt()` with new prompt texts - -**Files:** -- Modify: `problemreductions-cli/src/mcp/prompts.rs:46-128` - -**Step 1: Replace `get_prompt()` body** - -Replace the entire `get_prompt()` match block with 7 new arms: - -```rust -pub fn get_prompt( - name: &str, - arguments: &serde_json::Map<String, serde_json::Value>, -) -> Option<GetPromptResult> { - match name { - "what_is" => { - let problem = arguments - .get("problem") - .and_then(|v| v.as_str()) - .unwrap_or("MIS"); - - Some(GetPromptResult { - description: Some(format!("Explain the {} problem", problem)), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "Explain the \"{problem}\" problem to me.\n\n\ - What does it model in the real world? What are its variants \ - (graph types, weight types)? What other problems can it reduce \ - to, and which problems reduce to it?\n\n\ - Give me a concise summary suitable for someone encountering this \ - problem for the first time, then show the technical details." - ), - )], - }) - } - - "model_my_problem" => { - let description = arguments - .get("description") - .and_then(|v| v.as_str()) - .unwrap_or("(no description provided)"); - - Some(GetPromptResult { - description: Some("Map a real-world problem to an NP-hard problem type".into()), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "I have a real-world problem and I need help identifying which \ - NP-hard problem type it maps to.\n\n\ - Here's my problem: \"{description}\"\n\n\ - Look through the available problem types in the reduction graph \ - and identify which one(s) best model my problem. Explain why it's \ - a good fit, what the variables and constraints map to, and suggest \ - how I could encode my specific instance." - ), - )], - }) - } - - "compare" => { - let problem_a = arguments - .get("problem_a") - .and_then(|v| v.as_str()) - .unwrap_or("MIS"); - let problem_b = arguments - .get("problem_b") - .and_then(|v| v.as_str()) - .unwrap_or("VertexCover"); - - Some(GetPromptResult { - description: Some(format!("Compare {} and {}", problem_a, problem_b)), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "Compare \"{problem_a}\" and \"{problem_b}\".\n\n\ - How are they related? Is there a direct reduction between them, \ - or do they connect through intermediate problems? What are the \ - key differences in what they model? If one can be reduced to the \ - other, what is the overhead?" - ), - )], - }) - } - - "reduce" => { - let source = arguments - .get("source") - .and_then(|v| v.as_str()) - .unwrap_or("MIS"); - let target = arguments - .get("target") - .and_then(|v| v.as_str()) - .unwrap_or("QUBO"); - - Some(GetPromptResult { - description: Some(format!( - "Reduction walkthrough from {} to {}", - source, target - )), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "Walk me through reducing a \"{source}\" instance to \"{target}\", \ - step by step.\n\n\ - 1. Find the reduction path and explain the overhead.\n\ - 2. Create a small, concrete example instance of \"{source}\".\n\ - 3. Reduce it to \"{target}\" and show what the transformed instance \ - looks like.\n\ - 4. Solve the reduced instance.\n\ - 5. Explain how the solution maps back to the original problem.\n\n\ - Use a small example so I can follow each transformation by hand." - ), - )], - }) - } - - "solve" => { - let problem_type = arguments - .get("problem_type") - .and_then(|v| v.as_str()) - .unwrap_or("MIS"); - let instance = arguments - .get("instance") - .and_then(|v| v.as_str()) - .unwrap_or("edges: 0-1,1-2,2-0"); - - Some(GetPromptResult { - description: Some(format!("Solve a {} instance", problem_type)), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "Create a {problem_type} instance with these parameters: {instance}\n\n\ - Solve it and show me:\n\ - - The problem instance details (size, structure)\n\ - - The optimal solution and its objective value\n\ - - Why this solution is optimal (briefly)" - ), - )], - }) - } - - "find_reduction" => { - let source = arguments - .get("source") - .and_then(|v| v.as_str()) - .unwrap_or("SAT"); - let target = arguments - .get("target") - .and_then(|v| v.as_str()) - .unwrap_or("QUBO"); - - Some(GetPromptResult { - description: Some(format!( - "Find reduction path from {} to {}", - source, target - )), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - format!( - "Find the best way to reduce \"{source}\" to \"{target}\".\n\n\ - Show me the cheapest reduction path and explain the cost at each \ - step. Are there alternative paths? If so, compare them — which is \ - better for small instances vs. large instances?" - ), - )], - }) - } - - "overview" => Some(GetPromptResult { - description: Some("Overview of the NP-hard problem reduction landscape".into()), - messages: vec![PromptMessage::new_text( - PromptMessageRole::User, - "Give me an overview of the NP-hard problem reduction landscape.\n\n\ - How many problem types are registered? What are the major categories \ - (graph, SAT, optimization)? Which problems are the most connected hubs? \ - Which problems can reach the most targets through reductions?\n\n\ - Summarize the structure so I understand what's available and where to \ - start exploring." - .to_string(), - )], - }), - - _ => None, - } -} -``` - -**Step 2: Run `cargo check` to verify compilation** - -Run: `cargo check -p problemreductions-cli` -Expected: compiles without errors - -**Step 3: Commit** - -```bash -git add problemreductions-cli/src/mcp/prompts.rs -git commit -m "refactor(mcp): rewrite prompt texts to be task-oriented, not tool-centric" -``` - ---- - -### Task 3: Update integration test - -**Files:** -- Modify: `problemreductions-cli/tests/mcp_integration.rs` (the `test_mcp_server_prompts_list` test) - -**Step 1: Update the test assertions** - -Change the prompt count from 3 to 7 and update the name checks: - -```rust -assert_eq!( - prompts.len(), - 7, - "Expected 7 prompts, got {}: {:?}", - prompts.len(), - prompts - .iter() - .map(|p| p["name"].as_str().unwrap_or("?")) - .collect::<Vec<_>>() -); - -let prompt_names: Vec<&str> = prompts.iter().filter_map(|p| p["name"].as_str()).collect(); -assert!(prompt_names.contains(&"what_is")); -assert!(prompt_names.contains(&"model_my_problem")); -assert!(prompt_names.contains(&"compare")); -assert!(prompt_names.contains(&"reduce")); -assert!(prompt_names.contains(&"solve")); -assert!(prompt_names.contains(&"find_reduction")); -assert!(prompt_names.contains(&"overview")); -``` - -**Step 2: Run the integration test** - -Run: `cargo test -p problemreductions-cli --test mcp_integration test_mcp_server_prompts_list` -Expected: PASS - -**Step 3: Run full MCP test suite** - -Run: `make mcp-test` -Expected: all tests pass - -**Step 4: Commit** - -```bash -git add problemreductions-cli/tests/mcp_integration.rs -git commit -m "test(mcp): update prompt integration test for 7 task-oriented prompts" -``` - ---- - -### Task 4: Rebuild CLI and manual verification - -**Step 1: Rebuild the CLI** - -Run: `make cli` -Expected: successful install - -**Step 2: Verify prompts list via JSON-RPC** - -Run the MCP server and list prompts to confirm all 7 appear with correct names and argument counts. - -**Step 3: Verify one prompt get** - -Call `prompts/get` for `what_is` with `{"problem": "MaxCut"}` and confirm the response contains the new task-oriented text (no tool names like `show_problem`). - -**Step 4: Verify another prompt get** - -Call `prompts/get` for `solve` with `{"problem_type": "MIS", "instance": "edges: 0-1,1-2"}` and confirm the response text. diff --git a/docs/plans/2026-02-22-pred-mcp-design.md b/docs/plans/2026-02-22-pred-mcp-design.md deleted file mode 100644 index 8cdc8f3e6..000000000 --- a/docs/plans/2026-02-22-pred-mcp-design.md +++ /dev/null @@ -1,130 +0,0 @@ -# Design: `pred mcp` — MCP Server for problemreductions-cli - -## Summary - -Add a `pred mcp` subcommand that runs a stdio-based MCP (Model Context Protocol) server, exposing all CLI functionality as MCP tools. This lets AI assistants (Claude Desktop, Claude Code, Cursor, etc.) interact with the problem reductions library natively. - -## Approach - -Native `rmcp` integration (Approach A). The MCP server calls directly into the existing library and dispatch code — no subprocess shelling. Dependencies gated behind an `mcp` Cargo feature (included in default features). - -## User Experience - -```bash -# Install (mcp included by default) -cargo install problemreductions-cli - -# Configure in Claude Code (.mcp.json) or Claude Desktop -{ - "mcpServers": { - "problemreductions": { - "command": "pred", - "args": ["mcp"] - } - } -} - -# Verify -npx @modelcontextprotocol/inspector pred mcp -``` - -## Tools (10) - -All CLI subcommands (minus `completions`) exposed as MCP tools. Problem instances are passed as inline JSON strings (stateless — no server-side state). - -| Tool | Parameters | Returns | -|------|-----------|---------| -| `list_problems` | *(none)* | All registered problem types with aliases and reduction counts | -| `show_problem` | `problem: String` | Problem details: variants, size fields, reductions, schema | -| `neighbors_to` | `problem: String, hops?: u32` | Outgoing reduction neighbors | -| `neighbors_from` | `problem: String, hops?: u32` | Incoming reduction neighbors | -| `find_path` | `source: String, target: String, cost?: String, all?: bool` | Cheapest reduction path(s) | -| `export_graph` | *(none)* | Full reduction graph JSON | -| `create_problem` | `problem_type: String, params: JsonObject` | Problem instance JSON | -| `inspect_problem` | `problem_json: String` | Problem metadata (type, size, available solvers, reductions) | -| `evaluate` | `problem_json: String, config: Vec<usize>` | Evaluation result | -| `reduce` | `problem_json: String, target: String` | Reduction bundle JSON (source + target + path) | -| `solve` | `problem_json: String, solver?: String, timeout?: u64` | Solution + evaluation | - -Notes: -- `params` for `create_problem` is a free-form JSON object. Tool description includes examples for common problem types. -- Alias resolution (MIS, 3SAT, etc.) works the same as CLI. - -## Prompts (3) - -| Prompt | Parameters | Purpose | -|--------|-----------|---------| -| `analyze_problem` | `problem_type: String` | Show problem details, variants, reductions, and suggest strategies | -| `reduction_walkthrough` | `source: String, target: String` | End-to-end: find path, explain steps, create instance, reduce, solve, verify | -| `explore_graph` | *(none)* | List all problems, show graph structure, highlight hub problems | - -## Architecture - -``` -problemreductions-cli/ -├── Cargo.toml # add rmcp, tokio, schemars behind `mcp` feature -├── src/ -│ ├── main.rs # add Commands::Mcp variant -│ ├── cli.rs # add Mcp subcommand to enum -│ ├── mcp/ -│ │ ├── mod.rs # McpServer struct + ServerHandler impl -│ │ ├── tools.rs # #[tool_router] impl with all 10 tools -│ │ └── prompts.rs # 3 prompt templates -│ ├── commands/ # existing, unchanged -│ └── dispatch.rs # existing, reused by MCP tools -``` - -### Key Decisions - -1. **Feature-gated** — `mcp` feature adds rmcp + tokio + schemars. Included in default features so `cargo install` just works. - -2. **Reuse dispatch layer** — MCP tool handlers call the same functions the CLI commands use. No logic duplication. - -3. **Sync in async** — Existing CLI logic is synchronous. Tool handlers use `tokio::task::spawn_blocking` where needed (mainly `solve` with timeout). Graph query tools are fast enough to call directly. - -4. **Logging** — `tracing` output goes to stderr (required by MCP stdio protocol). Consistent with existing CLI stderr printing. - -5. **Error handling** — Tool errors return `CallToolResult` with `is_error: true` and human-readable message. No panics. - -### Data Flow - -``` -Claude Desktop - ↓ JSON-RPC: tools/call "solve" {problem_json, solver: "brute-force"} -McpServer::solve() - ↓ parse problem_json → serde_json::Value -dispatch::load_problem(name, variant, data) - ↓ → LoadedProblem -LoadedProblem::solve_brute_force() - ↓ → Option<Vec<usize>> - ↓ format result as JSON string -CallToolResult::success(vec![Content::text(result)]) - ↑ JSON-RPC response -Claude Desktop -``` - -## Dependencies - -```toml -[dependencies] -rmcp = { version = "0.16", features = ["server", "macros", "transport-io"], optional = true } -tokio = { version = "1", features = ["full"], optional = true } -schemars = { version = "1.0", optional = true } - -[features] -mcp = ["dep:rmcp", "dep:tokio", "dep:schemars"] -default = ["highs", "mcp"] -``` - -## Testing - -- Unit tests for each tool handler — call handler functions directly, assert JSON output structure -- Integration test: spawn `pred mcp` as subprocess, send JSON-RPC over stdin, validate responses -- Manual testing: `npx @modelcontextprotocol/inspector pred mcp` -- Add `make mcp-test` target - -## Documentation - -- Add `docs/book/mcp.md` to mdBook -- Update README with MCP quickstart section -- Include example conversation showing tool usage diff --git a/docs/plans/2026-02-22-pred-mcp-plan.md b/docs/plans/2026-02-22-pred-mcp-plan.md deleted file mode 100644 index 2e30236f2..000000000 --- a/docs/plans/2026-02-22-pred-mcp-plan.md +++ /dev/null @@ -1,775 +0,0 @@ -# `pred mcp` Implementation Plan - -> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. - -**Goal:** Add a `pred mcp` subcommand that runs a stdio-based MCP server exposing all CLI functionality as tools. - -**Architecture:** Feature-gated `mcp` module using `rmcp` crate. The MCP server struct wraps the existing dispatch/commands layer. Tools are defined via `#[tool_router]` macros, prompts via `ServerHandler::list_prompts`. - -**Tech Stack:** rmcp 0.16 (server + macros + transport-io), tokio 1, schemars 1.0 - -**Design doc:** `docs/plans/2026-02-22-pred-mcp-design.md` - ---- - -### Task 1: Add dependencies and feature gate - -**Files:** -- Modify: `problemreductions-cli/Cargo.toml` - -**Step 1: Add MCP dependencies to Cargo.toml** - -Add these to `[dependencies]`: -```toml -rmcp = { version = "0.16", features = ["server", "macros", "transport-io"], optional = true } -tokio = { version = "1", features = ["full"], optional = true } -schemars = { version = "1.0", optional = true } -tracing = { version = "0.1", optional = true } -tracing-subscriber = { version = "0.3", optional = true } -``` - -Add to `[features]`: -```toml -mcp = ["dep:rmcp", "dep:tokio", "dep:schemars", "dep:tracing", "dep:tracing-subscriber"] -``` - -Update `default`: -```toml -default = ["highs", "mcp"] -``` - -**Step 2: Verify it compiles** - -Run: `cd problemreductions-cli && cargo check --features mcp` -Expected: compiles without errors - -**Step 3: Verify it also compiles without the feature** - -Run: `cargo check --no-default-features` -Expected: compiles without errors (existing CLI still works without MCP) - -**Step 4: Commit** - -```bash -git add problemreductions-cli/Cargo.toml -git commit -m "feat(cli): add rmcp dependencies behind mcp feature gate" -``` - ---- - -### Task 2: Add `Mcp` subcommand to CLI - -**Files:** -- Modify: `problemreductions-cli/src/cli.rs` -- Modify: `problemreductions-cli/src/main.rs` - -**Step 1: Add `Mcp` variant to Commands enum** - -In `cli.rs`, add to the `Commands` enum (before `Completions`): - -```rust - /// Start MCP (Model Context Protocol) server for AI assistant integration - #[cfg(feature = "mcp")] - #[command(after_help = "\ -Start a stdio-based MCP server that exposes problem reduction tools -to AI assistants like Claude Desktop and Claude Code. - -Configuration (Claude Code .mcp.json): - { - \"mcpServers\": { - \"problemreductions\": { - \"command\": \"pred\", - \"args\": [\"mcp\"] - } - } - } - -Test with MCP Inspector: - npx @modelcontextprotocol/inspector pred mcp")] - Mcp, -``` - -**Step 2: Add match arm in main.rs** - -In `main.rs`, add the module declaration (gated): - -```rust -#[cfg(feature = "mcp")] -mod mcp; -``` - -Add the match arm in the `match cli.command` block: - -```rust - #[cfg(feature = "mcp")] - Commands::Mcp => mcp::run(), -``` - -**Step 3: Create stub mcp module** - -Create `problemreductions-cli/src/mcp/mod.rs`: - -```rust -pub fn run() -> anyhow::Result<()> { - eprintln!("MCP server starting..."); - Ok(()) -} -``` - -**Step 4: Verify it compiles and runs** - -Run: `cargo run --features mcp -- mcp` -Expected: prints "MCP server starting..." and exits - -Run: `cargo run --no-default-features -- --help` -Expected: help output does NOT show `mcp` subcommand - -**Step 5: Commit** - -```bash -git add problemreductions-cli/src/cli.rs problemreductions-cli/src/main.rs problemreductions-cli/src/mcp/mod.rs -git commit -m "feat(cli): add mcp subcommand (stub)" -``` - ---- - -### Task 3: Implement McpServer with graph query tools (list, show, neighbors, path, export) - -These 6 tools are read-only graph metadata queries — no problem instances involved. - -**Files:** -- Create: `problemreductions-cli/src/mcp/tools.rs` -- Modify: `problemreductions-cli/src/mcp/mod.rs` - -**Step 1: Write a test for list_problems tool** - -Create `problemreductions-cli/src/mcp/tests.rs`: - -```rust -#[cfg(test)] -mod tests { - use super::*; - - #[test] - fn test_list_problems_returns_json() { - let server = McpServer::new(); - let result = server.list_problems_inner(); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert!(json["num_types"].as_u64().unwrap() > 0); - assert!(json["problems"].as_array().unwrap().len() > 0); - } - - #[test] - fn test_show_problem_known() { - let server = McpServer::new(); - let result = server.show_problem_inner("MIS"); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert_eq!(json["name"], "MaximumIndependentSet"); - } - - #[test] - fn test_show_problem_unknown() { - let server = McpServer::new(); - let result = server.show_problem_inner("NonExistent"); - assert!(result.is_err()); - } - - #[test] - fn test_find_path() { - let server = McpServer::new(); - let result = server.find_path_inner("MIS", "QUBO", "minimize-steps", false); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert!(json["path"].as_array().unwrap().len() > 0); - } - - #[test] - fn test_neighbors_to() { - let server = McpServer::new(); - let result = server.neighbors_inner("MIS", 1, "out"); - assert!(result.is_ok()); - } - - #[test] - fn test_export_graph() { - let server = McpServer::new(); - let result = server.export_graph_inner(); - assert!(result.is_ok()); - } -} -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test --features mcp -p problemreductions-cli mcp::tests` -Expected: FAIL — `McpServer` not defined - -**Step 3: Implement McpServer struct and graph query tools** - -In `problemreductions-cli/src/mcp/tools.rs`, implement `McpServer` with inner methods that return `anyhow::Result<String>` (JSON strings), and `#[tool_router]` annotated async wrappers that call these inner methods. - -The inner methods reuse the logic from `commands/graph.rs` — specifically they construct `ReductionGraph::new()`, call the same methods, and build the same JSON structures. Do NOT call the existing command functions directly (they write to stdout via `OutputConfig`). Instead, replicate the JSON construction logic. - -Key patterns: -- `McpServer` holds a `ToolRouter<McpServer>` field (required by rmcp macros) -- Each tool's async handler calls the sync inner method and converts to `CallToolResult` -- Error results use `CallToolResult::error(vec![Content::text(msg)])` -- Success results use `CallToolResult::success(vec![Content::text(json_string)])` -- Use `crate::problem_name::resolve_alias` for alias resolution -- Use `crate::problem_name::parse_problem_spec` for variant parsing - -See `commands/graph.rs` lines 82-96 for the `list` JSON format, lines 99-250 for `show`, lines 252-450 for `neighbors`, lines 452-607 for `path`, and the `export` function. - -Parameter structs (derive `Deserialize` + `schemars::JsonSchema`): - -```rust -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct ShowProblemParams { - #[schemars(description = "Problem name or alias (e.g., MIS, QUBO, MaximumIndependentSet)")] - pub problem: String, -} - -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct NeighborsParams { - #[schemars(description = "Problem name or alias")] - pub problem: String, - #[schemars(description = "Number of hops to explore (default: 1)")] - pub hops: Option<usize>, -} - -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct FindPathParams { - #[schemars(description = "Source problem name or alias")] - pub source: String, - #[schemars(description = "Target problem name or alias")] - pub target: String, - #[schemars(description = "Cost function: minimize-steps (default), or minimize:<field>")] - pub cost: Option<String>, - #[schemars(description = "Return all paths instead of just the cheapest")] - pub all: Option<bool>, -} -``` - -**Step 4: Update mod.rs with ServerHandler impl** - -In `mod.rs`: -- Import rmcp types -- Implement `ServerHandler` for `McpServer` with `#[tool_handler]` -- Implement `run()` using `tokio::runtime::Runtime::new()` (since main.rs is sync) -- The `run()` function: build runtime, create `McpServer`, call `server.serve(stdio()).await`, then `service.waiting().await` - -```rust -use rmcp::{ServiceExt, transport::stdio}; - -pub fn run() -> anyhow::Result<()> { - let rt = tokio::runtime::Runtime::new()?; - rt.block_on(async { - tracing_subscriber::fmt() - .with_writer(std::io::stderr) - .with_ansi(false) - .init(); - - let server = tools::McpServer::new(); - let service = server.serve(stdio()).await - .map_err(|e| anyhow::anyhow!("MCP server error: {e}"))?; - service.waiting().await - .map_err(|e| anyhow::anyhow!("MCP server error: {e}"))?; - Ok(()) - }) -} -``` - -**Step 5: Run tests** - -Run: `cargo test --features mcp -p problemreductions-cli mcp::tests` -Expected: all 6 tests PASS - -**Step 6: Manual test with MCP inspector** - -Run: `npx @modelcontextprotocol/inspector cargo run --features mcp -- mcp` -Expected: inspector shows 6 tools, calling `list_problems` returns problem data - -**Step 7: Commit** - -```bash -git add problemreductions-cli/src/mcp/ -git commit -m "feat(cli): implement MCP graph query tools (list, show, neighbors, path, export)" -``` - ---- - -### Task 4: Implement instance tools (create, inspect, evaluate, reduce, solve) - -These tools operate on problem instances passed as JSON strings. - -**Files:** -- Modify: `problemreductions-cli/src/mcp/tools.rs` -- Modify: `problemreductions-cli/src/mcp/tests.rs` - -**Step 1: Write tests for instance tools** - -Add to `tests.rs`: - -```rust - #[test] - fn test_create_problem_mis() { - let server = McpServer::new(); - let params = serde_json::json!({ - "edges": "0-1,1-2,2-3" - }); - let result = server.create_problem_inner("MIS", ¶ms); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert_eq!(json["type"], "MaximumIndependentSet"); - } - - #[test] - fn test_create_problem_sat() { - let server = McpServer::new(); - let params = serde_json::json!({ - "num_vars": 3, - "clauses": "1,2;-1,3" - }); - let result = server.create_problem_inner("SAT", ¶ms); - assert!(result.is_ok()); - } - - #[test] - fn test_inspect_problem() { - let server = McpServer::new(); - let problem_json = create_test_mis(); - let result = server.inspect_problem_inner(&problem_json); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert_eq!(json["type"], "MaximumIndependentSet"); - } - - #[test] - fn test_evaluate() { - let server = McpServer::new(); - let problem_json = create_test_mis(); - let result = server.evaluate_inner(&problem_json, &[1, 0, 1, 0]); - assert!(result.is_ok()); - } - - #[test] - fn test_reduce() { - let server = McpServer::new(); - let problem_json = create_test_mis(); - let result = server.reduce_inner(&problem_json, "QUBO"); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert!(json["target"].is_object()); - } - - #[test] - fn test_solve() { - let server = McpServer::new(); - let problem_json = create_test_mis(); - let result = server.solve_inner(&problem_json, Some("brute-force"), None); - assert!(result.is_ok()); - let json: serde_json::Value = serde_json::from_str(&result.unwrap()).unwrap(); - assert!(json["solution"].is_array()); - } - - fn create_test_mis() -> String { - let server = McpServer::new(); - let params = serde_json::json!({"edges": "0-1,1-2,2-3"}); - server.create_problem_inner("MIS", ¶ms).unwrap() - } -``` - -**Step 2: Run tests to verify they fail** - -Run: `cargo test --features mcp -p problemreductions-cli mcp::tests` -Expected: new tests FAIL — methods not defined - -**Step 3: Implement create_problem** - -Add `CreateProblemParams`: -```rust -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct CreateProblemParams { - #[schemars(description = "Problem type (e.g., MIS, SAT, QUBO, MaxCut). Use list_problems to see all types.")] - pub problem_type: String, - #[schemars(description = "Problem parameters as JSON object. Graph problems: {\"edges\": \"0-1,1-2\", \"weights\": \"1,2,3\"}. SAT: {\"num_vars\": 3, \"clauses\": \"1,2;-1,3\"}. QUBO: {\"matrix\": \"1,0.5;0.5,2\"}. KColoring: {\"edges\": \"0-1,1-2\", \"k\": 3}. Factoring: {\"target\": 15, \"bits_m\": 4, \"bits_n\": 4}. Random graph: {\"random\": true, \"num_vertices\": 10, \"edge_prob\": 0.3}")] - pub params: serde_json::Value, -} -``` - -The `create_problem_inner` method should: -1. Parse `params` to extract fields (edges, weights, clauses, etc.) — same logic as `CreateArgs` parsing in `commands/create.rs` -2. Build the `CreateArgs` struct and call the same construction logic -3. Return the JSON string of `ProblemJsonOutput` - -Key: the `create` function in `commands/create.rs` writes to `OutputConfig`. Instead, extract the problem construction logic into a shared helper that returns `ProblemJsonOutput`, or replicate the construction in the MCP tool. The simplest approach: build a `CreateArgs` from the JSON params, then call the same graph/SAT/QUBO construction code. Since `CreateArgs` is just a data struct, you can construct it directly. - -**Step 4: Implement inspect, evaluate, reduce, solve** - -These are simpler — they take `problem_json: String` and parse it as `ProblemJson`: - -- `inspect_problem_inner(problem_json)`: parse as `ProblemJson` or `ReductionBundle`, build the same JSON as `commands/inspect.rs` -- `evaluate_inner(problem_json, config)`: parse, `load_problem`, call `evaluate_dyn`, return result -- `reduce_inner(problem_json, target)`: parse, find path, execute reduction chain, serialize bundle -- `solve_inner(problem_json, solver, timeout)`: parse, `load_problem`, call solver, return result - -Reuse from `dispatch.rs`: `load_problem`, `ProblemJson`, `ReductionBundle`, `serialize_any_problem` -Reuse from `problem_name.rs`: `resolve_alias`, `parse_problem_spec` - -Parameter structs: -```rust -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct InspectParams { - #[schemars(description = "Problem JSON string (from create_problem) or reduction bundle JSON")] - pub problem_json: String, -} - -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct EvaluateParams { - #[schemars(description = "Problem JSON string (from create_problem)")] - pub problem_json: String, - #[schemars(description = "Configuration to evaluate as array of integers (e.g., [1, 0, 1, 0])")] - pub config: Vec<usize>, -} - -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct ReduceParams { - #[schemars(description = "Problem JSON string (from create_problem)")] - pub problem_json: String, - #[schemars(description = "Target problem type (e.g., QUBO, ILP, SpinGlass)")] - pub target: String, -} - -#[derive(Debug, serde::Deserialize, schemars::JsonSchema)] -pub struct SolveParams { - #[schemars(description = "Problem JSON string (from create_problem or reduce)")] - pub problem_json: String, - #[schemars(description = "Solver: 'ilp' (default) or 'brute-force'")] - pub solver: Option<String>, - #[schemars(description = "Timeout in seconds (0 = no limit, default: 0)")] - pub timeout: Option<u64>, -} -``` - -**Step 5: Run tests** - -Run: `cargo test --features mcp -p problemreductions-cli mcp::tests` -Expected: all tests PASS - -**Step 6: Manual test with MCP inspector** - -Run: `npx @modelcontextprotocol/inspector cargo run --features mcp -- mcp` -Expected: all 10 tools visible. Test create → solve pipeline. - -**Step 7: Commit** - -```bash -git add problemreductions-cli/src/mcp/ -git commit -m "feat(cli): implement MCP instance tools (create, inspect, evaluate, reduce, solve)" -``` - ---- - -### Task 5: Add MCP prompts - -**Files:** -- Create: `problemreductions-cli/src/mcp/prompts.rs` -- Modify: `problemreductions-cli/src/mcp/mod.rs` - -**Step 1: Implement prompts** - -In `prompts.rs`, define 3 prompt templates. MCP prompts are returned via `ServerHandler::list_prompts` and `ServerHandler::get_prompt`. - -Each prompt returns a `GetPromptResult` with a list of `PromptMessage` objects containing the prompt text. - -```rust -use rmcp::model::*; - -pub fn list_prompts() -> Vec<Prompt> { - vec![ - Prompt { - name: "analyze_problem".into(), - description: Some("Analyze a problem type: show details, variants, reductions, and suggest strategies".into()), - arguments: Some(vec![PromptArgument { - name: "problem_type".into(), - description: Some("Problem name or alias (e.g., MIS, QUBO)".into()), - required: Some(true), - }]), - }, - Prompt { - name: "reduction_walkthrough".into(), - description: Some("End-to-end reduction walkthrough: find path, create instance, reduce, solve, verify".into()), - arguments: Some(vec![ - PromptArgument { - name: "source".into(), - description: Some("Source problem (e.g., MIS)".into()), - required: Some(true), - }, - PromptArgument { - name: "target".into(), - description: Some("Target problem (e.g., QUBO)".into()), - required: Some(true), - }, - ]), - }, - Prompt { - name: "explore_graph".into(), - description: Some("Explore the full reduction graph: list all problems, show structure, highlight hub problems".into()), - arguments: None, - }, - ] -} - -pub fn get_prompt(name: &str, args: &std::collections::HashMap<String, String>) -> Option<GetPromptResult> { - match name { - "analyze_problem" => { - let problem_type = args.get("problem_type")?; - Some(GetPromptResult { - description: Some(format!("Analyze the {} problem type", problem_type)), - messages: vec![PromptMessage { - role: Role::User, - content: Content::text(format!( - "Please analyze the {} problem type using the problemreductions MCP tools:\n\ - 1. Use show_problem to get its details, variants, and available reductions\n\ - 2. Use neighbors_to to see what it can reduce to\n\ - 3. Use neighbors_from to see what reduces to it\n\ - 4. Suggest the most useful reduction strategies based on the graph structure\n\ - 5. Create a small example instance and solve it to demonstrate", - problem_type - )), - }], - }) - } - "reduction_walkthrough" => { - let source = args.get("source")?; - let target = args.get("target")?; - Some(GetPromptResult { - description: Some(format!("Reduction walkthrough: {} to {}", source, target)), - messages: vec![PromptMessage { - role: Role::User, - content: Content::text(format!( - "Please do an end-to-end reduction walkthrough from {} to {} using the problemreductions MCP tools:\n\ - 1. Use find_path to find the cheapest reduction path\n\ - 2. Explain what each step in the path does\n\ - 3. Use create_problem to create a small {} instance\n\ - 4. Use reduce to reduce it to {}\n\ - 5. Use solve to solve both the original and reduced instances\n\ - 6. Verify that the solutions are consistent", - source, target, source, target - )), - }], - }) - } - "explore_graph" => { - Some(GetPromptResult { - description: Some("Explore the reduction graph".to_string()), - messages: vec![PromptMessage { - role: Role::User, - content: Content::text( - "Please explore the problem reduction graph using the problemreductions MCP tools:\n\ - 1. Use list_problems to see all available problem types\n\ - 2. Use export_graph to get the full graph structure\n\ - 3. Identify the most connected hub problems (most reductions to/from)\n\ - 4. Show the key reduction pathways between major problem families\n\ - 5. Summarize the overall structure of the reduction landscape" - ), - }], - }) - } - _ => None, - } -} -``` - -**Step 2: Wire prompts into ServerHandler** - -In `mod.rs`, override `list_prompts` and `get_prompt` in the `ServerHandler` impl. Update `ServerCapabilities` to include `.enable_prompts()`. - -**Step 3: Run existing tests** - -Run: `cargo test --features mcp -p problemreductions-cli mcp::tests` -Expected: all previous tests still PASS - -**Step 4: Manual test** - -Run: `npx @modelcontextprotocol/inspector cargo run --features mcp -- mcp` -Expected: 3 prompts visible in the inspector's Prompts tab - -**Step 5: Commit** - -```bash -git add problemreductions-cli/src/mcp/prompts.rs problemreductions-cli/src/mcp/mod.rs -git commit -m "feat(cli): add MCP prompt templates (analyze, walkthrough, explore)" -``` - ---- - -### Task 6: Integration test - -**Files:** -- Create: `problemreductions-cli/tests/mcp_integration.rs` - -**Step 1: Write integration test** - -The test spawns `pred mcp` as a subprocess, sends JSON-RPC initialize + tools/list requests, and validates responses. - -```rust -#[cfg(feature = "mcp")] -#[test] -fn test_mcp_server_initialize_and_list_tools() { - use std::io::{BufRead, BufReader, Write}; - use std::process::{Command, Stdio}; - - let mut child = Command::new(env!("CARGO_BIN_EXE_pred")) - .arg("mcp") - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::null()) - .spawn() - .expect("Failed to start pred mcp"); - - let stdin = child.stdin.as_mut().unwrap(); - let stdout = BufReader::new(child.stdout.as_mut().unwrap()); - - // Send initialize request - let init_req = serde_json::json!({ - "jsonrpc": "2.0", - "id": 1, - "method": "initialize", - "params": { - "protocolVersion": "2024-11-05", - "capabilities": {}, - "clientInfo": {"name": "test", "version": "0.1.0"} - } - }); - let msg = serde_json::to_string(&init_req).unwrap(); - writeln!(stdin, "{}", msg).unwrap(); - stdin.flush().unwrap(); - - // Read response - let mut line = String::new(); - stdout.read_line(&mut line).unwrap(); - let response: serde_json::Value = serde_json::from_str(&line).unwrap(); - assert_eq!(response["result"]["protocolVersion"], "2024-11-05"); - - // Send tools/list - let list_req = serde_json::json!({ - "jsonrpc": "2.0", - "id": 2, - "method": "tools/list", - "params": {} - }); - let msg = serde_json::to_string(&list_req).unwrap(); - writeln!(stdin, "{}", msg).unwrap(); - stdin.flush().unwrap(); - - let mut line = String::new(); - stdout.read_line(&mut line).unwrap(); - let response: serde_json::Value = serde_json::from_str(&line).unwrap(); - let tools = response["result"]["tools"].as_array().unwrap(); - assert_eq!(tools.len(), 10); - - drop(stdin); // close stdin to terminate server - child.wait().unwrap(); -} -``` - -Note: the exact transport framing depends on rmcp's stdio implementation. It may use Content-Length headers (LSP-style) rather than newline-delimited JSON. Adjust the test accordingly after checking rmcp's stdio transport format. If it uses headers, you'll need to write/read `Content-Length: N\r\n\r\n{json}` frames. - -**Step 2: Run the integration test** - -Run: `cargo test --features mcp -p problemreductions-cli mcp_integration` -Expected: PASS - -**Step 3: Commit** - -```bash -git add problemreductions-cli/tests/mcp_integration.rs -git commit -m "test(cli): add MCP server integration test" -``` - ---- - -### Task 7: Add Makefile target and documentation - -**Files:** -- Modify: `Makefile` -- Create: `docs/book/src/mcp.md` -- Modify: `docs/book/src/SUMMARY.md` -- Modify: `README.md` - -**Step 1: Add Makefile target** - -Add to Makefile: -```makefile -mcp-test: ## Run MCP server tests - cargo test --features mcp -p problemreductions-cli mcp -``` - -**Step 2: Add mdBook page** - -Create `docs/book/src/mcp.md` with: -- What is MCP -- How to install and configure -- Available tools (table from design doc) -- Available prompts -- Example usage with Claude -- Testing with MCP Inspector - -**Step 3: Add to SUMMARY.md** - -Add `- [MCP Server](mcp.md)` to the book summary. - -**Step 4: Update README** - -Add a brief MCP section to README.md with install + configure instructions (3-4 lines). - -**Step 5: Verify docs build** - -Run: `make doc` -Expected: mdBook builds without errors - -**Step 6: Run all tests** - -Run: `make test clippy` -Expected: all tests pass, no clippy warnings - -**Step 7: Commit** - -```bash -git add Makefile docs/book/src/mcp.md docs/book/src/SUMMARY.md README.md -git commit -m "docs: add MCP server documentation and Makefile target" -``` - ---- - -### Task 8: Final verification - -**Step 1: Full test suite** - -Run: `make check` -Expected: fmt + clippy + test all pass - -**Step 2: Build release binary** - -Run: `cargo build --release -p problemreductions-cli` -Expected: builds successfully - -**Step 3: End-to-end manual test** - -Run: `npx @modelcontextprotocol/inspector ./target/release/pred mcp` - -Test this sequence in the inspector: -1. Call `list_problems` — verify problem list -2. Call `create_problem` with `{problem_type: "MIS", params: {edges: "0-1,1-2,2-3"}}` — verify JSON -3. Call `solve` with the returned JSON — verify solution -4. Call `find_path` with `{source: "MIS", target: "QUBO"}` — verify path -5. Call `reduce` with the MIS JSON and target "QUBO" — verify bundle -6. Check prompts tab — verify 3 prompts appear - -**Step 4: Verify no-mcp build still works** - -Run: `cargo build --no-default-features --features highs -p problemreductions-cli` -Expected: builds successfully, `pred --help` shows no `mcp` subcommand diff --git a/docs/plans/2026-02-25-overhead-system-design.md b/docs/plans/2026-02-25-overhead-system-design.md deleted file mode 100644 index 731d78211..000000000 --- a/docs/plans/2026-02-25-overhead-system-design.md +++ /dev/null @@ -1,138 +0,0 @@ -# Overhead System Redesign - -**Issue:** #61 — Introduce overhead system -**Date:** 2026-02-25 -**Approach:** Macro-first dual emission - -## Summary - -Replace the current `Polynomial`-based overhead system with a general `Expr` AST, compile-time macro-parsed expression strings, and per-problem inherent getters. The proc macro emits both compiled Rust code (for evaluation + compiler validation) and symbolic `Expr` AST literals (for composition + export). - -## Motivation - -Three pain points with the current system: -1. **Ergonomics** — `problem_size_names()`/`problem_size_values()` parallel arrays are awkward; `poly!` macro is verbose -2. **Correctness** — variable name mismatches between overhead expressions and problem size fields are caught only at runtime -3. **Simplification** — `Polynomial` only supports sums of monomials; general math (exp, log) requires a new representation anyway - -## Design - -### 1. Expression AST (`Expr`) - -Replaces `Polynomial` and `Monomial` with a general math expression tree. - -```rust -// src/expr.rs (replaces src/polynomial.rs) - -#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] -pub enum Expr { - Const(f64), - Var(&'static str), - Add(Box<Expr>, Box<Expr>), - Mul(Box<Expr>, Box<Expr>), - Pow(Box<Expr>, Box<Expr>), - Exp(Box<Expr>), - Log(Box<Expr>), - Sqrt(Box<Expr>), -} -``` - -Key operations: -- `eval(&self, vars: &ProblemSize) -> f64` -- `substitute(&self, mapping: &HashMap<&str, &Expr>) -> Expr` -- `variables(&self) -> HashSet<&'static str>` -- `is_polynomial(&self) -> bool` -- `degree(&self) -> Option<u32>` -- `Display` for human-readable formulas -- `simplify(&self) -> Expr` — minimal constant folding - -### 2. Problem Getters - -Remove `problem_size_names()` and `problem_size_values()` from the `Problem` trait. Each problem type implements inherent getter methods instead. - -```rust -// Before: trait methods returning parallel arrays -impl Problem for MaximumIndependentSet<SimpleGraph, i32> { - fn problem_size_names() -> &'static [&'static str] { &["num_vertices", "num_edges"] } - fn problem_size_values(&self) -> Vec<usize> { - vec![self.graph().num_vertices(), self.graph().num_edges()] - } -} - -// After: inherent methods — natural, compiler-checked, IDE-friendly -impl<G: Graph, W: WeightElement> MaximumIndependentSet<G, W> { - pub fn num_vertices(&self) -> usize { self.graph().num_vertices() } - pub fn num_edges(&self) -> usize { self.graph().num_edges() } -} -``` - -### 3. Proc Macro — Dual Emission - -The `#[reduction]` macro parses expression strings at compile time and emits two outputs. - -User-facing syntax: -```rust -#[reduction(overhead = { - num_vars = "num_vertices", - num_constraints = "num_edges + num_vertices^2", -})] -impl ReduceTo<QUBO<f64>> for MaximumIndependentSet<SimpleGraph, i32> { ... } -``` - -Macro emits: -1. **Compiled evaluation function** — `src.num_vertices()`, `src.num_edges()` calls. Compiler catches missing getters. -2. **Symbolic Expr AST** — `Expr::Add(...)` construction for composition/export. - -Expression grammar (Pratt parser, ~200 LOC in proc macro crate): -``` -expr = term (('+' | '-') term)* -term = factor (('*' | '/') factor)* -factor = base ('^' factor)? -base = NUMBER | IDENT | func_call | '(' expr ')' -func_call = ('exp' | 'log' | 'sqrt') '(' expr ')' -``` - -### 4. Updated `ReductionOverhead` and `ReductionEntry` - -```rust -pub struct ReductionOverhead { - pub output_size: Vec<(&'static str, Expr)>, // Expr replaces Polynomial -} - -pub struct ReductionEntry { - // ...existing fields... - pub overhead_fn: fn() -> ReductionOverhead, // symbolic (composition/export) - pub overhead_eval_fn: fn(&dyn Any) -> ProblemSize, // compiled (evaluation) - // REMOVED: source_size_names_fn, target_size_names_fn -} -``` - -`PathCostFn` uses the symbolic `ReductionOverhead` (via `Expr::eval`) since it operates on type-erased `ProblemSize` during graph traversal. - -### 5. Export Pipeline - -JSON format gains both structured AST and display string: -```json -{ - "overhead": [{ - "field": "num_vars", - "expr": {"Pow": [{"Var": "num_vertices"}, {"Const": 2.0}]}, - "formula": "num_vertices^2" - }] -} -``` - -The paper reads `formula` strings — no Typst code changes needed. - -## Migration Strategy - -| Phase | Description | Files | Risk | -|-------|-------------|-------|------| -| 1 | Add `Expr` type alongside `Polynomial` | 2-3 new | Low (additive) | -| 2 | Update proc macro with Pratt parser, support new syntax | 1 file | Medium | -| 3 | Add inherent getters to all problem types | ~15 model files | Low (additive) | -| 4 | Migrate all reductions to new syntax | ~20 rule files | Low (mechanical) | -| 5 | Remove deprecated APIs (`problem_size_*`, `Polynomial`, `poly!`) | ~10 files | Medium (breaking) | -| 6 | Update documentation and regenerate exports | 3-4 files | Low | - -Phases 1-3 are purely additive. Phase 4 is bulk migration. Phase 5 is cleanup. diff --git a/docs/plans/2026-02-27-binpacking-model.md b/docs/plans/2026-02-27-binpacking-model.md deleted file mode 100644 index b576b45e2..000000000 --- a/docs/plans/2026-02-27-binpacking-model.md +++ /dev/null @@ -1,96 +0,0 @@ -# Plan: Add BinPacking Model - -**Issue:** #95 — [Model] BinPacking -**Skill:** add-model (Steps 1–7) - -## Overview - -Add a `BinPacking` optimization model: given items with sizes and a bin capacity, minimize the number of bins used to pack all items such that no bin exceeds capacity. - -## Design Decisions - -- **Category:** `specialized` — BinPacking is a domain-specific packing/scheduling problem. It doesn't fit `graph/` (no graph), `set/` (not subset selection), `optimization/` (reserved for generic formulations like QUBO/ILP), or `satisfiability/`. -- **Struct:** `BinPacking<W = i32>` with fields `sizes: Vec<W>` and `capacity: W`. Generic over weight type W for integer or real-valued sizes. -- **dims():** `vec![n; n]` where n = number of items. Each variable is a bin index in {0, ..., n−1}. This is the first non-binary configuration space in the codebase. -- **Objective:** Minimize the count of distinct bin indices used (always `i32`, regardless of W). So `Metric = SolutionSize<i32>`, `Value = i32`. -- **Feasibility:** For each bin j, the sum of sizes of items assigned to j must not exceed capacity. Uses `WeightElement::to_sum()` for size summation and capacity comparison. -- **variant():** `variant_params![W]` — exposes weight type (i32, f64). -- **Solver:** BruteForce (existing) — enumerates all n^n assignments. No ILP reduction in this PR. - -## Steps - -### Step 1: Determine category -Category: `specialized/` - -### Step 2: Implement the model -Create `src/models/specialized/bin_packing.rs`: - -```rust -// Structure: -// 1. inventory::submit! for ProblemSchemaEntry -// 2. BinPacking<W> struct with sizes: Vec<W>, capacity: W -// 3. Constructor: new(sizes, capacity), with_unit_sizes(sizes, capacity) if W: From<i32> -// 4. Accessors: sizes(), capacity(), num_items() -// 5. Problem impl: NAME="BinPacking", Metric=SolutionSize<i32>, dims()=vec![n;n] -// 6. evaluate(): check bin capacities, count distinct bins -// 7. OptimizationProblem impl: Value=i32, direction=Minimize -// 8. #[cfg(test)] #[path] link -``` - -Key implementation details for `evaluate()`: -``` -1. Group items by assigned bin index -2. For each bin, sum sizes via to_sum() and compare with capacity.to_sum() -3. If any bin exceeds capacity → SolutionSize::Invalid -4. Otherwise → SolutionSize::Valid(num_distinct_bins as i32) -``` - -### Step 3: Register the model -1. `src/models/specialized/mod.rs` — add `pub(crate) mod bin_packing;` and `pub use bin_packing::BinPacking;` -2. `src/models/mod.rs` — add `BinPacking` to the `specialized` re-export line - -### Step 4: Register in CLI -1. `problemreductions-cli/src/dispatch.rs`: - - `load_problem()`: add `"BinPacking" => deser_opt::<BinPacking<i32>>(data)` - - `serialize_any_problem()`: add `"BinPacking" => try_ser::<BinPacking<i32>>(any)` -2. `problemreductions-cli/src/problem_name.rs`: - - `resolve_alias()`: add `"binpacking" => "BinPacking".to_string()` - - Optionally add `("BP", "BinPacking")` to `ALIASES` - -### Step 5: Write unit tests -Create `src/unit_tests/models/specialized/bin_packing.rs`: - -Tests: -- `test_binpacking_creation` — construct instance, verify num_items, dims -- `test_binpacking_evaluation_valid` — valid packing returns SolutionSize::Valid(num_bins) -- `test_binpacking_evaluation_invalid` — overloaded bin returns SolutionSize::Invalid -- `test_binpacking_direction` — verify Direction::Minimize -- `test_binpacking_solver` — BruteForce finds optimal 3-bin solution for the example instance (6 items, sizes [6,6,5,5,4,4], capacity 10) -- `test_binpacking_serialization` — round-trip serde test - -Example instance from issue: -- 6 items, capacity C = 10, sizes = [6, 6, 5, 5, 4, 4] -- Optimal: 3 bins, e.g., x = (0, 1, 2, 2, 0, 1) - -### Step 6: Document in paper -Update `docs/paper/reductions.typ`: -1. Add to `display-name` dictionary: `"BinPacking": [Bin Packing]` -2. Add `#problem-def("BinPacking")[...]` block with mathematical definition - -### Step 7: Verify -```bash -make check # fmt + clippy + test -``` -Then run `/review-implementation` to verify completeness. - -## Files Changed - -| File | Action | -|------|--------| -| `src/models/specialized/bin_packing.rs` | **Create** — model implementation | -| `src/unit_tests/models/specialized/bin_packing.rs` | **Create** — unit tests | -| `src/models/specialized/mod.rs` | **Edit** — register module | -| `src/models/mod.rs` | **Edit** — add re-export | -| `problemreductions-cli/src/dispatch.rs` | **Edit** — CLI dispatch | -| `problemreductions-cli/src/problem_name.rs` | **Edit** — alias | -| `docs/paper/reductions.typ` | **Edit** — paper definition | diff --git a/docs/plans/2026-03-01-meta-power-design.md b/docs/plans/2026-03-01-meta-power-design.md deleted file mode 100644 index 60bcc35f6..000000000 --- a/docs/plans/2026-03-01-meta-power-design.md +++ /dev/null @@ -1,54 +0,0 @@ -# Design: meta-power skill - -## Purpose - -Batch-resolve open `[Model]` and `[Rule]` GitHub issues end-to-end with full autonomy: plan, implement, review, fix, merge. - -## Architecture - -**Outer orchestrator** pattern: meta-power runs in the main Claude session and shells out to `make run-plan` for each issue's implementation. This keeps the orchestrator's context clean while delegating heavy work to subprocess sessions. - -## Pipeline per Issue - -``` -Phase 1: Plan /issue-to-pr <number> → branch + PR with plan -Phase 2: Execute make run-plan → subprocess implements the plan -Phase 3: Review push, make copilot-review -Phase 4: Fix loop (up to 3 retries) - sleep 5m → /fix-pr → push → sleep 5m → check CI - if CI green → break -Phase 5: Merge gh pr merge --squash -Phase 6: Sync git checkout main && git pull -``` - -## Ordering - -1. All `[Model]` issues first (ascending issue number) -2. All `[Rule]` issues second (ascending issue number) - -No DAG — models-first is sufficient since rules depend on models. - -## Error Handling - -Every failure → log + skip to next issue. Never block the batch. - -| Phase | Failure | Action | -|-------|---------|--------| -| Plan | Validation fails | Skip | -| Execute | Subprocess exits non-zero | Skip | -| Fix loop | 3 retries exhausted | Leave PR open, skip | -| Merge | Conflict | Leave PR open, skip | - -## Parameters - -- `MAX_RETRIES = 3` -- `CI_WAIT = 5 minutes` -- Auto-merge: yes (squash) -- Summary table printed at end - -## Design Decisions - -- **Why outer orchestrator?** Each `make run-plan` gets a fresh 500-turn context. The outer session just monitors and coordinates. -- **Why models-first only?** Rules rarely depend on each other. If a rule's source model is missing, `issue-to-pr` validation catches it and skips. -- **Why 3 retries?** Most fixable issues resolve in 1-2 rounds. More retries burn tokens on genuinely hard problems. -- **Why auto-merge?** Full CI + Copilot review provides sufficient quality gate. The point of the skill is batch autonomy. diff --git a/docs/plans/2026-03-01-problem-categorization-design.md b/docs/plans/2026-03-01-problem-categorization-design.md deleted file mode 100644 index 4db0704f6..000000000 --- a/docs/plans/2026-03-01-problem-categorization-design.md +++ /dev/null @@ -1,84 +0,0 @@ -# Problem Categorization Redesign - -## Problem - -The current categorization mixes two axes: -- **Input structure** (graph, set system, formula) -- **Problem type** (optimization vs satisfaction) - -This creates ambiguity: MIS is both a graph problem and an optimization problem. QUBO is on a matrix but lives in `optimization/`. CircuitSAT is a satisfiability problem but lives in `specialized/`. The `specialized/` folder is a catch-all with no unifying principle. - -## Design - -**Single axis: primary input structure** — "what data type does the problem operate on?" - -The optimization/satisfaction distinction is already captured by the trait hierarchy (`OptimizationProblem` vs `SatisfactionProblem`), so folders should not duplicate it. - -### New folder structure - -``` -src/models/ -├── graph/ # Input: a graph (optionally weighted) -│ ├── maximum_independent_set.rs -│ ├── maximum_clique.rs -│ ├── max_cut.rs -│ ├── maximum_matching.rs -│ ├── minimum_vertex_cover.rs -│ ├── minimum_dominating_set.rs -│ ├── maximal_is.rs -│ ├── kcoloring.rs -│ ├── traveling_salesman.rs -│ ├── spin_glass.rs ← from optimization/ -│ └── biclique_cover.rs ← from specialized/ -│ -├── formula/ # Input: a logical formula or circuit -│ ├── sat.rs -│ ├── ksat.rs -│ └── circuit.rs ← from specialized/ -│ -├── set/ # Input: universe + collection of subsets -│ ├── minimum_set_covering.rs -│ └── maximum_set_packing.rs -│ -├── algebraic/ # Input: matrix, linear system, or lattice -│ ├── qubo.rs ← from optimization/ -│ ├── ilp.rs ← from optimization/ -│ ├── closest_vector_problem.rs ← from optimization/ -│ └── bmf.rs ← from specialized/ -│ -└── misc/ # Problems with unique input structures - ├── bin_packing.rs ← from optimization/ - ├── paintshop.rs ← from specialized/ - └── factoring.rs ← from specialized/ -``` - -### Decision rule for new problems - -> "What is the primary data structure in the struct definition?" -> - Graph → `graph/` -> - Boolean formula or circuit → `formula/` -> - Universe + subsets → `set/` -> - Matrix, linear system, or lattice → `algebraic/` -> - None of the above → `misc/` - -### What moves - -| Problem | From | To | Reason | -|---|---|---|---| -| SpinGlass | optimization/ | graph/ | Parameterized by G, operates on graph edges | -| BicliqueCover | specialized/ | graph/ | Input is a BipartiteGraph | -| CircuitSAT | specialized/ | formula/ | Input is a boolean circuit | -| QUBO | optimization/ | algebraic/ | Input is a Q matrix (no graph param) | -| ILP | optimization/ | algebraic/ | Input is constraint matrix + objective | -| CVP | optimization/ | algebraic/ | Input is lattice basis matrix | -| BMF | specialized/ | algebraic/ | Input is a boolean matrix | -| BinPacking | optimization/ | misc/ | Input is items + capacity | -| PaintShop | specialized/ | misc/ | Input is a car sequence | -| Factoring | specialized/ | misc/ | Input is an integer | - -### What doesn't change - -- Trait hierarchy (`OptimizationProblem`, `SatisfactionProblem`) -- All public API, type names, re-exports -- Only `mod.rs` files, `use` paths, and `#[path]` test references change -- The `optimization/` and `specialized/` folders are eliminated diff --git a/docs/plans/2026-03-11-asymptotic-normal-form-redesign-design.md b/docs/plans/2026-03-11-asymptotic-normal-form-redesign-design.md deleted file mode 100644 index 61ac37d23..000000000 --- a/docs/plans/2026-03-11-asymptotic-normal-form-redesign-design.md +++ /dev/null @@ -1,462 +0,0 @@ -# Asymptotic Normal Form Redesign - -**Date:** 2026-03-11 -**Approach:** Explicit exact-canonicalization layer plus Big-O projection - -## Summary - -Redesign expression normalization into two explicit phases: - -1. `canonical_form(expr)` performs exact symbolic simplification with signed coefficients. -2. `big_o_normal_form(expr)` projects the canonical expression into an asymptotic growth class. -3. `pred-sym` exposes the symbolic engine directly from a separate test-oriented binary for debugging and regression reproduction. - -The existing `asymptotic_normal_form(expr)` remains temporarily as a compatibility wrapper to `big_o_normal_form(expr)`. - -This change fixes three current pain points: - -- duplicate additive terms survive composition (`O(x + x)` instead of `O(x)`) -- exact formulas with negative coefficients fail normalization and fall back to raw display -- `Pow(_, 0.5)` renders ambiguously in user-facing output (`2^n^0.5`) - -## Current Status (2026-03-12) - -Design is complete, but the redesign itself is not implemented yet. - -Observed branch status: - -- `src/expr.rs` still exposes only the legacy `asymptotic_normal_form()` pipeline. -- `canonical_form()` and `big_o_normal_form()` do not exist yet. -- `problemreductions-cli/src/commands/graph.rs` still formats Big-O through `asymptotic_normal_form()`. -- `problemreductions-cli/Cargo.toml` still exposes only the `pred` binary; `pred-sym` has not been added. -- `src/unit_tests/expr.rs` still encodes legacy behavior such as rejecting negative forms. -- The working tree already contains local WIP in `src/expr.rs`, related tests, and CLI files. That WIP should be reviewed before broad refactors so we do not overwrite useful progress. - -One prerequisite has already landed separately: the CLI fallback now always wraps non-normalizable expressions in `O(...)`. This plan assumes that fix stays as-is while the engine is redesigned underneath it. - -## Problems With The Current Design - -The current `asymptotic_normal_form()` in `src/expr.rs` tries to do two jobs at once: - -- algebraic simplification -- asymptotic reduction - -That conflation makes the function brittle. It rejects subtraction because negative coefficients are treated as unsupported, even when the overall expression is a valid exact size formula. It also deduplicates only partially because nested additive structure is flattened incompletely. As a result, the function is too strict for exact formulas and too weak as a canonicalizer. - -The public API also hides the distinction between exact symbolic equivalence and Big-O equivalence. That makes callers and tests harder to reason about and pushes formatting work into ad hoc fallbacks. - -## Goals - -- Preserve exact symbolic structure before asymptotic dropping. -- Support signed polynomial-style expressions such as `n^3 - n^2 + 2*n + 4*n*m`. -- Keep current safe transcendental identities: - - `sqrt(x)` and `x^(1/2)` - - `exp(a) * exp(b)` - - `c^a * c^b` for constant positive `c` - - simple `log` reductions already supported today -- Produce deterministic canonical ASTs and deterministic display order. -- Make Big-O output conservative: only drop terms when dominance is provable. -- Provide a lightweight separate binary for symbolic normalization testing without going through the full graph explorer. - -## Non-Goals - -- Build a general computer algebra system. -- Prove arbitrary symbolic inequalities across `exp`, `log`, and nested non-polynomial forms. -- Change the `Expr` surface syntax or parser grammar. -- Remove `asymptotic_normal_form()` immediately. - -## Public API - -Add two new public functions in `src/expr.rs` and re-export them from `src/lib.rs`: - -```rust -pub fn canonical_form(expr: &Expr) -> Result<Expr, CanonicalizationError>; -pub fn big_o_normal_form(expr: &Expr) -> Result<Expr, AsymptoticAnalysisError>; -``` - -`asymptotic_normal_form(expr)` remains public for compatibility but becomes a thin wrapper: - -```rust -pub fn asymptotic_normal_form(expr: &Expr) -> Result<Expr, AsymptoticAnalysisError> { - big_o_normal_form(expr) -} -``` - -This makes the contract explicit: - -- `canonical_form()` means exact symbolic normalization -- `big_o_normal_form()` means asymptotic projection -- `asymptotic_normal_form()` means legacy alias - -## Companion CLI Tool - -Add a second binary to `problemreductions-cli`: - -```toml -[[bin]] -name = "pred-sym" -path = "src/bin/pred-sym.rs" -``` - -`pred-sym` is a focused test/developer tool for the symbolic engine. It should remain a separate binary, share parsing and formatting helpers with `pred` where practical, and keep its command surface narrow. - -Recommended commands: - -- `pred-sym parse '<expr>'` - - parse and echo the raw AST / formula -- `pred-sym canon '<expr>'` - - run `canonical_form()` -- `pred-sym big-o '<expr>'` - - run `big_o_normal_form()` -- `pred-sym compare '<expr-a>' '<expr-b>'` - - compare exact canonical forms and, optionally, Big-O-normal forms -- `pred-sym eval '<expr>' --vars n=10,m=20` - - evaluate expressions against a `ProblemSize`-style variable map - -Output modes: - -- human-readable text by default -- `--json` for ASTs, canonical forms, and normalization results - -This tool serves three purposes: - -- reproducible debugging for normalization bugs -- easier test-case authoring during engine development -- isolated testing of symbolic behavior independent of reduction metadata - -Design choice: prefer a separate binary over a `pred sym ...` subcommand. `pred-sym` is mainly a testing harness for the symbolic engine, so keeping it outside the main `pred` command surface avoids mixing developer/test workflows into the user-facing reduction explorer. - -## Canonicalization Model - -`canonical_form()` should normalize through a small internal algebra layer rather than by rewriting the AST directly. - -Recommended internal model: - -- a canonical sum of terms -- each term contains: - - one signed numeric coefficient - - a canonical multiset of multiplicative factors - -For polynomial pieces, factors collapse into monomials such as `n^3 * m`. For supported non-polynomial pieces, the canonicalized subexpression is treated as an opaque factor atom, for example: - -- `exp(m + n)` -- `2^(m + n)` -- `log(n)` -- `n^0.5` - -This is intentionally bounded. It provides exact cancellation and deduplication without requiring general symbolic theorem proving. - -## Canonicalization Rules - -### Addition - -- Flatten nested `Add` -- Canonicalize children first -- Merge equal factor-multisets by summing coefficients -- Drop zero-coefficient terms -- Sort terms deterministically - -Examples: - -- `n + n` -> `2 * n` -- `n - n` -> `0` -- `n + n - m + 2 * m` -> `2 * n + m` - -### Multiplication - -- Flatten nested `Mul` -- Multiply numeric coefficients -- Merge repeated factors into powers -- Preserve current safe identities: - - `exp(a) * exp(b)` -> `exp(a + b)` - - `2^a * 2^b` -> `2^(a + b)` - - `sqrt(x)` -> `x^0.5` -- Sort factors deterministically - -Examples: - -- `n * n^(1/2)` -> `n^1.5` -- `2^n * 2^m` -> `2^(m + n)` - -### Powers And Logs - -- Constant folding remains allowed where safe -- Negative exponents remain unsupported for Big-O purposes because they imply division -- Existing simple `log` identities stay: - - `log(exp(x)) -> x` - - `log(c^x) -> x` for constant positive `c` - - `log(x^k) -> log(x)` for constant positive `k` - -## Big-O Projection - -`big_o_normal_form()` operates only on canonical output. - -### Additive Dominance - -For sums, keep only terms not dominated by another term. - -Dominance rule: - -- use the existing conservative monomial comparison model from `src/rules/analysis.rs` -- drop a term only when dominance is provable -- keep incomparable terms - -Examples: - -- `n^3 + n^2` -> `n^3` -- `n^2 + n * m` -> `n^2 + n * m` -- `n^3 - n^2 + 2 * n + 4 * n * m` -> `n^3 + n * m` - -Negative lower-order terms disappear naturally because they do not contribute to the dominant positive growth class. - -### Multiplicative Projection - -- discard overall nonzero constant factors -- preserve canonical symbolic products otherwise -- do not perform extra “improvement” beyond canonicalization - -Examples: - -- `3 * n^2` -> `n^2` -- `-4 * n^2` should be rejected as an asymptotic growth form unless embedded in a larger expression with surviving positive dominant terms - -### Fallback Policy - -For supported but incomparable opaque terms, keep all survivors rather than erroring. - -Examples: - -- `exp(n) + n^10` stays as `exp(n) + n^10` if no safe dominance rule exists -- `2^sqrt(n) + n^3` stays as both terms unless the comparison logic is extended - -Errors should be reserved for genuinely invalid asymptotic outputs: - -- division / negative exponents -- expressions with no surviving nonzero growth term -- sign-only or zero-only results that cannot be represented meaningfully as Big-O - -## Display Rules - -User-facing formatting should avoid precedence ambiguity. The preferred output for `Pow(base, Const(0.5))` is `sqrt(base)` when the exponent is exactly `0.5`. - -This can be implemented either: - -- in `Display for Expr`, or -- in a Big-O-specific formatter used by the CLI - -Recommendation: keep canonical ASTs numeric (`0.5`) and improve display formatting. That preserves algebraic uniformity while fixing the ambiguous `O(2^num_vertices^0.5)` output. - -## Implementation Plan - -### Phase 0: Reconcile Current WIP - -**Status:** In progress - -Before changing architecture, review the existing local modifications in: - -- `src/expr.rs` -- `src/unit_tests/expr.rs` -- `src/unit_tests/rules/analysis.rs` -- `problemreductions-cli/src/commands/graph.rs` -- adjacent CLI/test files already modified in the working tree - -Goal: - -- identify which edits are unrelated and should be preserved as-is -- identify any edits that already move toward this redesign -- avoid rewriting user work accidentally - -Exit criteria: - -- current local changes categorized as either “keep and adapt” or “leave untouched” -- baseline compile/test command chosen before structural work begins - -### Phase 1: Add Exact Canonicalization - -**Status:** Not started - -Implement `canonical_form()` and its internal term model in `src/expr.rs`. - -Scope: - -- exact signed coefficients -- additive flattening and deduplication -- multiplicative flattening and power merging -- deterministic rebuild to `Expr` -- bounded transcendental identities already described above - -Files: - -- `src/expr.rs` -- `src/lib.rs` -- `src/unit_tests/expr.rs` - -Exit criteria: - -- `canonical_form()` exists and is exported -- exact-form tests pass -- legacy `asymptotic_normal_form()` behavior is unchanged for existing callers - -### Phase 2: Add Big-O Projection - -**Status:** Not started - -Implement `big_o_normal_form()` on top of `canonical_form()`, then convert `asymptotic_normal_form()` into a compatibility wrapper. - -Scope: - -- dominant-term extraction for provable polynomial dominance -- conservative fallback for incomparable opaque terms -- rejection of invalid negative-power / division forms - -Files: - -- `src/expr.rs` -- `src/lib.rs` -- `src/unit_tests/expr.rs` - -Exit criteria: - -- `big_o_normal_form()` exists -- duplicate additive terms collapse correctly -- signed polynomial exact counts produce useful Big-O output -- `asymptotic_normal_form()` delegates to the new implementation - -### Phase 3: Integrate Callers And Display - -**Status:** Not started - -Update current call sites to use the explicit APIs and fix ambiguous formatting of `^0.5`. - -Scope: - -- switch CLI formatting to `big_o_normal_form()` -- decide whether `sqrt(...)` rendering lives in `Display` or a Big-O formatter -- update rule-analysis preparation paths to use `canonical_form()` and/or `big_o_normal_form()` intentionally - -Files: - -- `problemreductions-cli/src/commands/graph.rs` -- `src/rules/analysis.rs` -- `src/expr.rs` -- affected tests - -Exit criteria: - -- CLI no longer depends on the legacy name internally -- no ambiguous `2^n^0.5`-style output remains -- analysis code uses the new API deliberately rather than via legacy fallback - -### Phase 4: Add `pred-sym` - -**Status:** Not started - -Add the separate test-oriented binary for symbolic-engine inspection. - -Scope: - -- `parse` -- `canon` -- `big-o` -- `compare` -- `eval` -- optional `--json` - -Files: - -- `problemreductions-cli/Cargo.toml` -- `problemreductions-cli/src/bin/pred-sym.rs` -- shared helper modules if needed -- CLI tests - -Exit criteria: - -- `pred-sym` builds as a separate binary -- symbolic engine behavior can be reproduced without invoking `pred show` -- CLI output is stable enough for regression tests - -### Phase 5: Expand And Rebaseline Tests - -**Status:** Not started - -Split tests by layer and replace obsolete expectations from the old engine. - -Scope: - -- exact canonicalization tests -- Big-O projection tests -- CLI regression tests for `pred` -- CLI tests for `pred-sym` - -Exit criteria: - -- negative-form tests reflect the new split API instead of the old monolithic rejection model -- ILP/TSP/composed-overhead regression cases are covered -- old expectations that rely on accidental formatting are removed - -## Test Matrix - -### Canonical Form - -- flatten nested sums and products -- combine duplicate additive terms -- combine repeated multiplicative factors -- preserve signed exact formulas -- cancel zero terms -- preserve deterministic order -- preserve current transcendental identities - -Key cases: - -- `n + n` -> `2 * n` -- `n - n` -> `0` -- `n + n - m + 2 * m` -> `2 * n + m` -- `n * n^(1/2)` -> `n^1.5` -- `2^n * 2^m` -> `2^(m + n)` -- `sqrt(n * m)` canonicalizes equivalently to `(n * m)^(1/2)` -- `n^3 - n^2 + 2 * n + 4 * n * m` remains exact - -### Big-O Normal Form - -- duplicate composed terms collapse -- lower-order positive terms drop -- lower-order negative terms drop -- incomparable dominant terms remain -- invalid negative-power forms still error - -Key cases: - -- `n + n` -> `n` -- `n^3 - n^2 + 2 * n + 4 * n * m` -> `n^3 + n * m` -- `(n + m) + (m + n)` -> `m + n` -- `2^(sqrt(n))` displays clearly as `2^sqrt(n)` or `2^(sqrt(n))` - -### CLI Regressions - -- ILP complexity always renders as `O(...)` -- TSP `num_constraints` shows a simplified dominant Big-O expression -- composed reduction overheads do not show duplicate additive terms -- no ambiguous `^0.5` formatting in Big-O output - -### `pred-sym` CLI - -- `pred-sym canon 'n + n'` prints `2 * n` -- `pred-sym big-o 'n^3 - n^2 + 2*n + 4*n*m'` prints `n^3 + n * m` -- `pred-sym big-o '2^(n^0.5)'` renders using `sqrt(n)` -- `pred-sym eval 'n + m' --vars n=3,m=4` prints `7` -- `pred-sym --json ...` returns stable machine-readable output - -## Risks And Tradeoffs - -- `canonical_form()` adds internal complexity compared with the current tree-rewrite approach. -- Reusing the monomial-dominance logic from `src/rules/analysis.rs` reduces duplication conceptually, but care is needed to avoid circular abstractions or subtly different semantics. -- Keeping `asymptotic_normal_form()` as an alias avoids breakage, but it also extends the migration window. A later cleanup should deprecate it formally. - -## Recommendation - -Implement the redesign as an explicit two-phase pipeline: - -1. exact symbolic canonicalization -2. conservative Big-O projection - -This is the smallest design that fixes the known bugs while giving the expression system a clear contract for future work. diff --git a/docs/plans/2026-03-12-asymptotic-normal-form-redesign-impl.md b/docs/plans/2026-03-12-asymptotic-normal-form-redesign-impl.md deleted file mode 100644 index 9ba82cffe..000000000 --- a/docs/plans/2026-03-12-asymptotic-normal-form-redesign-impl.md +++ /dev/null @@ -1,1717 +0,0 @@ -# Asymptotic Normal Form Redesign — Implementation Plan - -> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. - -**Goal:** Redesign expression normalization into two explicit phases: exact `canonical_form()` and asymptotic `big_o_normal_form()`, plus a `pred-sym` CLI tool for debugging. - -**Architecture:** The existing monolithic `asymptotic_normal_form()` is split into two layers. `canonical_form()` does exact symbolic simplification (signed coefficients, term merging, deterministic ordering). `big_o_normal_form()` projects the canonical result into an asymptotic growth class by dropping dominated terms and constant factors. A `pred-sym` binary exposes the symbolic engine directly. - -**Tech Stack:** Rust, `problemreductions` crate (`src/expr.rs`), `problemreductions-cli` crate, `clap` CLI framework. - ---- - -## File Structure - -### New files -- `src/canonical.rs` — `CanonicalTerm`, `CanonicalSum`, `canonical_form()`, internal algebra model -- `src/big_o.rs` — `big_o_normal_form()`, additive dominance, multiplicative projection -- `src/unit_tests/canonical.rs` — Tests for exact canonicalization -- `src/unit_tests/big_o.rs` — Tests for Big-O projection -- `problemreductions-cli/src/bin/pred_sym.rs` — `pred-sym` binary - -### Modified files -- `src/expr.rs` — Add `CanonicalizationError`, keep `asymptotic_normal_form()` as wrapper -- `src/lib.rs` — Re-export new public functions -- `problemreductions-cli/Cargo.toml` — Add `pred-sym` binary -- `problemreductions-cli/src/commands/graph.rs` — Switch to `big_o_normal_form()` - -### Reference files (read, don't modify unless specified) -- `src/rules/analysis.rs:96-263` — Existing `Monomial` / `NormalizedPoly` / `normalize_polynomial()` for dominance comparison -- `src/unit_tests/expr.rs` — Existing tests for `asymptotic_normal_form()` (some expectations change) -- `docs/plans/2026-03-11-asymptotic-normal-form-redesign-design.md` — Design document - ---- - -## Chunk 1: Exact Canonicalization Engine - -### Task 1: Create canonical.rs with internal algebra model - -**Files:** -- Create: `src/canonical.rs` -- Create: `src/unit_tests/canonical.rs` -- Modify: `src/expr.rs` (add `CanonicalizationError` enum) -- Modify: `src/lib.rs` (add module declarations and re-exports) - -The internal model is a **canonical sum of terms**, where each term is a signed coefficient times a canonical multiset of factors. For polynomial pieces, factors collapse into monomials. For non-polynomial pieces (exp, log, fractional-power), the canonicalized subexpression is an opaque factor atom. - -#### Step 1.1: Define the error type - -- [ ] **Add `CanonicalizationError` to `src/expr.rs`** - -Add the error enum right after the existing `AsymptoticAnalysisError` definition (around line 286): - -```rust -/// Error returned when exact canonicalization fails. -#[derive(Clone, Debug, PartialEq, Eq)] -pub enum CanonicalizationError { - /// Expression cannot be canonicalized (e.g., variable in both base and exponent). - Unsupported(String), -} - -impl fmt::Display for CanonicalizationError { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - match self { - Self::Unsupported(expr) => write!(f, "unsupported expression for canonicalization: {expr}"), - } - } -} - -impl std::error::Error for CanonicalizationError {} -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: add CanonicalizationError type -``` - -#### Step 1.2: Create canonical.rs with CanonicalFactor and CanonicalTerm - -- [ ] **Write the core data structures in `src/canonical.rs`** - -```rust -//! Exact symbolic canonicalization for `Expr`. -//! -//! Normalizes expressions into a canonical sum-of-terms form with signed -//! coefficients and deterministic ordering, without losing algebraic precision. - -use std::collections::BTreeMap; -use std::fmt; - -use crate::expr::{CanonicalizationError, Expr}; - -/// An opaque non-polynomial factor (exp, log, fractional-power base). -/// -/// Stored by its canonical string representation for deterministic ordering. -#[derive(Clone, Debug, PartialEq)] -struct OpaqueFactor { - /// The canonical string form (used for equality and ordering). - key: String, - /// The original `Expr` for reconstruction. - expr: Expr, -} - -impl Eq for OpaqueFactor {} - -impl PartialOrd for OpaqueFactor { - fn partial_cmp(&self, other: &Self) -> Option<std::cmp::Ordering> { - Some(self.cmp(other)) - } -} - -impl Ord for OpaqueFactor { - fn cmp(&self, other: &Self) -> std::cmp::Ordering { - self.key.cmp(&other.key) - } -} - -/// A single additive term: coefficient × product of canonical factors. -#[derive(Clone, Debug)] -struct CanonicalTerm { - /// Signed numeric coefficient. - coeff: f64, - /// Polynomial variable exponents (variable_name → exponent). - vars: BTreeMap<&'static str, f64>, - /// Non-polynomial opaque factors, sorted by key. - opaque: Vec<OpaqueFactor>, -} - -/// Try to merge a new opaque factor into an existing list using transcendental identities. -/// Returns `Some(updated_list)` if a merge happened, `None` if no identity applies. -fn try_merge_opaque(existing: &[OpaqueFactor], new: &OpaqueFactor) -> Option<Vec<OpaqueFactor>> { - for (i, existing_factor) in existing.iter().enumerate() { - // exp(a) * exp(b) -> exp(a + b) - if let (Expr::Exp(a), Expr::Exp(b)) = (&existing_factor.expr, &new.expr) { - let merged_arg = (**a).clone() + (**b).clone(); - let merged_expr = Expr::Exp(Box::new( - canonical_form(&merged_arg).unwrap_or(merged_arg), - )); - let mut result = existing.to_vec(); - result[i] = OpaqueFactor { - key: merged_expr.to_string(), - expr: merged_expr, - }; - return Some(result); - } - - // c^a * c^b -> c^(a+b) for matching constant base c - if let ( - Expr::Pow(base1, exp1), - Expr::Pow(base2, exp2), - ) = (&existing_factor.expr, &new.expr) - { - if let (Some(c1), Some(c2)) = (base1.constant_value(), base2.constant_value()) { - if (c1 - c2).abs() < 1e-15 { - let merged_exp = (**exp1).clone() + (**exp2).clone(); - let canon_exp = canonical_form(&merged_exp).unwrap_or(merged_exp); - let merged_expr = Expr::Pow(base1.clone(), Box::new(canon_exp)); - let mut result = existing.to_vec(); - result[i] = OpaqueFactor { - key: merged_expr.to_string(), - expr: merged_expr, - }; - return Some(result); - } - } - } - } - None -} - -/// A canonical sum of terms: the exact normal form of an expression. -#[derive(Clone, Debug)] -pub(crate) struct CanonicalSum { - terms: Vec<CanonicalTerm>, -} -``` - -- [ ] **Add the module declaration to `src/expr.rs` or `src/lib.rs`** - -In `src/lib.rs`, add after the existing `pub(crate) mod expr;` line: - -```rust -pub(crate) mod canonical; -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: add canonical.rs with core data structures -``` - -#### Step 1.3: Implement CanonicalTerm operations - -- [ ] **Write term operations in `src/canonical.rs`** - -Add to the `CanonicalTerm` impl block: - -```rust -impl CanonicalTerm { - fn constant(c: f64) -> Self { - Self { - coeff: c, - vars: BTreeMap::new(), - opaque: Vec::new(), - } - } - - fn variable(name: &'static str) -> Self { - let mut vars = BTreeMap::new(); - vars.insert(name, 1.0); - Self { - coeff: 1.0, - vars, - opaque: Vec::new(), - } - } - - fn opaque_factor(expr: Expr) -> Self { - let key = expr.to_string(); - Self { - coeff: 1.0, - vars: BTreeMap::new(), - opaque: vec![OpaqueFactor { key, expr }], - } - } - - /// Multiply two terms, applying transcendental identities: - /// - `exp(a) * exp(b) -> exp(a + b)` - /// - `c^a * c^b -> c^(a + b)` for matching constant base `c` - fn mul(&self, other: &CanonicalTerm) -> CanonicalTerm { - let coeff = self.coeff * other.coeff; - let mut vars = self.vars.clone(); - for (&v, &e) in &other.vars { - *vars.entry(v).or_insert(0.0) += e; - } - // Remove zero-exponent variables - vars.retain(|_, e| e.abs() > 1e-15); - - // Merge opaque factors with transcendental identities - let mut opaque = self.opaque.clone(); - for other_factor in &other.opaque { - if let Some(merged) = try_merge_opaque(&opaque, other_factor) { - opaque = merged; - } else { - opaque.push(other_factor.clone()); - } - } - opaque.sort(); - CanonicalTerm { - coeff, - vars, - opaque, - } - } - - /// The "signature" of this term for merging: same variables with same exponents - /// and same opaque factors, ignoring coefficient. - fn signature(&self) -> (Vec<(&'static str, i64)>, Vec<&str>) { - let vars: Vec<_> = self - .vars - .iter() - .map(|(&k, &v)| (k, (v * 1000.0).round() as i64)) - .collect(); - let opaque: Vec<_> = self.opaque.iter().map(|o| o.key.as_str()).collect(); - (vars, opaque) - } - - /// Deterministic sort key for ordering terms in a sum. - fn sort_key(&self) -> (Vec<(&'static str, i64)>, Vec<String>) { - let vars: Vec<_> = self - .vars - .iter() - .map(|(&k, &v)| (k, (v * 1000.0).round() as i64)) - .collect(); - let opaque: Vec<_> = self.opaque.iter().map(|o| o.key.clone()).collect(); - (vars, opaque) - } -} -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: add CanonicalTerm operations -``` - -#### Step 1.4: Implement CanonicalSum operations - -- [ ] **Write sum operations in `src/canonical.rs`** - -```rust -impl CanonicalSum { - fn zero() -> Self { - Self { terms: vec![] } - } - - fn from_term(term: CanonicalTerm) -> Self { - Self { - terms: vec![term], - } - } - - fn add(mut self, other: CanonicalSum) -> Self { - self.terms.extend(other.terms); - self - } - - fn mul(&self, other: &CanonicalSum) -> CanonicalSum { - let mut terms = Vec::new(); - for a in &self.terms { - for b in &other.terms { - terms.push(a.mul(b)); - } - } - CanonicalSum { terms } - } - - fn scale(mut self, c: f64) -> Self { - for term in &mut self.terms { - term.coeff *= c; - } - self - } - - /// Merge terms with the same signature and drop zero-coefficient terms. - /// Sort the result deterministically. - fn simplify(self) -> Self { - use std::collections::BTreeMap as Map; - - // Signature → (representative term with coeff=0, accumulated coefficient) - let mut groups: Map<(Vec<(&'static str, i64)>, Vec<String>), CanonicalTerm> = Map::new(); - - for term in self.terms { - let key = term.sort_key(); - groups - .entry(key) - .and_modify(|existing| existing.coeff += term.coeff) - .or_insert(term); - } - - let mut terms: Vec<_> = groups - .into_values() - .filter(|t| t.coeff.abs() > 1e-15) - .collect(); - - terms.sort_by(|a, b| a.sort_key().cmp(&b.sort_key())); - - CanonicalSum { terms } - } -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: add CanonicalSum operations with simplify -``` - -#### Step 1.5: Implement canonical_form() — the main entry point - -- [ ] **Write the `canonical_form()` function and `to_expr()` reconstruction** - -The conversion from `Expr` to `CanonicalSum`: - -```rust -/// Normalize an expression into its exact canonical sum-of-terms form. -/// -/// This performs exact symbolic simplification: -/// - Flattens nested Add/Mul -/// - Merges duplicate additive terms by summing coefficients -/// - Merges repeated multiplicative factors into powers -/// - Preserves signed coefficients (supports subtraction) -/// - Preserves transcendental identities: exp(a)*exp(b)=exp(a+b), etc. -/// - Produces deterministic ordering -/// -/// Does NOT drop terms or constant factors — use `big_o_normal_form()` for that. -pub fn canonical_form(expr: &Expr) -> Result<Expr, CanonicalizationError> { - let sum = expr_to_canonical(expr)?; - let simplified = sum.simplify(); - Ok(canonical_sum_to_expr(&simplified)) -} - -fn expr_to_canonical(expr: &Expr) -> Result<CanonicalSum, CanonicalizationError> { - match expr { - Expr::Const(c) => Ok(CanonicalSum::from_term(CanonicalTerm::constant(*c))), - Expr::Var(name) => Ok(CanonicalSum::from_term(CanonicalTerm::variable(name))), - Expr::Add(a, b) => { - let ca = expr_to_canonical(a)?; - let cb = expr_to_canonical(b)?; - Ok(ca.add(cb)) - } - Expr::Mul(a, b) => { - let ca = expr_to_canonical(a)?; - let cb = expr_to_canonical(b)?; - Ok(ca.mul(&cb)) - } - Expr::Pow(base, exp) => canonicalize_pow(base, exp), - Expr::Exp(arg) => { - // Treat exp(canonicalized_arg) as an opaque factor - let inner = canonical_form(arg)?; - Ok(CanonicalSum::from_term(CanonicalTerm::opaque_factor( - Expr::Exp(Box::new(inner)), - ))) - } - Expr::Log(arg) => { - let inner = canonical_form(arg)?; - Ok(CanonicalSum::from_term(CanonicalTerm::opaque_factor( - Expr::Log(Box::new(inner)), - ))) - } - Expr::Sqrt(arg) => { - // sqrt(x) = x^0.5 — canonicalize as power - canonicalize_pow(arg, &Expr::Const(0.5)) - } - } -} - -fn canonicalize_pow( - base: &Expr, - exp: &Expr, -) -> Result<CanonicalSum, CanonicalizationError> { - match (base, exp) { - // Constant base, constant exp → numeric constant - (_, _) if base.constant_value().is_some() && exp.constant_value().is_some() => { - let b = base.constant_value().unwrap(); - let e = exp.constant_value().unwrap(); - Ok(CanonicalSum::from_term(CanonicalTerm::constant(b.powf(e)))) - } - // Variable ^ constant integer exponent → polynomial - (Expr::Var(name), _) if exp.constant_value().is_some() => { - let e = exp.constant_value().unwrap(); - if e >= 0.0 && (e - e.round()).abs() < 1e-10 { - let n = e.round() as usize; - // Build x^n as repeated multiplication - if n == 0 { - return Ok(CanonicalSum::from_term(CanonicalTerm::constant(1.0))); - } - let mut vars = BTreeMap::new(); - vars.insert(*name, e); - Ok(CanonicalSum::from_term(CanonicalTerm { - coeff: 1.0, - vars, - opaque: Vec::new(), - })) - } else { - // Fractional or negative exponent → opaque factor - Ok(CanonicalSum::from_term(CanonicalTerm::opaque_factor( - Expr::Pow(Box::new(base.clone()), Box::new(exp.clone())), - ))) - } - } - // Polynomial base ^ constant integer exponent → expand - (_, _) if exp.constant_value().is_some() => { - let e = exp.constant_value().unwrap(); - if e >= 0.0 && (e - e.round()).abs() < 1e-10 { - let n = e.round() as usize; - let base_sum = expr_to_canonical(base)?; - if n == 0 { - return Ok(CanonicalSum::from_term(CanonicalTerm::constant(1.0))); - } - let mut result = base_sum.clone(); - for _ in 1..n { - result = result.mul(&base_sum); - } - Ok(result) - } else { - // Fractional exponent with non-variable base → opaque - let canon_base = canonical_form(base)?; - Ok(CanonicalSum::from_term(CanonicalTerm::opaque_factor( - Expr::Pow(Box::new(canon_base), Box::new(Expr::Const(e))), - ))) - } - } - // Constant base ^ variable exponent → opaque (exponential growth) - (_, _) if base.constant_value().is_some() => { - let canon_exp = canonical_form(exp)?; - Ok(CanonicalSum::from_term(CanonicalTerm::opaque_factor( - Expr::Pow(Box::new(base.clone()), Box::new(canon_exp)), - ))) - } - // Variable base ^ variable exponent → unsupported - _ => Err(CanonicalizationError::Unsupported(format!( - "{}^{}", - base, exp - ))), - } -} -``` - -The reconstruction from `CanonicalSum` back to `Expr`: - -```rust -fn canonical_sum_to_expr(sum: &CanonicalSum) -> Expr { - if sum.terms.is_empty() { - return Expr::Const(0.0); - } - - let term_exprs: Vec<Expr> = sum.terms.iter().map(canonical_term_to_expr).collect(); - - let mut result = term_exprs[0].clone(); - for term in &term_exprs[1..] { - result = result + term.clone(); - } - result -} - -fn canonical_term_to_expr(term: &CanonicalTerm) -> Expr { - let mut factors: Vec<Expr> = Vec::new(); - - // Add coefficient if not 1.0 (or -1.0, handled specially) - let (coeff_factor, sign) = if term.coeff < 0.0 { - (term.coeff.abs(), true) - } else { - (term.coeff, false) - }; - - let has_other_factors = !term.vars.is_empty() || !term.opaque.is_empty(); - - if (coeff_factor - 1.0).abs() > 1e-15 || !has_other_factors { - factors.push(Expr::Const(coeff_factor)); - } - - // Add variable powers - for (&var, &exp) in &term.vars { - if (exp - 1.0).abs() < 1e-15 { - factors.push(Expr::Var(var)); - } else { - factors.push(Expr::pow(Expr::Var(var), Expr::Const(exp))); - } - } - - // Add opaque factors - for opaque in &term.opaque { - factors.push(opaque.expr.clone()); - } - - let mut result = if factors.is_empty() { - Expr::Const(1.0) - } else { - let mut r = factors[0].clone(); - for f in &factors[1..] { - r = r * f.clone(); - } - r - }; - - if sign { - result = -result; - } - - result -} -``` - -**Important:** The `constant_value()` method is currently a private helper on `Expr` in `src/expr.rs`. It needs to be made `pub(crate)` so `canonical.rs` can call it. Edit `src/expr.rs` line ~180: - -```rust - // Change: fn constant_value(&self) -> Option<f64> { - // To: - pub(crate) fn constant_value(&self) -> Option<f64> { -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: implement canonical_form() with expr-to-canonical conversion -``` - -#### Step 1.6: Write tests for canonical_form() - -- [ ] **Create `src/unit_tests/canonical.rs` and wire it up** - -Add to `src/canonical.rs` at the bottom: - -```rust -#[cfg(test)] -#[path = "unit_tests/canonical.rs"] -mod tests; -``` - -Write the test file: - -```rust -use super::*; -use crate::expr::Expr; - -#[test] -fn test_canonical_identity() { - let e = Expr::Var("n"); - let c = canonical_form(&e).unwrap(); - assert_eq!(c.to_string(), "n"); -} - -#[test] -fn test_canonical_add_like_terms() { - // n + n → 2 * n - let e = Expr::Var("n") + Expr::Var("n"); - let c = canonical_form(&e).unwrap(); - assert_eq!(c.to_string(), "2 * n"); -} - -#[test] -fn test_canonical_subtract_to_zero() { - // n - n → 0 - let e = Expr::Var("n") - Expr::Var("n"); - let c = canonical_form(&e).unwrap(); - assert_eq!(c.to_string(), "0"); -} - -#[test] -fn test_canonical_mixed_addition() { - // n + n - m + 2*m → 2*n + m - let e = Expr::Var("n") + Expr::Var("n") - Expr::Var("m") + Expr::Const(2.0) * Expr::Var("m"); - let c = canonical_form(&e).unwrap(); - assert_eq!(c.to_string(), "m + 2 * n"); -} - -#[test] -fn test_canonical_power_merge() { - // n * n^(1/2) → n^1.5 - let e = Expr::Var("n") * Expr::pow(Expr::Var("n"), Expr::Const(0.5)); - let c = canonical_form(&e).unwrap(); - // Note: n^0.5 becomes an opaque factor, so this merges to n * n^0.5 in product form - // The exact representation depends on how opaque factors merge. - // This test should be adjusted based on actual behavior. - let size = crate::types::ProblemSize::new(vec![("n", 4)]); - assert!((c.eval(&size) - 4.0_f64.powf(1.5)).abs() < 1e-10); -} - -#[test] -fn test_canonical_exp_product_identity() { - // exp(n) * exp(m) -> exp(n + m) (transcendental identity) - let e = Expr::Exp(Box::new(Expr::Var("n"))) * Expr::Exp(Box::new(Expr::Var("m"))); - let c = canonical_form(&e).unwrap(); - let s = c.to_string(); - // Should merge into a single exp() factor - assert!(s.contains("exp"), "expected exp in result, got: {s}"); - // Verify numerical equivalence - let size = crate::types::ProblemSize::new(vec![("n", 2), ("m", 3)]); - assert!((c.eval(&size) - (2.0_f64.exp() * 3.0_f64.exp())).abs() < 1e-6); -} - -#[test] -fn test_canonical_constant_base_exp_identity() { - // 2^n * 2^m -> 2^(n + m) - let e = Expr::pow(Expr::Const(2.0), Expr::Var("n")) - * Expr::pow(Expr::Const(2.0), Expr::Var("m")); - let c = canonical_form(&e).unwrap(); - let size = crate::types::ProblemSize::new(vec![("n", 3), ("m", 4)]); - assert!((c.eval(&size) - 2.0_f64.powf(7.0)).abs() < 1e-6); -} - -#[test] -fn test_canonical_polynomial_expansion() { - // (n + m)^2 = n^2 + 2*n*m + m^2 - let e = Expr::pow(Expr::Var("n") + Expr::Var("m"), Expr::Const(2.0)); - let c = canonical_form(&e).unwrap(); - let size = crate::types::ProblemSize::new(vec![("n", 3), ("m", 4)]); - assert_eq!(c.eval(&size), 49.0); // (3+4)^2 = 49 -} - -#[test] -fn test_canonical_signed_polynomial() { - // n^3 - n^2 + 2*n + 4*n*m — should remain exact - let e = Expr::pow(Expr::Var("n"), Expr::Const(3.0)) - - Expr::pow(Expr::Var("n"), Expr::Const(2.0)) - + Expr::Const(2.0) * Expr::Var("n") - + Expr::Const(4.0) * Expr::Var("n") * Expr::Var("m"); - let c = canonical_form(&e).unwrap(); - let size = crate::types::ProblemSize::new(vec![("n", 3), ("m", 2)]); - // 27 - 9 + 6 + 24 = 48 - assert_eq!(c.eval(&size), 48.0); -} - -#[test] -fn test_canonical_division_becomes_negative_exponent() { - // n / m should canonicalize; the division is represented as m^(-1) - // which becomes an opaque factor (negative exponent) - let e = Expr::Var("n") / Expr::Var("m"); - let c = canonical_form(&e).unwrap(); - let size = crate::types::ProblemSize::new(vec![("n", 6), ("m", 3)]); - assert!((c.eval(&size) - 2.0).abs() < 1e-10); -} - -#[test] -fn test_canonical_deterministic_order() { - // m + n and n + m should produce the same canonical form - let a = canonical_form(&(Expr::Var("m") + Expr::Var("n"))).unwrap(); - let b = canonical_form(&(Expr::Var("n") + Expr::Var("m"))).unwrap(); - assert_eq!(a.to_string(), b.to_string()); -} - -#[test] -fn test_canonical_constant_folding() { - // 2 + 3 → 5 - let e = Expr::Const(2.0) + Expr::Const(3.0); - let c = canonical_form(&e).unwrap(); - assert_eq!(c.to_string(), "5"); -} - -#[test] -fn test_canonical_sqrt_as_power() { - // sqrt(n) should canonicalize the same as n^0.5 - let a = canonical_form(&Expr::Sqrt(Box::new(Expr::Var("n")))).unwrap(); - let b = canonical_form(&Expr::pow(Expr::Var("n"), Expr::Const(0.5))).unwrap(); - assert_eq!(a.to_string(), b.to_string()); -} -``` - -- [ ] **Run tests to see which pass** - -Run: `cargo test -p problemreductions canonical -- --nocapture` - -- [ ] **Fix any test failures by adjusting expectations or the implementation** - -The exact output format of `canonical_form()` depends on the reconstruction logic. Adjust test expectations to match actual output. The critical invariant is: `canonical_form(a).eval(vars) == a.eval(vars)` for all inputs. - -- [ ] **Commit** - -``` -feat: add canonical_form tests -``` - -#### Step 1.7: Re-export canonical_form from lib.rs - -- [ ] **Update `src/lib.rs`** - -Change the re-export line to include the new items: - -```rust -pub use expr::{asymptotic_normal_form, AsymptoticAnalysisError, CanonicalizationError, Expr}; -pub use canonical::canonical_form; -``` - -But `canonical` is `pub(crate)`, so we need to re-export the function directly. Add to `src/lib.rs`: - -```rust -pub use canonical::canonical_form; -``` - -And make `canonical_form` pub in `src/canonical.rs` (it should already be `pub`). - -- [ ] **Run full test suite** - -Run: `make check` - -- [ ] **Commit** - -``` -feat: export canonical_form from crate root -``` - ---- - -## Chunk 2: Big-O Projection Layer - -### Task 2: Implement big_o_normal_form() - -**Files:** -- Create: `src/big_o.rs` -- Create: `src/unit_tests/big_o.rs` -- Modify: `src/lib.rs` (add module and re-export) -- Modify: `src/expr.rs` (convert `asymptotic_normal_form()` to wrapper) - -The Big-O projection takes the output of `canonical_form()` and: -1. Drops constant terms -2. Drops constant multiplicative factors -3. Drops terms dominated by other terms (using monomial comparison) -4. Rejects invalid negative-only results - -#### Step 2.1: Create big_o.rs with the projection function - -- [ ] **Write `src/big_o.rs`** - -```rust -//! Big-O asymptotic projection for canonical expressions. -//! -//! Takes the output of `canonical_form()` and projects it into an -//! asymptotic growth class by dropping dominated terms and constant factors. - -use crate::canonical::canonical_form; -use crate::expr::{AsymptoticAnalysisError, CanonicalizationError, Expr}; - -/// Compute the Big-O normal form of an expression. -/// -/// This is a two-phase pipeline: -/// 1. `canonical_form()` — exact symbolic simplification -/// 2. Asymptotic projection — drop dominated terms and constant factors -/// -/// Returns an expression representing the asymptotic growth class. -pub fn big_o_normal_form(expr: &Expr) -> Result<Expr, AsymptoticAnalysisError> { - let canonical = canonical_form(expr).map_err(|e| match e { - CanonicalizationError::Unsupported(s) => AsymptoticAnalysisError::Unsupported(s), - })?; - - project_big_o(&canonical) -} - -/// Project a canonicalized expression into its Big-O growth class. -fn project_big_o(expr: &Expr) -> Result<Expr, AsymptoticAnalysisError> { - // Decompose into additive terms - let mut terms = Vec::new(); - collect_additive_terms(expr, &mut terms); - - // Project each term: drop constant multiplicative factors - let mut projected: Vec<Expr> = Vec::new(); - for term in &terms { - if let Some(projected_term) = project_term(term) { - projected.push(projected_term); - } - // Pure constants are dropped (asymptotically irrelevant) - } - - // Remove dominated terms - let survivors = remove_dominated_terms(projected); - - if survivors.is_empty() { - // All terms were constants → O(1) - return Ok(Expr::Const(1.0)); - } - - // Deduplicate - let mut seen = std::collections::BTreeSet::new(); - let mut deduped = Vec::new(); - for term in survivors { - let key = term.to_string(); - if seen.insert(key) { - deduped.push(term); - } - } - - // Rebuild sum - let mut result = deduped[0].clone(); - for term in &deduped[1..] { - result = result + term.clone(); - } - - Ok(result) -} - -fn collect_additive_terms(expr: &Expr, out: &mut Vec<Expr>) { - match expr { - Expr::Add(a, b) => { - collect_additive_terms(a, out); - collect_additive_terms(b, out); - } - other => out.push(other.clone()), - } -} - -/// Project a single multiplicative term: strip constant factors. -/// Returns None if the term is a pure constant. -fn project_term(term: &Expr) -> Option<Expr> { - if term.constant_value().is_some() { - return None; // Pure constant → dropped - } - - // Collect multiplicative factors - let mut factors = Vec::new(); - collect_multiplicative_factors(term, &mut factors); - - // Remove constant factors, keep symbolic ones - let symbolic: Vec<&Expr> = factors - .iter() - .filter(|f| f.constant_value().is_none()) - .collect(); - - if symbolic.is_empty() { - return None; - } - - // Check for negative coefficient (Const(-1) * ...) — take absolute value - let mut result = symbolic[0].clone(); - for f in &symbolic[1..] { - result = result * (*f).clone(); - } - Some(result) -} - -fn collect_multiplicative_factors(expr: &Expr, out: &mut Vec<Expr>) { - match expr { - Expr::Mul(a, b) => { - collect_multiplicative_factors(a, out); - collect_multiplicative_factors(b, out); - } - other => out.push(other.clone()), - } -} - -/// Remove terms dominated by other terms using monomial comparison. -/// -/// A term `t` is dominated if there exists another term `s` such that -/// `t` grows no faster than `s` asymptotically. -fn remove_dominated_terms(terms: Vec<Expr>) -> Vec<Expr> { - if terms.len() <= 1 { - return terms; - } - - let mut survivors = Vec::new(); - for (i, term) in terms.iter().enumerate() { - let is_dominated = terms.iter().enumerate().any(|(j, other)| { - i != j && term_dominated_by(term, other) - }); - if !is_dominated { - survivors.push(term.clone()); - } - } - survivors -} - -/// Check if `small` is asymptotically dominated by `big`. -/// -/// Conservative: only returns true when dominance is provable -/// via monomial exponent comparison. -fn term_dominated_by(small: &Expr, big: &Expr) -> bool { - // Extract monomial exponents for comparison - let small_exps = extract_var_exponents(small); - let big_exps = extract_var_exponents(big); - - // Both must be pure polynomial monomials for comparison - let (Some(se), Some(be)) = (small_exps, big_exps) else { - return false; // Can't compare non-polynomial terms - }; - - // small ≤ big if: for every variable in small, big has ≥ exponent - // AND big has at least one strictly greater exponent or has a variable small doesn't - let mut all_leq = true; - let mut any_strictly_less = false; - - for (var, small_exp) in &se { - let big_exp = be.get(var).copied().unwrap_or(0.0); - if *small_exp > big_exp + 1e-15 { - all_leq = false; - break; - } - if *small_exp < big_exp - 1e-15 { - any_strictly_less = true; - } - } - - // Also check variables in big not in small (those have implicit exponent 0 in small) - if all_leq { - for (var, big_exp) in &be { - if !se.contains_key(var) && *big_exp > 1e-15 { - any_strictly_less = true; - } - } - } - - // Dominated if all exponents ≤ AND at least one is strictly less. - // Equal terms are NOT dominated — they get deduped in a separate step. - all_leq && any_strictly_less -} - -/// Extract variable → exponent mapping from a monomial expression. -/// Returns None for non-polynomial terms (exp, log, etc.). -fn extract_var_exponents(expr: &Expr) -> Option<std::collections::BTreeMap<&'static str, f64>> { - use std::collections::BTreeMap; - let mut exps = BTreeMap::new(); - extract_var_exponents_inner(expr, &mut exps)?; - Some(exps) -} - -fn extract_var_exponents_inner( - expr: &Expr, - exps: &mut std::collections::BTreeMap<&'static str, f64>, -) -> Option<()> { - match expr { - Expr::Var(name) => { - *exps.entry(name).or_insert(0.0) += 1.0; - Some(()) - } - Expr::Pow(base, exp) => { - if let (Expr::Var(name), Some(e)) = (base.as_ref(), exp.constant_value()) { - *exps.entry(name).or_insert(0.0) += e; - Some(()) - } else { - None // Non-simple power - } - } - Expr::Mul(a, b) => { - extract_var_exponents_inner(a, exps)?; - extract_var_exponents_inner(b, exps) - } - Expr::Const(_) => Some(()), // Constants don't affect exponents - _ => None, // exp, log, sqrt → not a polynomial monomial - } -} -``` - -- [ ] **Add module declaration to `src/lib.rs`** - -```rust -pub(crate) mod big_o; -``` - -- [ ] **Verify it compiles** - -Run: `cargo check` - -- [ ] **Commit** - -``` -feat: implement big_o_normal_form() projection layer -``` - -#### Step 2.2: Write tests for big_o_normal_form() - -- [ ] **Create `src/unit_tests/big_o.rs`** - -Add to `src/big_o.rs`: - -```rust -#[cfg(test)] -#[path = "unit_tests/big_o.rs"] -mod tests; -``` - -Write the tests: - -```rust -use crate::big_o::big_o_normal_form; -use crate::expr::Expr; - -#[test] -fn test_big_o_drops_constant_factors() { - let e = Expr::parse("3 * n^2"); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), "n^2"); -} - -#[test] -fn test_big_o_drops_additive_constants() { - let e = Expr::parse("n + 1"); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), "n"); -} - -#[test] -fn test_big_o_duplicate_terms_collapse() { - // n + n → (canonical: 2*n) → big-o: n - let e = Expr::parse("n + n"); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), "n"); -} - -#[test] -fn test_big_o_lower_order_drops() { - // n^3 + n^2 → n^3 - let e = Expr::parse("n^3 + n^2"); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), "n^3"); -} - -#[test] -fn test_big_o_signed_polynomial() { - // n^3 - n^2 + 2*n + 4*n*m → n^3 + n*m - let e = Expr::parse("n^3 - n^2 + 2 * n + 4 * n * m"); - let result = big_o_normal_form(&e).unwrap(); - // n^3 dominates n^2 and n; n*m is incomparable with n^3 - // Expected: n^3 + n*m (or m*n depending on ordering) - let s = result.to_string(); - assert!(s.contains("n^3"), "missing n^3 term, got: {s}"); - assert!(s.contains("m") && s.contains("n"), "missing n*m term, got: {s}"); -} - -#[test] -fn test_big_o_commutative_sum() { - let a = big_o_normal_form(&Expr::parse("n + m")).unwrap(); - let b = big_o_normal_form(&Expr::parse("m + n")).unwrap(); - assert_eq!(a, b); -} - -#[test] -fn test_big_o_commutative_product() { - let a = big_o_normal_form(&Expr::parse("n * m")).unwrap(); - let b = big_o_normal_form(&Expr::parse("m * n")).unwrap(); - assert_eq!(a, b); -} - -#[test] -fn test_big_o_incomparable_terms_survive() { - // n^2 + n*m — incomparable, both survive - let e = Expr::parse("n^2 + n * m"); - let result = big_o_normal_form(&e).unwrap(); - let s = result.to_string(); - assert!(s.contains("n"), "got: {s}"); - assert!(s.contains("m"), "got: {s}"); -} - -#[test] -fn test_big_o_composed_overhead_duplicate() { - // (n + m) + (m + n) should reduce to m + n - let e = Expr::parse("n + m + m + n"); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), big_o_normal_form(&Expr::parse("m + n")).unwrap().to_string()); -} - -#[test] -fn test_big_o_exp_with_polynomial() { - // exp(n) + n^10 — incomparable, both survive - let e = Expr::Exp(Box::new(Expr::Var("n"))) + Expr::pow(Expr::Var("n"), Expr::Const(10.0)); - let result = big_o_normal_form(&e).unwrap(); - let s = result.to_string(); - assert!(s.contains("exp"), "expected exp term to survive, got: {s}"); - assert!(s.contains("n"), "expected polynomial term to survive, got: {s}"); -} - -#[test] -fn test_big_o_pure_constant_returns_one() { - let e = Expr::Const(42.0); - let result = big_o_normal_form(&e).unwrap(); - assert_eq!(result.to_string(), "1"); -} -``` - -- [ ] **Run tests** - -Run: `cargo test -p problemreductions big_o -- --nocapture` - -- [ ] **Fix failures by adjusting expectations or implementation** - -- [ ] **Commit** - -``` -feat: add big_o_normal_form tests -``` - -#### Step 2.3: Wire asymptotic_normal_form as wrapper and re-export - -- [ ] **Update `src/expr.rs`** - -Replace the existing `asymptotic_normal_form()` function body (keep the function and its docs): - -```rust -/// Return a normalized `Expr` representing the asymptotic behavior of `expr`. -/// -/// This is now a compatibility wrapper for `big_o_normal_form()`. -pub fn asymptotic_normal_form(expr: &Expr) -> Result<Expr, AsymptoticAnalysisError> { - crate::big_o::big_o_normal_form(expr) -} -``` - -**Important:** Keep the old implementation available temporarily. A safe approach: -1. Rename the old function to `asymptotic_normal_form_legacy()` -2. Make the new `asymptotic_normal_form()` call `big_o_normal_form()` -3. Run all existing tests — if any fail, investigate whether the new behavior is correct or needs adjustment -4. Once all tests pass, delete the legacy function and all its internal helpers - -- [ ] **Update `src/lib.rs` re-exports** - -```rust -pub use big_o::big_o_normal_form; -pub use canonical::canonical_form; -pub use expr::{asymptotic_normal_form, AsymptoticAnalysisError, CanonicalizationError, Expr}; -``` - -- [ ] **Run full test suite** - -Run: `make check` - -Expect some existing `asymptotic_normal_form` tests to fail if behavior changed (e.g., `test_asymptotic_normal_form_rejects_negative_forms` — the new engine handles negatives). Adjust test expectations to match the redesign: - -- `n - m` may now succeed (canonical form) but Big-O projection rejects if no positive dominant term -- Exponential identities should still work -- Duplicate term collapse should now work correctly - -- [ ] **Commit** - -``` -feat: wire asymptotic_normal_form as wrapper for big_o_normal_form -``` - -#### Step 2.4: Clean up legacy code - -- [ ] **Remove old normalization helpers from `src/expr.rs`** - -Once all tests pass with the new pipeline, delete: -- `asymptotic_normal_form_legacy()` (if renamed) -- `normalize_pow()` -- `collect_sum_term()` -- `collect_product_factor()` -- `build_sum()` -- `build_product()` -- `build_pow()` -- `build_exp()` -- `build_exp_base()` -- `build_log()` -- `into_base_and_exponent()` -- `combine_add_chain()` -- `combine_mul_chain()` -- `format_float()` - -Keep only: `asymptotic_normal_form()` (the wrapper), the `Expr` type, parsing, `Display`, operator impls, and `CanonicalizationError`. - -- [ ] **Run full test suite** - -Run: `make check` - -- [ ] **Commit** - -``` -refactor: remove legacy asymptotic normalization helpers -``` - ---- - -## Chunk 3: Display Improvements and Caller Integration - -### Task 3: Fix sqrt display and update CLI callers - -**Files:** -- Modify: `src/expr.rs` (Display impl for Pow with 0.5 exponent) -- Modify: `problemreductions-cli/src/commands/graph.rs` (switch to `big_o_normal_form`) -- Modify: `src/rules/analysis.rs` (use `canonical_form` where appropriate) -- Modify: relevant test files - -#### Step 3.1: Improve Display for Pow(_, 0.5) - -- [ ] **Update `Display for Expr` in `src/expr.rs`** - -In the `Pow` arm of the `Display` impl, add a special case: - -```rust -Expr::Pow(base, exp) => { - // Special case: x^0.5 → sqrt(x) - if let Expr::Const(e) = exp.as_ref() { - if (*e - 0.5).abs() < 1e-15 { - return write!(f, "sqrt({base})"); - } - } - // ... existing parenthesization logic ... -} -``` - -- [ ] **Add test** - -In `src/unit_tests/expr.rs`: - -```rust -#[test] -fn test_expr_display_pow_half_as_sqrt() { - let e = Expr::pow(Expr::Var("n"), Expr::Const(0.5)); - assert_eq!(format!("{e}"), "sqrt(n)"); -} - -#[test] -fn test_expr_display_pow_half_complex_base() { - let e = Expr::pow(Expr::Var("n") * Expr::Var("m"), Expr::Const(0.5)); - assert_eq!(format!("{e}"), "sqrt(n * m)"); -} - -#[test] -fn test_expr_display_pow_half_in_exponent() { - // 2^(n^0.5) should display as 2^sqrt(n), NOT 2^n^0.5 - let e = Expr::pow(Expr::Const(2.0), Expr::pow(Expr::Var("n"), Expr::Const(0.5))); - let s = format!("{e}"); - assert!(s.contains("sqrt"), "expected sqrt notation, got: {s}"); - assert!(!s.contains("0.5"), "should not contain raw 0.5, got: {s}"); -} -``` - -- [ ] **Run tests** - -Run: `make test` - -- [ ] **Commit** - -``` -fix: display Pow(x, 0.5) as sqrt(x) to avoid ambiguous notation -``` - -#### Step 3.2: Switch CLI to use big_o_normal_form - -- [ ] **Update `problemreductions-cli/src/commands/graph.rs`** - -Change the import: - -```rust -use problemreductions::{big_o_normal_form, Expr}; -``` - -Update `big_o_of()`: - -```rust -fn big_o_of(expr: &Expr) -> String { - match big_o_normal_form(expr) { - Ok(norm) => format!("O({})", norm), - Err(_) => format!("O({})", expr), - } -} -``` - -Remove the now-unused `asymptotic_normal_form` import. - -- [ ] **Run CLI tests** - -Run: `cargo test -p problemreductions-cli` - -- [ ] **Commit** - -``` -refactor: switch CLI to big_o_normal_form -``` - -#### Step 3.3: Update analysis.rs to use canonical_form for comparisons - -- [ ] **Read `src/rules/analysis.rs` compare_overhead function** - -The existing `compare_overhead()` at line ~273 calls `asymptotic_normal_form()` then `normalize_polynomial()`. With the redesign, it should: -1. Call `canonical_form()` for exact normalization -2. Then use its own polynomial extraction for dominance comparison - -Update the import and the function body: - -```rust -use crate::canonical::canonical_form; -``` - -In `compare_overhead()`, replace `asymptotic_normal_form(prim_expr)` calls with `canonical_form(prim_expr)` (converting the error type). The polynomial extraction (`normalize_polynomial`) still operates on the canonical `Expr`, so it should work without changes. - -- [ ] **Run analysis tests** - -Run: `cargo test -p problemreductions analysis -- --nocapture` - -- [ ] **Commit** - -``` -refactor: use canonical_form in overhead comparison -``` - ---- - -## Chunk 4: pred-sym Binary - -### Task 4: Add pred-sym CLI tool - -**Files:** -- Modify: `problemreductions-cli/Cargo.toml` (add binary) -- Create: `problemreductions-cli/src/bin/pred_sym.rs` (main binary) -- Add CLI integration tests - -#### Step 4.1: Add binary to Cargo.toml - -- [ ] **Update `problemreductions-cli/Cargo.toml`** - -Add after the existing `[[bin]]` section: - -```toml -[[bin]] -name = "pred-sym" -path = "src/bin/pred_sym.rs" -``` - -- [ ] **Create a minimal `src/bin/pred_sym.rs`** - -```rust -use clap::{Parser, Subcommand}; -use problemreductions::{Expr, canonical_form, big_o_normal_form}; -use problemreductions::types::ProblemSize; - -#[derive(Parser)] -#[command(name = "pred-sym", about = "Symbolic expression engine for problemreductions")] -struct Cli { - #[command(subcommand)] - command: Commands, -} - -#[derive(Subcommand)] -enum Commands { - /// Parse and echo an expression - Parse { - /// Expression string - expr: String, - }, - /// Compute exact canonical form - Canon { - /// Expression string - expr: String, - }, - /// Compute Big-O normal form - BigO { - /// Expression string - #[arg(name = "expr")] - expr: String, - }, - /// Compare two expressions - Compare { - /// First expression - a: String, - /// Second expression - b: String, - }, - /// Evaluate an expression with variable bindings - Eval { - /// Expression string - expr: String, - /// Variable bindings (e.g., n=10,m=20) - #[arg(long)] - vars: String, - }, -} - -fn main() { - let cli = Cli::parse(); - - match cli.command { - Commands::Parse { expr } => { - let parsed = Expr::parse(&expr); - println!("{parsed}"); - } - Commands::Canon { expr } => { - let parsed = Expr::parse(&expr); - match canonical_form(&parsed) { - Ok(result) => println!("{result}"), - Err(e) => { - eprintln!("Error: {e}"); - std::process::exit(1); - } - } - } - Commands::BigO { expr } => { - let parsed = Expr::parse(&expr); - match big_o_normal_form(&parsed) { - Ok(result) => println!("O({result})"), - Err(e) => { - eprintln!("Error: {e}"); - std::process::exit(1); - } - } - } - Commands::Compare { a, b } => { - let expr_a = Expr::parse(&a); - let expr_b = Expr::parse(&b); - let canon_a = canonical_form(&expr_a); - let canon_b = canonical_form(&expr_b); - let big_o_a = big_o_normal_form(&expr_a); - let big_o_b = big_o_normal_form(&expr_b); - - let exact_equal = match (&canon_a, &canon_b) { - (Ok(a), Ok(b)) => Some(a == b), - _ => None, - }; - let big_o_equal = match (&big_o_a, &big_o_b) { - (Ok(a), Ok(b)) => Some(a == b), - _ => None, - }; - - println!("Expression A: {a}"); - println!("Expression B: {b}"); - if let (Ok(ca), Ok(cb)) = (&canon_a, &canon_b) { - println!("Canonical A: {ca}"); - println!("Canonical B: {cb}"); - println!("Exact equal: {}", ca == cb); - } - if let (Ok(ba), Ok(bb)) = (&big_o_a, &big_o_b) { - println!("Big-O A: O({ba})"); - println!("Big-O B: O({bb})"); - println!("Big-O equal: {}", ba == bb); - } - } - Commands::Eval { expr, vars } => { - let parsed = Expr::parse(&expr); - let bindings: Vec<(&str, usize)> = vars - .split(',') - .filter_map(|pair| { - let mut parts = pair.splitn(2, '='); - let name = parts.next()?.trim(); - let value: usize = parts.next()?.trim().parse().ok()?; - // Leak the name for &'static str compatibility - let leaked: &'static str = Box::leak(name.to_string().into_boxed_str()); - Some((leaked, value)) - }) - .collect(); - let size = ProblemSize::new(bindings); - let result = parsed.eval(&size); - - // Format as integer if it's a whole number - if (result - result.round()).abs() < 1e-10 { - println!("{}", result.round() as i64); - } else { - println!("{result}"); - } - } - } -} -``` - -- [ ] **Verify it builds** - -Run: `cargo build -p problemreductions-cli` - -- [ ] **Commit** - -``` -feat: add pred-sym binary for symbolic engine inspection -``` - -#### Step 4.2: Add pred-sym integration tests - -- [ ] **Add tests to an existing CLI test file or create a new one** - -Add to `problemreductions-cli/tests/` (check the existing test structure first — the CLI uses `assert_cmd` or similar): - -```rust -#[test] -fn test_pred_sym_parse() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["parse", "n + m"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert_eq!(stdout.trim(), "n + m"); -} - -#[test] -fn test_pred_sym_canon_merge_terms() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["canon", "n + n"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert_eq!(stdout.trim(), "2 * n"); -} - -#[test] -fn test_pred_sym_big_o() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["big-o", "3 * n^2 + n"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert_eq!(stdout.trim(), "O(n^2)"); -} - -#[test] -fn test_pred_sym_eval() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["eval", "n + m", "--vars", "n=3,m=4"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert_eq!(stdout.trim(), "7"); -} - -#[test] -fn test_pred_sym_big_o_signed_polynomial() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["big-o", "n^3 - n^2 + 2*n + 4*n*m"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - // n^3 dominates n^2 and n; n*m is incomparable - assert!(stdout.contains("n^3"), "got: {}", stdout.trim()); -} - -#[test] -fn test_pred_sym_big_o_sqrt_display() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["big-o", "2^(n^(1/2))"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert!(stdout.contains("sqrt"), "expected sqrt notation, got: {}", stdout.trim()); -} - -#[test] -fn test_pred_sym_compare() { - let output = Command::cargo_bin("pred-sym") - .unwrap() - .args(["compare", "n + n", "2 * n"]) - .output() - .unwrap(); - assert!(output.status.success()); - let stdout = String::from_utf8(output.stdout).unwrap(); - assert!(stdout.contains("true"), "expected exact equality, got: {}", stdout.trim()); -} - -``` - -- [ ] **Run tests** - -Run: `cargo test -p problemreductions-cli pred_sym` - -- [ ] **Commit** - -``` -feat: add pred-sym integration tests -``` - ---- - -## Chunk 5: Test Rebaseline and Final Verification - -### Task 5: Update existing tests and full verification - -**Files:** -- Modify: `src/unit_tests/expr.rs` (update asymptotic_normal_form test expectations) -- Modify: `src/unit_tests/rules/analysis.rs` (adjust if comparison logic changed) - -#### Step 5.1: Rebaseline asymptotic_normal_form tests - -- [ ] **Run all tests and collect failures** - -Run: `make test 2>&1 | grep "FAILED\|failures"` - -- [ ] **For each failing test, decide:** - -1. Is the old expectation still correct under the new design? → Keep -2. Does the new design produce a different but correct result? → Update expectation -3. Does the test encode behavior that the redesign explicitly changes? → Rewrite - -Key expected changes: -- `test_asymptotic_normal_form_rejects_negative_forms`: `n - m` should now succeed through `canonical_form()` + `big_o_normal_form()` (both terms are O(n) and O(m), incomparable, both survive). Change the test from expecting `Err` to expecting `Ok` with both terms. -- Duplicate term tests should now pass where they previously produced `O(x + x)`. - -- [ ] **Update each failing test** - -- [ ] **Run full suite** - -Run: `make check` - -- [ ] **Commit** - -``` -test: rebaseline asymptotic_normal_form tests for two-phase pipeline -``` - -#### Step 5.2: Run coverage and verify >95% - -- [ ] **Check coverage** - -Run: `make coverage` - -If new code in `canonical.rs` or `big_o.rs` has uncovered paths, add targeted tests. - -- [ ] **Commit any coverage improvements** - -``` -test: improve coverage for canonical and big_o modules -``` - -#### Step 5.3: Final full check - -- [ ] **Run the complete check suite** - -Run: `make check` - -All of: fmt, clippy, test must pass. - -- [ ] **Run CLI demo** - -Run: `make cli-demo` - -Verify Big-O output looks correct in the CLI. - -- [ ] **Test pred-sym manually** - -```bash -cargo run -p problemreductions-cli --bin pred-sym -- canon 'n + n' -cargo run -p problemreductions-cli --bin pred-sym -- big-o 'n^3 - n^2 + 2*n + 4*n*m' -cargo run -p problemreductions-cli --bin pred-sym -- big-o '2^(n^0.5)' -cargo run -p problemreductions-cli --bin pred-sym -- eval 'n + m' --vars n=3,m=4 -cargo run -p problemreductions-cli --bin pred-sym -- compare 'n + n' '2 * n' -``` - -Expected output: -``` -2 * n -O(n^3 + m * n) (or similar with n*m) -O(2^sqrt(n)) -7 -Exact equal: true, Big-O equal: true -``` - -- [ ] **Commit and push** - -``` -feat: complete asymptotic normal form redesign - -Two-phase normalization pipeline: -- canonical_form(): exact symbolic simplification -- big_o_normal_form(): asymptotic projection -- pred-sym binary for symbolic engine inspection -- sqrt display for Pow(x, 0.5) -``` diff --git a/scripts/make_helpers.sh b/scripts/make_helpers.sh index ef81e6a9b..5ab23b0c2 100644 --- a/scripts/make_helpers.sh +++ b/scripts/make_helpers.sh @@ -20,6 +20,23 @@ skill_prompt() { fi } +# Build a prompt and optionally append structured context for Codex. +# skill_prompt_with_context <skill> <slash-cmd> <codex-desc> <context-label> <context-json> +skill_prompt_with_context() { + skill=$1 + slash_cmd=$2 + codex_desc=${3-} + context_label=${4-} + context_json=${5-} + + base_prompt=$(skill_prompt "$skill" "$slash_cmd" "$codex_desc") + if [ "${RUNNER:-codex}" = "claude" ] || [ -z "$context_json" ]; then + echo "$base_prompt" + else + printf '%s\n\n## %s\n%s\n' "$base_prompt" "$context_label" "$context_json" + fi +} + # Run an agent with the configured runner (claude or codex). # run_agent <log-file> <prompt> run_agent() { @@ -43,29 +60,148 @@ run_agent() { # --- Project board --- -project_items_json() { - gh project item-list 8 --owner CodingThrust --format json --limit 500 -} - # Detect the next eligible item and preserve retryable state in a queue. -# poll_project_items <mode> <state-file> [repo] +# poll_project_items <mode> <state-file> [repo] [number] [format] poll_project_items() { mode=$1 state_file=$2 repo=${3-} - board_json=$(project_items_json) || return $? + number=${4-} + fmt=${5-text} + set -- scripts/pipeline_board.py next "$mode" "$state_file" --format "$fmt" if [ -n "$repo" ]; then - printf '%s\n' "$board_json" | python3 scripts/project_board_poll.py poll "$mode" "$state_file" --repo "$repo" - else - printf '%s\n' "$board_json" | python3 scripts/project_board_poll.py poll "$mode" "$state_file" + set -- "$@" --repo "$repo" + fi + if [ -n "$number" ]; then + set -- "$@" --number "$number" fi + python3 "$@" } ack_polled_item() { state_file=$1 item_id=$2 - python3 scripts/project_board_poll.py ack "$state_file" "$item_id" + python3 scripts/pipeline_board.py ack "$state_file" "$item_id" +} + +board_next_json() { + mode=$1 + repo=${2-} + number=${3-} + state_file=${4-} + + if [ -z "$state_file" ]; then + state_file="/tmp/problemreductions-${mode}-state.json" + fi + + poll_project_items "$mode" "$state_file" "$repo" "$number" json +} + +claim_project_items() { + mode=$1 + state_file=$2 + repo=${3-} + number=${4-} + fmt=${5-json} + + set -- scripts/pipeline_board.py claim-next "$mode" "$state_file" --format "$fmt" + if [ -n "$repo" ]; then + set -- "$@" --repo "$repo" + fi + if [ -n "$number" ]; then + set -- "$@" --number "$number" + fi + python3 "$@" +} + +board_claim_json() { + mode=$1 + repo=${2-} + number=${3-} + state_file=${4-} + + if [ -z "$state_file" ]; then + state_file="/tmp/problemreductions-${mode}-state.json" + fi + + claim_project_items "$mode" "$state_file" "$repo" "$number" json +} + +move_board_item() { + item_id=$1 + status=$2 + python3 scripts/pipeline_board.py move "$item_id" "$status" +} + +# --- PR helpers --- + +pr_snapshot() { + repo=$1 + pr=$2 + python3 scripts/pipeline_pr.py snapshot --repo "$repo" --pr "$pr" --format json +} + +pr_wait_ci() { + repo=$1 + pr=$2 + timeout=${3:-900} + interval=${4:-30} + python3 scripts/pipeline_pr.py wait-ci --repo "$repo" --pr "$pr" --timeout "$timeout" --interval "$interval" --format json +} + +review_pipeline_context() { + repo=$1 + pr=${2-} + state_file=${3:-/tmp/problemreductions-review-state.json} + fmt=${4:-json} + + set -- scripts/pipeline_skill_context.py review-pipeline --repo "$repo" --state-file "$state_file" --format "$fmt" + if [ -n "$pr" ]; then + set -- "$@" --pr "$pr" + fi + python3 "$@" +} + +# --- Issue helpers --- + +issue_guards() { + repo=$1 + issue=$2 + repo_root=${3:-.} + python3 scripts/pipeline_checks.py issue-guards --repo "$repo" --issue "$issue" --repo-root "$repo_root" --format json +} + +issue_context() { + repo=$1 + issue=$2 + repo_root=${3:-.} + python3 scripts/pipeline_checks.py issue-context --repo "$repo" --issue "$issue" --repo-root "$repo_root" --format json +} + +# --- Worktree helpers --- + +create_issue_worktree() { + issue=$1 + slug=$2 + base=${3:-origin/main} + python3 scripts/pipeline_worktree.py create-issue --issue "$issue" --slug "$slug" --base "$base" --format json +} + +checkout_pr_worktree() { + repo=$1 + pr=$2 + python3 scripts/pipeline_worktree.py checkout-pr --repo "$repo" --pr "$pr" --format json +} + +merge_main_worktree() { + worktree=$1 + python3 scripts/pipeline_worktree.py merge-main --worktree "$worktree" --format json +} + +cleanup_pipeline_worktree() { + worktree=$1 + python3 scripts/pipeline_worktree.py cleanup --worktree "$worktree" --format json } # Poll a board column and dispatch a make target when new items appear. diff --git a/scripts/pipeline_board.py b/scripts/pipeline_board.py new file mode 100644 index 000000000..a596b08da --- /dev/null +++ b/scripts/pipeline_board.py @@ -0,0 +1,1113 @@ +#!/usr/bin/env python3 +"""Shared project-board logic for polling, recovery, and board CLI helpers.""" + +from __future__ import annotations + +import argparse +import json +import subprocess +import sys +from collections import Counter +from datetime import datetime, timezone +from pathlib import Path +from typing import Callable + +PROJECT_ID = "PVT_kwDOBrtarc4BRNVy" +STATUS_FIELD_ID = "PVTSSF_lADOBrtarc4BRNVyzg_GmQc" + +COPILOT_REVIEWER = "copilot-pull-request-reviewer[bot]" +COPILOT_REVIEWERS = { + "copilot-pull-request-reviewer", + COPILOT_REVIEWER, +} + +STATUS_BACKLOG = "Backlog" +STATUS_READY = "Ready" +STATUS_IN_PROGRESS = "In progress" +STATUS_REVIEW_POOL = "Review pool" +STATUS_UNDER_REVIEW = "Under review" +STATUS_FINAL_REVIEW = "Final review" +STATUS_ON_HOLD = "OnHold" +STATUS_DONE = "Done" + +STATUS_OPTION_IDS = { + STATUS_BACKLOG: "ab337660", + STATUS_READY: "f37d0d80", + STATUS_IN_PROGRESS: "a12cfc9c", + STATUS_REVIEW_POOL: "7082ed60", + STATUS_UNDER_REVIEW: "f04790ca", + STATUS_FINAL_REVIEW: "51a3d8bb", + STATUS_ON_HOLD: "48dfe446", + STATUS_DONE: "6aca54fa", +} + +STATUS_ALIASES = { + "backlog": STATUS_BACKLOG, + "ready": STATUS_READY, + "in-progress": STATUS_IN_PROGRESS, + "in_progress": STATUS_IN_PROGRESS, + "in progress": STATUS_IN_PROGRESS, + "review-pool": STATUS_REVIEW_POOL, + "review_pool": STATUS_REVIEW_POOL, + "review pool": STATUS_REVIEW_POOL, + "under-review": STATUS_UNDER_REVIEW, + "under_review": STATUS_UNDER_REVIEW, + "under review": STATUS_UNDER_REVIEW, + "final-review": STATUS_FINAL_REVIEW, + "final_review": STATUS_FINAL_REVIEW, + "final review": STATUS_FINAL_REVIEW, + "on-hold": STATUS_ON_HOLD, + "on_hold": STATUS_ON_HOLD, + "on hold": STATUS_ON_HOLD, + "onhold": STATUS_ON_HOLD, + "done": STATUS_DONE, +} + +FAILURE_LABELS = {"PoorWritten", "Wrong", "Trivial", "Useless"} + + +def run_gh(*args: str) -> str: + return subprocess.check_output(["gh", *args], text=True) + + +def fetch_board_items(owner: str, project_number: int, limit: int) -> dict: + return json.loads( + run_gh( + "project", + "item-list", + str(project_number), + "--owner", + owner, + "--format", + "json", + "--limit", + str(limit), + ) + ) + + +def fetch_pr_reviews(repo: str, pr_number: int) -> list[dict]: + data = json.loads(run_gh("api", f"repos/{repo}/pulls/{pr_number}/reviews")) + if not isinstance(data, list): + raise ValueError(f"Unexpected PR review payload for #{pr_number}: {data!r}") + return data + + +def fetch_pr_state(repo: str, pr_number: int) -> str: + return run_gh( + "pr", + "view", + str(pr_number), + "--repo", + repo, + "--json", + "state", + "--jq", + ".state", + ).strip() + + +def fetch_pr_info(repo: str, pr_number: int) -> dict: + data = json.loads( + run_gh( + "pr", + "view", + str(pr_number), + "--repo", + repo, + "--json", + "number,state,title,url", + ) + ) + if not isinstance(data, dict): + raise ValueError(f"Unexpected PR payload for #{pr_number}: {data!r}") + return data + + +def resolve_issue_pr(repo: str, issue_number: int) -> int | None: + data = json.loads( + run_gh( + "pr", + "list", + "-R", + repo, + "--search", + f"Fix #{issue_number} in:title state:open", + "--json", + "number", + "--limit", + "1", + ) + ) + if not data: + return None + return int(data[0]["number"]) + + +def item_identity(item: dict) -> str: + item_id = item.get("id") + if item_id is not None: + return str(item_id) + + content = item.get("content") or {} + number = content.get("number") + item_type = content.get("type", "item") + if number is not None: + return f"{item_type}:{number}" + + title = item.get("title") + if title: + return str(title) + + raise ValueError(f"Board item has no stable identity: {item!r}") + + +def load_state(state_file: Path) -> dict: + if not state_file.exists(): + return {"visible": {}, "pending": []} + + raw = state_file.read_text().strip() + if not raw: + return {"visible": {}, "pending": []} + + data = json.loads(raw) + if not isinstance(data, dict): + raise ValueError(f"State file must contain a JSON object: {state_file}") + + visible = data.get("visible", {}) + pending = data.get("pending", []) + if not isinstance(visible, dict) or not isinstance(pending, list): + raise ValueError(f"Invalid poll state format: {state_file}") + + return {"visible": visible, "pending": [str(item_id) for item_id in pending]} + + +def save_state(state_file: Path, state: dict) -> None: + state_file.parent.mkdir(parents=True, exist_ok=True) + state_file.write_text(json.dumps(state, indent=2, sort_keys=True) + "\n") + + +def has_copilot_review(reviews: list[dict]) -> bool: + return any( + (review.get("author") or review.get("user") or {}).get("login") + in COPILOT_REVIEWERS + for review in reviews + ) + + +def linked_pr_numbers(item: dict, repo: str | None = None) -> list[int]: + urls = item.get("linked pull requests") or [] + numbers: list[int] = [] + + if repo is not None: + prefix = f"https://github.com/{repo}/pull/" + for url in urls: + if not isinstance(url, str) or not url.startswith(prefix): + continue + suffix = url.removeprefix(prefix) + if suffix.isdigit(): + numbers.append(int(suffix)) + return numbers + + for url in urls: + if not isinstance(url, str): + continue + try: + numbers.append(int(url.rstrip("/").split("/")[-1])) + except ValueError: + continue + return numbers + + +def linked_repo_pr_numbers(item: dict, repo: str) -> list[int]: + return linked_pr_numbers(item, repo) + + +def entry_title(item: dict) -> str | None: + content = item.get("content") or {} + return content.get("title") or item.get("title") + + +def build_entry( + item: dict, + *, + number: int, + issue_number: int | None = None, + pr_number: int | None = None, +) -> dict: + return { + "number": number, + "issue_number": issue_number, + "pr_number": pr_number, + "status": item.get("status"), + "title": entry_title(item), + } + + +def ready_entries(board_data: dict) -> dict[str, dict]: + entries = {} + for item in board_data.get("items", []): + if item.get("status") != STATUS_READY: + continue + + content = item.get("content") or {} + number = content.get("number") + if number is None: + continue + + issue_number = int(number) + entries[item_identity(item)] = build_entry( + item, + number=issue_number, + issue_number=issue_number, + ) + return entries + + +def status_items(board_data: dict, status_name: str) -> list[dict]: + items = [] + for item in board_data.get("items", []): + if item.get("status") != status_name: + continue + + content = item.get("content") or {} + if content.get("type") != "Issue": + continue + + number = content.get("number") + if number is None: + continue + + entry = build_entry( + item, + number=int(number), + issue_number=int(number), + ) + entry["item_id"] = item_identity(item) + items.append(entry) + + return sorted(items, key=lambda entry: (entry["number"], entry["item_id"])) + + +def review_entries( + board_data: dict, + repo: str, + review_fetcher: Callable[[str, int], list[dict]], + pr_resolver: Callable[[str, int], int | None] | None, + pr_state_fetcher: Callable[[str, int], str], +) -> dict[str, dict]: + entries = {} + for item in board_data.get("items", []): + if item.get("status") != STATUS_REVIEW_POOL: + continue + + content = item.get("content") or {} + item_type = content.get("type") + number = content.get("number") + if number is None: + continue + + pr_number: int | None + if item_type == "PullRequest": + pr_number = int(number) + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + elif item_type == "Issue": + linked_numbers = linked_pr_numbers(item, repo) + if len(linked_numbers) > 1: + continue + if len(linked_numbers) == 1: + pr_number = linked_numbers[0] + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + else: + if pr_resolver is None: + raise ValueError("review mode requires pr_resolver for issue cards without linked PRs") + pr_number = pr_resolver(repo, int(number)) + if pr_number is None: + continue + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + else: + pr_number = None + + if pr_number is None: + continue + + reviews = review_fetcher(repo, pr_number) + if has_copilot_review(reviews): + issue_number = int(number) if item_type == "Issue" else None + entries[item_identity(item)] = build_entry( + item, + number=pr_number, + issue_number=issue_number, + pr_number=pr_number, + ) + return entries + + +def review_candidates( + board_data: dict, + repo: str, + review_fetcher: Callable[[str, int], list[dict]], + pr_resolver: Callable[[str, int], int | None] | None, + pr_info_fetcher: Callable[[str, int], dict], +) -> list[dict]: + candidates = [] + for item in board_data.get("items", []): + if item.get("status") != STATUS_REVIEW_POOL: + continue + + content = item.get("content") or {} + item_type = content.get("type") + number = content.get("number") + if number is None: + continue + + base_entry = build_entry(item, number=0) + base_entry["item_id"] = item_identity(item) + issue_number = int(number) if item_type == "Issue" else None + + if item_type == "PullRequest": + pr_number = int(number) + pr_info = pr_info_fetcher(repo, pr_number) + state = pr_info.get("state") + base_entry.update({"number": pr_number, "pr_number": pr_number}) + if state != "OPEN": + base_entry.update( + { + "eligibility": "stale-closed-pr", + "reason": f"linked PR #{pr_number} is {state}", + } + ) + candidates.append(base_entry) + continue + + reviews = review_fetcher(repo, pr_number) + if has_copilot_review(reviews): + base_entry.update({"eligibility": "eligible", "reason": "copilot reviewed"}) + else: + base_entry.update( + { + "eligibility": "waiting-for-copilot", + "reason": f"open PR #{pr_number} waiting for Copilot review", + } + ) + candidates.append(base_entry) + continue + + if item_type != "Issue": + continue + + base_entry["issue_number"] = issue_number + linked_numbers = linked_pr_numbers(item, repo) + if len(linked_numbers) > 1: + linked_infos = [pr_info_fetcher(repo, pr_number) for pr_number in linked_numbers] + open_numbers = [ + int(info["number"]) + for info in linked_infos + if str(info.get("state")).upper() == "OPEN" + ] + recommendation = open_numbers[0] if len(open_numbers) == 1 else None + base_entry.update( + { + "number": recommendation or int(linked_infos[0]["number"]), + "pr_number": recommendation, + "eligibility": "ambiguous-linked-prs", + "reason": "multiple linked repo PRs require confirmation", + "recommendation": recommendation, + "linked_repo_prs": [ + { + "number": int(info["number"]), + "state": str(info.get("state")), + "title": info.get("title"), + } + for info in linked_infos + ], + } + ) + candidates.append(base_entry) + continue + + if len(linked_numbers) == 1: + pr_number = linked_numbers[0] + pr_info = pr_info_fetcher(repo, pr_number) + state = pr_info.get("state") + base_entry.update({"number": pr_number, "pr_number": pr_number}) + if state != "OPEN": + base_entry.update( + { + "eligibility": "stale-closed-pr", + "reason": f"linked PR #{pr_number} is {state}", + } + ) + candidates.append(base_entry) + continue + else: + if pr_resolver is None: + raise ValueError("review candidate listing requires pr_resolver for issue cards without linked PRs") + pr_number = pr_resolver(repo, issue_number) + if pr_number is None: + base_entry.update( + { + "number": issue_number, + "pr_number": None, + "eligibility": "no-open-pr", + "reason": f"issue #{issue_number} has no open PR", + } + ) + candidates.append(base_entry) + continue + base_entry.update({"number": pr_number, "pr_number": pr_number}) + + reviews = review_fetcher(repo, pr_number) + if has_copilot_review(reviews): + base_entry.update({"eligibility": "eligible", "reason": "copilot reviewed"}) + else: + base_entry.update( + { + "eligibility": "waiting-for-copilot", + "reason": f"open PR #{pr_number} waiting for Copilot review", + } + ) + candidates.append(base_entry) + + return sorted( + candidates, + key=lambda entry: ( + entry["pr_number"] is None, + entry["number"], + entry["item_id"], + ), + ) + + +def final_review_entries( + board_data: dict, + repo: str, + pr_resolver: Callable[[str, int], int | None] | None, + pr_state_fetcher: Callable[[str, int], str], +) -> dict[str, dict]: + entries = {} + for item in board_data.get("items", []): + if item.get("status") != STATUS_FINAL_REVIEW: + continue + + content = item.get("content") or {} + item_type = content.get("type") + number = content.get("number") + if number is None: + continue + + pr_number: int | None + if item_type == "PullRequest": + pr_number = int(number) + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + elif item_type == "Issue": + linked_numbers = linked_pr_numbers(item, repo) + if len(linked_numbers) > 1: + continue + if len(linked_numbers) == 1: + pr_number = linked_numbers[0] + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + else: + if pr_resolver is None: + raise ValueError( + "final-review mode requires pr_resolver for issue cards without linked PRs" + ) + pr_number = pr_resolver(repo, int(number)) + if pr_number is None: + continue + if pr_state_fetcher(repo, pr_number) != "OPEN": + continue + else: + pr_number = None + + if pr_number is None: + continue + + issue_number = int(number) if item_type == "Issue" else None + entries[item_identity(item)] = build_entry( + item, + number=pr_number, + issue_number=issue_number, + pr_number=pr_number, + ) + return entries + + +def current_entries( + mode: str, + board_data: dict, + repo: str | None = None, + review_fetcher: Callable[[str, int], list[dict]] | None = None, + pr_resolver: Callable[[str, int], int | None] | None = None, + pr_state_fetcher: Callable[[str, int], str] | None = None, +) -> dict[str, dict]: + if mode == "ready": + return ready_entries(board_data) + if mode == "review": + if repo is None: + raise ValueError("repo is required in review mode") + if review_fetcher is None or pr_state_fetcher is None: + raise ValueError("review mode requires review_fetcher and pr_state_fetcher") + return review_entries( + board_data, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, + ) + if mode == "final-review": + if repo is None: + raise ValueError("repo is required in final-review mode") + if pr_state_fetcher is None: + raise ValueError("final-review mode requires pr_state_fetcher") + return final_review_entries( + board_data, + repo, + pr_resolver, + pr_state_fetcher, + ) + raise ValueError(f"Unsupported mode: {mode}") + + +def process_snapshot( + mode: str, + board_data: dict, + state_file: Path, + repo: str | None = None, + review_fetcher: Callable[[str, int], list[dict]] | None = None, + pr_resolver: Callable[[str, int], int | None] | None = None, + pr_state_fetcher: Callable[[str, int], str] | None = None, + target_number: int | None = None, +) -> tuple[str, int] | None: + next_entry = select_next_entry( + mode, + board_data, + state_file, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, + target_number, + ) + if next_entry is None: + return None + return str(next_entry["item_id"]), int(next_entry["number"]) + + +def select_next_entry( + mode: str, + board_data: dict, + state_file: Path, + repo: str | None = None, + review_fetcher: Callable[[str, int], list[dict]] | None = None, + pr_resolver: Callable[[str, int], int | None] | None = None, + pr_state_fetcher: Callable[[str, int], str] | None = None, + target_number: int | None = None, +) -> dict | None: + state = load_state(state_file) + previous_visible = state["visible"] + current_visible = current_entries( + mode, + board_data, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, + ) + + pending = [item_id for item_id in state["pending"] if item_id in current_visible] + entered = sorted( + (item_id for item_id in current_visible if item_id not in previous_visible), + key=lambda item_id: (current_visible[item_id]["number"], item_id), + ) + for item_id in entered: + if item_id not in pending: + pending.append(item_id) + + state["visible"] = current_visible + state["pending"] = pending + save_state(state_file, state) + + if target_number is not None: + matching_item_id = next( + ( + item_id + for item_id, entry in current_visible.items() + if int(entry["number"]) == target_number + ), + None, + ) + if matching_item_id is None: + return None + entry = dict(current_visible[matching_item_id]) + entry["item_id"] = matching_item_id + return entry + + if not pending: + return None + + item_id = pending[0] + entry = dict(current_visible[item_id]) + entry["item_id"] = item_id + return entry + + +def ack_item(state_file: Path, item_id: str) -> None: + state = load_state(state_file) + state["pending"] = [ + pending_id for pending_id in state["pending"] if pending_id != item_id + ] + save_state(state_file, state) + + +def label_names(issue: dict) -> set[str]: + return {label["name"] for label in issue.get("labels", [])} + + +def is_tracked_issue_title(title: str | None) -> bool: + if not title: + return False + return title.startswith("[Model]") or title.startswith("[Rule]") + + +def all_checks_green(pr: dict) -> bool: + statuses = pr.get("statusCheckRollup") or [] + if not statuses: + return False + + for status in statuses: + typename = status.get("__typename") + if typename == "CheckRun": + if status.get("status") != "COMPLETED": + return False + if status.get("conclusion") not in {"SUCCESS", "SKIPPED", "NEUTRAL"}: + return False + elif typename == "StatusContext": + if status.get("state") != "SUCCESS": + return False + return True + + +def infer_issue_status( + issue: dict, + linked_prs: list[dict], + pr_reviews: dict[int, list[dict]], +) -> tuple[str, str]: + labels = label_names(issue) + merged_prs = [pr for pr in linked_prs if pr.get("mergedAt")] + open_prs = [pr for pr in linked_prs if pr.get("state") == "OPEN"] + + if merged_prs: + pr_numbers = ", ".join(f"#{pr['number']}" for pr in merged_prs) + return STATUS_DONE, f"linked merged PR {pr_numbers}" + + if issue.get("state") == "CLOSED": + return STATUS_DONE, "issue itself is closed" + + if open_prs: + waiting_for_copilot = [ + pr + for pr in open_prs + if not has_copilot_review(pr_reviews.get(int(pr["number"]), [])) + ] + if waiting_for_copilot: + pr_numbers = ", ".join(f"#{pr['number']}" for pr in waiting_for_copilot) + return STATUS_REVIEW_POOL, f"open PR {pr_numbers} waiting for Copilot review" + + green_prs = [pr for pr in open_prs if all_checks_green(pr)] + if len(green_prs) == len(open_prs): + pr_numbers = ", ".join(f"#{pr['number']}" for pr in open_prs) + return STATUS_FINAL_REVIEW, f"Copilot reviewed green open PR {pr_numbers}" + + pr_numbers = ", ".join(f"#{pr['number']}" for pr in open_prs) + return STATUS_REVIEW_POOL, f"open PR {pr_numbers} still implementing or fixing review" + + if "Good" in labels: + return STATUS_READY, 'label "Good" present and no linked PR' + + if labels & FAILURE_LABELS: + bad = ", ".join(sorted(labels & FAILURE_LABELS)) + return STATUS_BACKLOG, f"failure labels present: {bad}" + + return STATUS_BACKLOG, "default backlog: no linked PR and no Ready signal" + + +def build_recovery_plan( + board_data: dict, + issues: list[dict], + prs: list[dict], + pr_reviews: dict[int, list[dict]], +) -> list[dict]: + issues_by_number = {issue["number"]: issue for issue in issues} + prs_by_number = {pr["number"]: pr for pr in prs} + + plan = [] + for item in board_data.get("items", []): + content = item.get("content") or {} + issue_number = content.get("number") + if issue_number is None: + continue + + issue = issues_by_number.get(issue_number) + if issue is None: + continue + + title = content.get("title") or issue.get("title") + if not is_tracked_issue_title(title): + continue + + linked_prs = [ + prs_by_number[pr_number] + for pr_number in linked_pr_numbers(item) + if pr_number in prs_by_number + ] + status_name, reason = infer_issue_status(issue, linked_prs, pr_reviews) + plan.append( + { + "item_id": item["id"], + "issue_number": issue_number, + "title": title, + "current_status": item.get("status"), + "proposed_status": status_name, + "option_id": STATUS_OPTION_IDS[status_name], + "reason": reason, + } + ) + + return sorted(plan, key=lambda entry: entry["issue_number"]) + + +def save_backup( + backup_file: Path, + *, + board_data: dict, + issues: list[dict], + prs: list[dict], + pr_reviews: dict[int, list[dict]], + plan: list[dict], +) -> None: + backup_file.parent.mkdir(parents=True, exist_ok=True) + payload = { + "generated_at": datetime.now(timezone.utc).isoformat(), + "board_data": board_data, + "issues": issues, + "prs": prs, + "pr_reviews": pr_reviews, + "plan": plan, + } + backup_file.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") + + +def default_backup_path(project_number: int) -> Path: + stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") + return Path("/tmp") / f"project-{project_number}-status-recovery-{stamp}.json" + + +def print_summary(plan: list[dict]) -> None: + counts = Counter(entry["proposed_status"] for entry in plan) + print("Proposed status counts:") + for status_name in [ + STATUS_BACKLOG, + STATUS_READY, + STATUS_REVIEW_POOL, + STATUS_FINAL_REVIEW, + STATUS_DONE, + ]: + print(f" {status_name}: {counts.get(status_name, 0)}") + + +def print_examples(plan: list[dict], limit: int = 20) -> None: + print("") + print(f"First {min(limit, len(plan))} assignments:") + for entry in plan[:limit]: + print( + f" #{entry['issue_number']:<4} {entry['proposed_status']:<13} " + f"{entry['reason']} | {entry['title']}" + ) + + +def normalize_status_name(status: str) -> str: + normalized = status.strip() + if normalized in STATUS_OPTION_IDS: + return normalized + + alias = STATUS_ALIASES.get(normalized.lower()) + if alias is None: + choices = ", ".join(sorted(STATUS_OPTION_IDS)) + raise ValueError(f"Unsupported status {status!r}. Expected one of: {choices}") + return alias + + +def claimed_status_for_mode(mode: str) -> str: + if mode == "ready": + return STATUS_IN_PROGRESS + if mode == "review": + return STATUS_UNDER_REVIEW + raise ValueError(f"Unsupported claim-next mode: {mode}") + + +def claim_next_entry( + mode: str, + board_data: dict, + state_file: Path, + repo: str | None = None, + review_fetcher: Callable[[str, int], list[dict]] | None = None, + pr_resolver: Callable[[str, int], int | None] | None = None, + pr_state_fetcher: Callable[[str, int], str] | None = None, + target_number: int | None = None, + mover: Callable[[str, str], None] | None = None, +) -> dict | None: + next_entry = select_next_entry( + mode, + board_data, + state_file, + repo=repo, + review_fetcher=review_fetcher, + pr_resolver=pr_resolver, + pr_state_fetcher=pr_state_fetcher, + target_number=target_number, + ) + if next_entry is None: + return None + + claimed_status = claimed_status_for_mode(mode) + move = mover or move_item + move(str(next_entry["item_id"]), claimed_status) + return { + **next_entry, + "claimed": True, + "claimed_status": claimed_status, + } + + +def move_item( + item_id: str, + status: str, + *, + project_id: str = PROJECT_ID, + field_id: str = STATUS_FIELD_ID, +) -> None: + status_name = normalize_status_name(status) + subprocess.check_call( + [ + "gh", + "project", + "item-edit", + "--project-id", + project_id, + "--id", + item_id, + "--field-id", + field_id, + "--single-select-option-id", + STATUS_OPTION_IDS[status_name], + ] + ) + + +def apply_plan( + plan: list[dict], + *, + project_id: str = PROJECT_ID, + field_id: str = STATUS_FIELD_ID, +) -> int: + changed = 0 + for entry in plan: + if entry["current_status"] == entry["proposed_status"]: + continue + move_item( + entry["item_id"], + entry["proposed_status"], + project_id=project_id, + field_id=field_id, + ) + changed += 1 + return changed + + +def print_next_item( + next_item: dict | None, + *, + mode: str, + fmt: str = "text", +) -> int: + if next_item is None: + return 1 + + if fmt == "json": + payload = {"mode": mode, **next_item} + print(json.dumps(payload)) + else: + print(f"{next_item['item_id']}\t{next_item['number']}") + return 0 + + +def print_claim_result( + claim_result: dict | None, + *, + mode: str, + fmt: str = "json", +) -> int: + if claim_result is None: + return 1 + + if fmt == "json": + payload = {"mode": mode, **claim_result} + print(json.dumps(payload)) + else: + print( + f"{claim_result['item_id']}\t" + f"{claim_result['number']}\t" + f"{claim_result['claimed_status']}" + ) + return 0 + + +def print_candidate_list( + mode: str, + items: list[dict], + *, + fmt: str = "text", +) -> int: + if fmt == "json": + print(json.dumps({"mode": mode, "items": items})) + return 0 + + for item in items: + number = item.get("pr_number") or item.get("issue_number") or item["number"] + title = item.get("title") or "" + eligibility = item.get("eligibility") or "" + print(f"{item['item_id']}\t{number}\t{eligibility}\t{title}") + return 0 + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="Project board automation helpers.") + subparsers = parser.add_subparsers(dest="command", required=True) + + next_parser = subparsers.add_parser("next") + next_parser.add_argument("mode", choices=["ready", "review", "final-review"]) + next_parser.add_argument("state_file", type=Path) + next_parser.add_argument("--repo") + next_parser.add_argument("--owner", default="CodingThrust") + next_parser.add_argument("--project-number", type=int, default=8) + next_parser.add_argument("--limit", type=int, default=500) + next_parser.add_argument("--number", type=int) + next_parser.add_argument("--format", choices=["text", "json"], default="text") + + claim_parser = subparsers.add_parser("claim-next") + claim_parser.add_argument("mode", choices=["ready", "review"]) + claim_parser.add_argument("state_file", type=Path) + claim_parser.add_argument("--repo") + claim_parser.add_argument("--owner", default="CodingThrust") + claim_parser.add_argument("--project-number", type=int, default=8) + claim_parser.add_argument("--limit", type=int, default=500) + claim_parser.add_argument("--number", type=int) + claim_parser.add_argument("--format", choices=["text", "json"], default="json") + claim_parser.add_argument("--project-id", default=PROJECT_ID) + claim_parser.add_argument("--field-id", default=STATUS_FIELD_ID) + + ack_parser = subparsers.add_parser("ack") + ack_parser.add_argument("state_file", type=Path) + ack_parser.add_argument("item_id") + + list_parser = subparsers.add_parser("list") + list_parser.add_argument("mode", choices=["ready", "in-progress", "review"]) + list_parser.add_argument("--repo") + list_parser.add_argument("--owner", default="CodingThrust") + list_parser.add_argument("--project-number", type=int, default=8) + list_parser.add_argument("--limit", type=int, default=500) + list_parser.add_argument("--format", choices=["text", "json"], default="text") + + move_parser = subparsers.add_parser("move") + move_parser.add_argument("item_id") + move_parser.add_argument("status") + move_parser.add_argument("--project-id", default=PROJECT_ID) + move_parser.add_argument("--field-id", default=STATUS_FIELD_ID) + + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + + if args.command == "ack": + ack_item(args.state_file, args.item_id) + return 0 + + if args.command == "move": + move_item( + args.item_id, + args.status, + project_id=args.project_id, + field_id=args.field_id, + ) + return 0 + + if args.command == "claim-next": + if args.mode == "review" and not args.repo: + raise SystemExit("--repo is required in claim-next review mode") + board_data = fetch_board_items(args.owner, args.project_number, args.limit) + claim_result = claim_next_entry( + args.mode, + board_data, + args.state_file, + repo=args.repo, + review_fetcher=fetch_pr_reviews, + pr_resolver=resolve_issue_pr, + pr_state_fetcher=fetch_pr_state, + target_number=args.number, + mover=lambda item_id, status: move_item( + item_id, + status, + project_id=args.project_id, + field_id=args.field_id, + ), + ) + return print_claim_result(claim_result, mode=args.mode, fmt=args.format) + + if args.command == "list": + if args.mode == "review" and not args.repo: + raise SystemExit("--repo is required in list review mode") + board_data = fetch_board_items(args.owner, args.project_number, args.limit) + if args.mode == "ready": + items = status_items(board_data, STATUS_READY) + return print_candidate_list(args.mode, items, fmt=args.format) + if args.mode == "in-progress": + items = status_items(board_data, STATUS_IN_PROGRESS) + return print_candidate_list(args.mode, items, fmt=args.format) + if args.mode == "review": + items = review_candidates( + board_data, + args.repo, + fetch_pr_reviews, + resolve_issue_pr, + fetch_pr_info, + ) + return print_candidate_list(args.mode, items, fmt=args.format) + raise SystemExit(f"Unsupported list mode: {args.mode}") + + if args.mode in {"review", "final-review"} and not args.repo: + raise SystemExit(f"--repo is required in {args.mode} mode") + + board_data = fetch_board_items(args.owner, args.project_number, args.limit) + next_item = select_next_entry( + args.mode, + board_data, + args.state_file, + repo=args.repo, + review_fetcher=fetch_pr_reviews, + pr_resolver=resolve_issue_pr, + pr_state_fetcher=fetch_pr_state, + target_number=args.number, + ) + return print_next_item(next_item, mode=args.mode, fmt=args.format) + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/pipeline_checks.py b/scripts/pipeline_checks.py new file mode 100644 index 000000000..489a95298 --- /dev/null +++ b/scripts/pipeline_checks.py @@ -0,0 +1,758 @@ +#!/usr/bin/env python3 +"""Deterministic review checks for scope detection and file whitelists.""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from pathlib import Path + + +MODEL_WHITELIST = [ + "src/models/", + "src/unit_tests/models/", + "src/example_db/model_builders.rs", + "src/example_db/rule_builders.rs", + "docs/paper/reductions.typ", + "docs/src/reductions/problem_schemas.json", + "docs/src/reductions/reduction_graph.json", + "tests/suites/trait_consistency.rs", +] + +RULE_WHITELIST = [ + "src/rules/", + "src/unit_tests/rules/", + "src/example_db/rule_builders.rs", + "src/models/", + "docs/paper/reductions.typ", + "docs/src/reductions/reduction_graph.json", + "docs/src/reductions/problem_schemas.json", +] + +IGNORED_RULE_FILES = { + "src/rules/mod.rs", + "src/rules/traits.rs", + "src/rules/cost.rs", + "src/rules/graph.rs", + "src/rules/registry.rs", +} + + +def snake_to_camel(name: str) -> str: + return "".join(part.capitalize() for part in name.split("_") if part) + + +def camel_to_snake(name: str) -> str: + chars: list[str] = [] + for index, char in enumerate(name): + if char.isupper() and index > 0 and ( + not name[index - 1].isupper() + or (index + 1 < len(name) and name[index + 1].islower()) + ): + chars.append("_") + chars.append(char.lower()) + return "".join(chars) + + +def is_new_model_file(path: str) -> bool: + return ( + path.startswith("src/models/") + and path.endswith(".rs") + and not path.endswith("/mod.rs") + ) + + +def is_new_rule_file(path: str) -> bool: + return ( + path.startswith("src/rules/") + and path.endswith(".rs") + and path not in IGNORED_RULE_FILES + and not path.endswith("/mod.rs") + ) + + +def detect_scope_from_paths(*, added_files: list[str], changed_files: list[str]) -> dict: + models = [] + rules = [] + + for path in added_files: + if is_new_model_file(path): + parts = path.split("/") + category = parts[2] + file_stem = Path(path).stem + models.append( + { + "path": path, + "category": category, + "file_stem": file_stem, + "problem_name": snake_to_camel(file_stem), + } + ) + elif is_new_rule_file(path): + rules.append( + { + "path": path, + "rule_stem": Path(path).stem, + } + ) + + if models and rules: + review_type = "model+rule" + elif models: + review_type = "model" + elif rules: + review_type = "rule" + else: + review_type = "generic" + + return { + "review_type": review_type, + "models": models, + "rules": rules, + "added_files": list(added_files), + "changed_files": list(changed_files), + } + + +def path_allowed(kind: str, path: str) -> tuple[bool, str | None]: + if kind == "model": + allowed = any( + path == prefix or path.startswith(prefix) + for prefix in MODEL_WHITELIST + ) + if not allowed: + return False, "not in whitelist for model PR" + return True, None + + if kind == "rule": + allowed = any( + path == prefix or path.startswith(prefix) + for prefix in RULE_WHITELIST + ) + if not allowed: + return False, "not in whitelist for rule PR" + return True, None + + raise ValueError(f"Unsupported whitelist kind: {kind}") + + +def file_whitelist_check(kind: str, files: list[str]) -> dict: + violations = [] + for path in files: + ok, reason = path_allowed(kind, path) + if not ok: + violations.append({"path": path, "reason": reason}) + + return { + "kind": kind, + "ok": not violations, + "files": list(files), + "violations": violations, + } + + +def read_text(path: Path) -> str: + return path.read_text() if path.exists() else "" + + +def find_model_file(repo_root: Path, file_stem: str) -> Path | None: + matches = sorted((repo_root / "src/models").glob(f"*/{file_stem}.rs")) + return matches[0] if matches else None + + +def find_problem_declaration(repo_root: Path, problem_name: str) -> Path | None: + pattern = re.compile(rf"\b(?:pub\s+)?(?:struct|enum)\s+{re.escape(problem_name)}\b") + model_root = repo_root / "src/models" + if not model_root.exists(): + return None + + for path in sorted(model_root.rglob("*.rs")): + if pattern.search(path.read_text()): + return path + return None + + +def check_entry( + *, + status: str, + path: str | None = None, + detail: str | None = None, +) -> dict: + return { + "status": status, + "path": path, + "detail": detail, + } + + +def model_completeness(repo_root: Path, name: str) -> dict: + file_stem = camel_to_snake(name) + model_file = find_model_file(repo_root, file_stem) + test_file = None + if model_file is not None: + category = model_file.parent.name + test_file = repo_root / "src/unit_tests/models" / category / f"{file_stem}.rs" + + model_text = read_text(model_file) if model_file is not None else "" + trait_text = read_text(repo_root / "src/unit_tests/trait_consistency.rs") + paper_text = read_text(repo_root / "docs/paper/reductions.typ") + + is_optimization = f"impl OptimizationProblem for {name}" in model_text + + checks = { + "model_file": ( + check_entry(status="pass", path=str(model_file.relative_to(repo_root))) + if model_file is not None + else check_entry(status="fail", detail="missing model implementation file") + ), + "problem_schema": ( + check_entry(status="pass", path=str(model_file.relative_to(repo_root))) + if model_file is not None and f'name: "{name}"' in model_text + else check_entry(status="fail", detail="missing ProblemSchemaEntry for model") + ), + "declare_variants": ( + check_entry(status="pass", path=str(model_file.relative_to(repo_root))) + if model_file is not None + and "crate::declare_variants!" in model_text + and re.search(r"\b(?:default\s+)?(?:opt|sat)\b", model_text) + else check_entry(status="fail", detail="missing declare_variants! with opt/sat entries") + ), + "canonical_example": ( + check_entry(status="pass", path=str(model_file.relative_to(repo_root))) + if model_file is not None and "canonical_model_example_specs" in model_text + else check_entry(status="fail", detail="missing canonical_model_example_specs") + ), + "unit_tests": ( + check_entry(status="pass", path=str(test_file.relative_to(repo_root))) + if test_file is not None and test_file.exists() + else check_entry(status="fail", detail="missing model unit tests") + ), + "paper_definition": ( + check_entry(status="pass", path="docs/paper/reductions.typ") + if f'#problem-def("{name}")' in paper_text + else check_entry(status="fail", detail="missing problem-def entry in paper") + ), + "paper_display_name": ( + check_entry(status="pass", path="docs/paper/reductions.typ") + if f'"{name}":' in paper_text + else check_entry(status="fail", detail="missing display-name entry in paper") + ), + "trait_consistency": ( + check_entry(status="pass", path="src/unit_tests/trait_consistency.rs") + if name in trait_text and "test_all_problems_implement_trait_correctly" in trait_text + else check_entry(status="fail", detail="missing trait consistency entry") + ), + "trait_direction": ( + check_entry(status="pass", path="src/unit_tests/trait_consistency.rs") + if not is_optimization + else ( + check_entry(status="pass", path="src/unit_tests/trait_consistency.rs") + if name in trait_text.split("fn test_direction()", 1)[-1] + else check_entry(status="fail", detail="missing optimization direction check") + ) + ), + } + + missing = [check_id for check_id, entry in checks.items() if entry["status"] == "fail"] + return { + "kind": "model", + "name": name, + "ok": not missing, + "checks": checks, + "missing": missing, + } + + +def rule_completeness( + repo_root: Path, + name: str, + *, + source: str | None = None, + target: str | None = None, +) -> dict: + rule_file = repo_root / "src/rules" / f"{name}.rs" + test_file = repo_root / "src/unit_tests/rules" / f"{name}.rs" + mod_file = repo_root / "src/rules/mod.rs" + paper_file = repo_root / "docs/paper/reductions.typ" + + rule_text = read_text(rule_file) + mod_text = read_text(mod_file) + paper_text = read_text(paper_file) + + paper_pattern = None + if source and target: + paper_pattern = f'#reduction-rule("{source}", "{target}"' + + checks = { + "rule_file": ( + check_entry(status="pass", path=str(rule_file.relative_to(repo_root))) + if rule_file.exists() + else check_entry(status="fail", detail="missing rule implementation file") + ), + "module_registration": ( + check_entry(status="pass", path=str(mod_file.relative_to(repo_root))) + if rule_file.exists() and name in mod_text + else check_entry(status="fail", detail="missing src/rules/mod.rs registration") + ), + "unit_tests": ( + check_entry(status="pass", path=str(test_file.relative_to(repo_root))) + if test_file.exists() + else check_entry(status="fail", detail="missing rule unit tests") + ), + "overhead_form": ( + check_entry(status="pass", path=str(rule_file.relative_to(repo_root))) + if rule_file.exists() and "#[reduction(overhead = {" in rule_text + else check_entry(status="fail", detail="missing #[reduction(overhead = {...})] form") + ), + "canonical_example": ( + check_entry(status="pass", path=str(rule_file.relative_to(repo_root))) + if rule_file.exists() and "canonical_rule_example_specs" in rule_text + else check_entry(status="fail", detail="missing canonical_rule_example_specs") + ), + "paper_rule": ( + check_entry(status="pass", path=str(paper_file.relative_to(repo_root))) + if paper_pattern is not None and paper_pattern in paper_text + else check_entry( + status="fail", + detail=( + "missing reduction-rule entry in paper" + if paper_pattern is not None + else "source/target required to check paper reduction-rule entry" + ), + ) + ), + } + + missing = [check_id for check_id, entry in checks.items() if entry["status"] == "fail"] + return { + "kind": "rule", + "name": name, + "source": source, + "target": target, + "ok": not missing, + "checks": checks, + "missing": missing, + } + + +def completeness_check( + kind: str, + repo_root: str | Path, + *, + name: str, + source: str | None = None, + target: str | None = None, +) -> dict: + repo_root = Path(repo_root) + if kind == "model": + return model_completeness(repo_root, name) + if kind == "rule": + return rule_completeness(repo_root, name, source=source, target=target) + raise ValueError(f"Unsupported completeness kind: {kind}") + + +def infer_review_subject( + scope: dict, + *, + kind: str | None = None, + name: str | None = None, + source: str | None = None, + target: str | None = None, +) -> dict: + if kind is not None: + return { + "kind": kind, + "name": name, + "source": source, + "target": target, + "inferred": False, + } + + review_type = scope.get("review_type") + if review_type == "model" and len(scope.get("models", [])) == 1: + model = scope["models"][0] + return { + "kind": "model", + "name": model.get("problem_name"), + "source": None, + "target": None, + "inferred": True, + } + + if review_type == "rule" and len(scope.get("rules", [])) == 1: + rule = scope["rules"][0] + return { + "kind": "rule", + "name": rule.get("rule_stem"), + "source": source, + "target": target, + "inferred": True, + } + + return { + "kind": "generic", + "name": None, + "source": None, + "target": None, + "inferred": True, + } + + +def skipped_check(reason: str) -> dict: + return { + "ok": True, + "skipped": True, + "reason": reason, + } + + +def build_review_context( + repo_root: str | Path, + *, + diff_stat: str, + scope: dict, + subject: dict, +) -> dict: + changed_files = list(scope.get("changed_files", [])) + kind = subject.get("kind") + + if kind in {"model", "rule"}: + whitelist = file_whitelist_check(kind, changed_files) + whitelist["skipped"] = False + whitelist["reason"] = None + else: + whitelist = skipped_check("no model/rule subject available") + + if kind == "model" and subject.get("name"): + completeness = completeness_check( + "model", + repo_root, + name=subject["name"], + ) + completeness["skipped"] = False + completeness["reason"] = None + elif kind == "rule" and subject.get("name") and subject.get("source") and subject.get("target"): + completeness = completeness_check( + "rule", + repo_root, + name=subject["name"], + source=subject["source"], + target=subject["target"], + ) + completeness["skipped"] = False + completeness["reason"] = None + elif kind == "rule": + completeness = skipped_check("rule completeness requires source and target") + else: + completeness = skipped_check("no model/rule subject available") + + return { + "diff_stat": diff_stat, + "changed_files": changed_files, + "scope": scope, + "subject": subject, + "whitelist": whitelist, + "completeness": completeness, + } + + +RULE_TITLE_RE = re.compile(r"^\[Rule\]\s+(?P<source>.+?)\s+to\s+(?P<target>.+?)\s*$") +MODEL_TITLE_RE = re.compile(r"^\[Model\]\s+(?P<name>.+?)\s*$") + + +def issue_kind_from_title(title: str | None) -> tuple[str, str | None, str | None]: + title = (title or "").strip() + + rule_match = RULE_TITLE_RE.match(title) + if rule_match: + return "rule", rule_match.group("source"), rule_match.group("target") + + if MODEL_TITLE_RE.match(title): + return "model", None, None + + return "other", None, None + + +def normalize_issue_comment(comment: dict) -> dict: + author = comment.get("author") or comment.get("user") or {} + return { + "author": author.get("login") or author.get("name") or "", + "body": comment.get("body", ""), + } + + +def normalize_existing_pr(pr: dict) -> dict: + return { + "number": pr.get("number"), + "head_ref_name": pr.get("headRefName"), + "url": pr.get("url"), + } + + +def issue_guard_check( + repo_root: str | Path, + *, + issue: dict, + existing_prs: list[dict], +) -> dict: + repo_root = Path(repo_root) + title = issue.get("title", "") + labels = [label.get("name") for label in issue.get("labels", []) if label.get("name")] + comments = [normalize_issue_comment(comment) for comment in issue.get("comments", [])] + kind, source_problem, target_problem = issue_kind_from_title(title) + + checks = { + "good_label": ( + check_entry(status="pass", detail='label "Good" present') + if "Good" in labels + else check_entry(status="fail", detail='missing required "Good" label') + ), + "source_model": check_entry(status="skip", detail="not a rule issue"), + "target_model": check_entry(status="skip", detail="not a rule issue"), + } + + if kind == "rule": + source_path = find_problem_declaration(repo_root, source_problem or "") + target_path = find_problem_declaration(repo_root, target_problem or "") + checks["source_model"] = ( + check_entry(status="pass", path=str(source_path.relative_to(repo_root))) + if source_path is not None + else check_entry( + status="fail", + detail=f"model {source_problem!r} not found under src/models/", + ) + ) + checks["target_model"] = ( + check_entry(status="pass", path=str(target_path.relative_to(repo_root))) + if target_path is not None + else check_entry( + status="fail", + detail=f"model {target_problem!r} not found under src/models/", + ) + ) + + missing = [name for name, entry in checks.items() if entry["status"] == "fail"] + normalized_existing_prs = [normalize_existing_pr(pr) for pr in existing_prs] + resume_pr = normalized_existing_prs[0] if normalized_existing_prs else None + + return { + "issue_number": issue.get("number"), + "title": title, + "body": issue.get("body"), + "state": issue.get("state"), + "url": issue.get("url"), + "labels": labels, + "comments": comments, + "kind": kind, + "source_problem": source_problem, + "target_problem": target_problem, + "ok": not missing, + "checks": checks, + "missing": missing, + "existing_prs": normalized_existing_prs, + "resume_pr": resume_pr, + "action": "resume-pr" if resume_pr is not None else "create-pr", + } + + +def issue_context_check( + repo_root: str | Path, + *, + issue: dict, + existing_prs: list[dict], +) -> dict: + return issue_guard_check( + repo_root, + issue=issue, + existing_prs=existing_prs, + ) + + +def run_gh_json(*args: str): + return json.loads(subprocess.check_output(["gh", *args], text=True)) + + +def fetch_issue(repo: str, issue_number: int) -> dict: + return run_gh_json( + "issue", + "view", + str(issue_number), + "--repo", + repo, + "--json", + "number,title,body,state,url,labels,comments", + ) + + +def fetch_existing_prs(repo: str, issue_number: int) -> list[dict]: + data = run_gh_json( + "pr", + "list", + "--repo", + repo, + "--state", + "open", + "--search", + f"Fixes #{issue_number}", + "--json", + "number,headRefName,url", + ) + if not isinstance(data, list): + raise ValueError(f"Unexpected PR list payload for issue #{issue_number}: {data!r}") + return data + + +def git_output(*args: str) -> list[str]: + output = subprocess.check_output(["git", *args], text=True) + return [line for line in output.splitlines() if line] + + +def git_text(*args: str) -> str: + return subprocess.check_output(["git", *args], text=True) + + +def load_file_list(path: str | Path) -> list[str]: + lines = Path(path).read_text().splitlines() + return [line.strip() for line in lines if line.strip()] + + +def emit_result(result: dict, fmt: str) -> None: + print(json.dumps(result, indent=2, sort_keys=True)) + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="Pipeline review checks.") + subparsers = parser.add_subparsers(dest="command", required=True) + + detect = subparsers.add_parser("detect-scope") + detect.add_argument("--base", required=True) + detect.add_argument("--head", required=True) + detect.add_argument("--format", choices=["json", "text"], default="json") + + whitelist = subparsers.add_parser("file-whitelist") + whitelist.add_argument("--kind", choices=["model", "rule"], required=True) + whitelist.add_argument("--files-file", required=True) + whitelist.add_argument("--format", choices=["json", "text"], default="json") + + completeness = subparsers.add_parser("completeness") + completeness.add_argument("--kind", choices=["model", "rule"], required=True) + completeness.add_argument("--name", required=True) + completeness.add_argument("--source") + completeness.add_argument("--target") + completeness.add_argument("--repo-root", default=".") + completeness.add_argument("--format", choices=["json", "text"], default="json") + + review_context = subparsers.add_parser("review-context") + review_context.add_argument("--repo-root", default=".") + review_context.add_argument("--base", required=True) + review_context.add_argument("--head", required=True) + review_context.add_argument("--kind", choices=["model", "rule", "generic"]) + review_context.add_argument("--name") + review_context.add_argument("--source") + review_context.add_argument("--target") + review_context.add_argument("--format", choices=["json", "text"], default="json") + + issue_guards = subparsers.add_parser("issue-guards") + issue_guards.add_argument("--repo", required=True) + issue_guards.add_argument("--issue", required=True, type=int) + issue_guards.add_argument("--repo-root", default=".") + issue_guards.add_argument("--format", choices=["json", "text"], default="json") + + issue_context = subparsers.add_parser("issue-context") + issue_context.add_argument("--repo", required=True) + issue_context.add_argument("--issue", required=True, type=int) + issue_context.add_argument("--repo-root", default=".") + issue_context.add_argument("--format", choices=["json", "text"], default="json") + + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + + if args.command == "detect-scope": + changed_files = git_output("diff", "--name-only", f"{args.base}..{args.head}") + added_files = git_output( + "diff", + "--name-only", + "--diff-filter=A", + f"{args.base}..{args.head}", + ) + emit_result( + detect_scope_from_paths( + added_files=added_files, + changed_files=changed_files, + ), + args.format, + ) + return 0 + + if args.command == "file-whitelist": + emit_result( + file_whitelist_check(args.kind, load_file_list(args.files_file)), + args.format, + ) + return 0 + + if args.command == "completeness": + emit_result( + completeness_check( + args.kind, + args.repo_root, + name=args.name, + source=args.source, + target=args.target, + ), + args.format, + ) + return 0 + + if args.command == "review-context": + changed_files = git_output("diff", "--name-only", f"{args.base}..{args.head}") + added_files = git_output( + "diff", + "--name-only", + "--diff-filter=A", + f"{args.base}..{args.head}", + ) + scope = detect_scope_from_paths( + added_files=added_files, + changed_files=changed_files, + ) + subject = infer_review_subject( + scope, + kind=args.kind, + name=args.name, + source=args.source, + target=args.target, + ) + emit_result( + build_review_context( + args.repo_root, + diff_stat=git_text("diff", "--stat", f"{args.base}..{args.head}"), + scope=scope, + subject=subject, + ), + args.format, + ) + return 0 + + if args.command in {"issue-guards", "issue-context"}: + emit_result( + issue_context_check( + args.repo_root, + issue=fetch_issue(args.repo, args.issue), + existing_prs=fetch_existing_prs(args.repo, args.issue), + ), + args.format, + ) + return 0 + + raise AssertionError(f"Unhandled command: {args.command}") + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/pipeline_pr.py b/scripts/pipeline_pr.py new file mode 100644 index 000000000..9ab2c61e4 --- /dev/null +++ b/scripts/pipeline_pr.py @@ -0,0 +1,891 @@ +#!/usr/bin/env python3 +"""Shared PR metadata, comments, CI, and codecov helpers.""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +import time +from typing import Callable +from urllib.parse import unquote + +COPILOT_REVIEWERS = { + "copilot-pull-request-reviewer", + "copilot-pull-request-reviewer[bot]", +} +CODECOV_REVIEWER = "codecov[bot]" + +_CLOSING_ISSUE_RE = re.compile( + r"(?i)\b(?:fix(?:e[sd])?|close[sd]?|resolve[sd]?)\s+" + r"(?:(?:[-.\w]+/[-.\w]+)#)?(\d+)\b" +) +_GENERIC_ISSUE_RE = re.compile(r"(?<![A-Za-z0-9_])#(\d+)\b") +_PATCH_COVERAGE_RE = re.compile( + r"(?i)patch coverage(?:\s+is|:)?\s*`?(\d+(?:\.\d+)?)%`?" +) +_PROJECT_COVERAGE_RE = re.compile( + r"(?i)project coverage(?:\s+is|:)?\s*`?(\d+(?:\.\d+)?)%`?" +) +_FILEPATH_RE = re.compile(r"filepath=([^&\"\s)]+)") + + +def run_gh(*args: str) -> str: + return subprocess.check_output(["gh", *args], text=True) + + +def run_gh_json(*args: str): + return json.loads(run_gh(*args)) + + +def run_gh_checked(*args: str) -> None: + subprocess.check_call(["gh", *args]) + + +def login_for(entry: dict) -> str: + return (entry.get("user") or entry.get("author") or {}).get("login", "") + + +def is_bot_login(login: str) -> bool: + return login.endswith("[bot]") or login in COPILOT_REVIEWERS + + +def extract_linked_issue_number(title: str | None, body: str | None) -> int | None: + for text in [body or "", title or ""]: + match = _CLOSING_ISSUE_RE.search(text) + if match: + return int(match.group(1)) + + for text in [body or "", title or ""]: + match = _GENERIC_ISSUE_RE.search(text) + if match: + return int(match.group(1)) + + return None + + +def normalize_issue_thread_comment(comment: dict) -> dict: + login = login_for(comment) + created_at = comment.get("createdAt") or comment.get("created_at") + return { + "author": login, + "body": comment.get("body", ""), + "created_at": created_at, + "is_bot": is_bot_login(login), + } + + +def format_issue_context(issue: dict | None, comments: list[dict] | None = None) -> str: + if not issue: + return "No linked issue found." + + title = issue.get("title") or f"Issue #{issue.get('number')}" + body = issue.get("body") or "" + lines = [f"# {title}", ""] + if body: + lines.extend([body, ""]) + + human_comments = [ + comment for comment in (comments or []) if not comment.get("is_bot") + ] + if human_comments: + lines.extend(["## Comments", ""]) + for comment in human_comments: + author = comment.get("author") or "unknown" + created_at = comment.get("created_at") or "unknown-time" + lines.append(f"**{author}** ({created_at}):") + lines.append(comment.get("body", "")) + lines.append("") + + return "\n".join(lines).strip() + + +def summarize_comments( + inline_comments: list[dict], + reviews: list[dict], + issue_comments: list[dict], + linked_issue_comments: list[dict] | None = None, +) -> dict: + linked_issue_comments = linked_issue_comments or [] + + normalized_inline = [] + for comment in inline_comments: + login = login_for(comment) + normalized_inline.append( + { + "user": login, + "body": comment.get("body", ""), + "path": comment.get("path"), + "line": comment.get("line") or comment.get("original_line"), + "is_bot": is_bot_login(login), + "is_copilot": login in COPILOT_REVIEWERS, + } + ) + + normalized_reviews = [] + for review in reviews: + login = login_for(review) + normalized_reviews.append( + { + "user": login, + "body": review.get("body", ""), + "state": review.get("state"), + "is_bot": is_bot_login(login), + "is_copilot": login in COPILOT_REVIEWERS, + } + ) + + normalized_issue_comments = [] + for comment in issue_comments: + login = login_for(comment) + normalized_issue_comments.append( + { + "user": login, + "body": comment.get("body", ""), + "is_bot": is_bot_login(login), + "is_codecov": login == CODECOV_REVIEWER, + } + ) + + normalized_linked_issue_comments = [] + for comment in linked_issue_comments: + login = login_for(comment) + normalized_linked_issue_comments.append( + { + "user": login, + "body": comment.get("body", ""), + "is_bot": is_bot_login(login), + } + ) + + human_reviews = [ + review + for review in normalized_reviews + if not review["is_bot"] and review["body"].strip() + ] + codecov_comments = [ + comment for comment in normalized_issue_comments if comment["is_codecov"] + ] + + return { + "inline_comments": normalized_inline, + "reviews": normalized_reviews, + "issue_comments": normalized_issue_comments, + "linked_issue_comments": normalized_linked_issue_comments, + "human_inline_comments": [ + comment for comment in normalized_inline if not comment["is_bot"] + ], + "copilot_inline_comments": [ + comment for comment in normalized_inline if comment["is_copilot"] + ], + "human_reviews": human_reviews, + "human_issue_comments": [ + comment + for comment in normalized_issue_comments + if not comment["is_bot"] and not comment["is_codecov"] + ], + "human_linked_issue_comments": [ + comment + for comment in normalized_linked_issue_comments + if not comment["is_bot"] + ], + "codecov_comments": codecov_comments, + "counts": { + "inline_comments": len(normalized_inline), + "copilot_inline_comments": sum( + 1 for comment in normalized_inline if comment["is_copilot"] + ), + "human_inline_comments": sum( + 1 for comment in normalized_inline if not comment["is_bot"] + ), + "reviews": len(normalized_reviews), + "human_reviews": len(human_reviews), + "issue_comments": len(normalized_issue_comments), + "human_issue_comments": sum( + 1 + for comment in normalized_issue_comments + if not comment["is_bot"] and not comment["is_codecov"] + ), + "linked_issue_comments": len(normalized_linked_issue_comments), + "human_linked_issue_comments": sum( + 1 for comment in normalized_linked_issue_comments if not comment["is_bot"] + ), + "codecov_comments": len(codecov_comments), + }, + } + + +def extract_codecov_summary(issue_comments: list[dict]) -> dict: + codecov_comments = [ + comment for comment in issue_comments if login_for(comment) == CODECOV_REVIEWER + ] + if not codecov_comments: + return { + "found": False, + "body": None, + "patch_coverage": None, + "project_coverage": None, + "filepaths": [], + } + + body = codecov_comments[-1].get("body", "") + patch_match = _PATCH_COVERAGE_RE.search(body) + project_match = _PROJECT_COVERAGE_RE.search(body) + + filepaths: list[str] = [] + seen: set[str] = set() + for encoded in _FILEPATH_RE.findall(body): + path = unquote(encoded) + if path not in seen: + seen.add(path) + filepaths.append(path) + + return { + "found": True, + "body": body, + "patch_coverage": float(patch_match.group(1)) if patch_match else None, + "project_coverage": float(project_match.group(1)) if project_match else None, + "filepaths": filepaths, + } + + +def summarize_check_runs(payload: dict) -> dict: + runs = payload.get("check_runs") or [] + normalized_runs = [] + pending = 0 + failing = 0 + succeeding = 0 + + for run in runs: + status = (run.get("status") or "").lower() + conclusion = run.get("conclusion") + normalized_conclusion = conclusion.lower() if isinstance(conclusion, str) else None + normalized_runs.append( + { + "name": run.get("name"), + "status": status, + "conclusion": normalized_conclusion, + "details_url": run.get("details_url"), + } + ) + + if status != "completed" or normalized_conclusion is None: + pending += 1 + elif normalized_conclusion in {"success", "skipped", "neutral"}: + succeeding += 1 + else: + failing += 1 + + if failing: + state = "failure" + elif pending or not normalized_runs: + state = "pending" + else: + state = "success" + + return { + "state": state, + "total": len(normalized_runs), + "pending": pending, + "failing": failing, + "succeeding": succeeding, + "runs": normalized_runs, + } + + +def build_snapshot( + pr_data: dict, + *, + linked_issue_number: int | None = None, + linked_issue: dict | None = None, + ci_summary: dict | None = None, + codecov_summary: dict | None = None, +) -> dict: + if linked_issue_number is None: + linked_issue_number = extract_linked_issue_number( + pr_data.get("title"), + pr_data.get("body"), + ) + + labels = [label.get("name") for label in pr_data.get("labels", []) if label.get("name")] + files = [ + file_info.get("path") or file_info.get("filename") + for file_info in pr_data.get("files", []) + if file_info.get("path") or file_info.get("filename") + ] + commits = [ + commit.get("oid") or commit.get("commit", {}).get("oid") + for commit in pr_data.get("commits", []) + ] + + author_data = pr_data.get("author") or {} + return { + "number": pr_data.get("number"), + "title": pr_data.get("title"), + "body": pr_data.get("body"), + "state": pr_data.get("state"), + "url": pr_data.get("url"), + "mergeable": pr_data.get("mergeable"), + "author": author_data.get("login", ""), + "head_ref_name": pr_data.get("headRefName"), + "base_ref_name": pr_data.get("baseRefName"), + "head_sha": pr_data.get("headRefOid"), + "linked_issue_number": linked_issue_number, + "linked_issue": linked_issue, + "labels": labels, + "files": files, + "commits": commits, + "additions": pr_data.get("additions", 0), + "deletions": pr_data.get("deletions", 0), + "ci": ci_summary, + "codecov": codecov_summary, + "counts": { + "labels": len(labels), + "files": len(files), + "commits": len(commits), + }, + } + + +def build_current_pr_context(repo: str, pr_data: dict) -> dict: + return { + "repo": repo, + "pr_number": pr_data.get("number"), + "title": pr_data.get("title"), + "head_ref_name": pr_data.get("headRefName"), + "url": pr_data.get("url"), + } + + +def build_linked_issue_result( + *, + pr_number: int, + linked_issue_number: int | None, + linked_issue: dict | None, + linked_issue_comments: list[dict] | None = None, +) -> dict: + normalized_comments = [ + normalize_issue_thread_comment(comment) + for comment in (linked_issue_comments or []) + ] + human_comments = [ + comment for comment in normalized_comments if not comment["is_bot"] + ] + return { + "pr_number": pr_number, + "linked_issue_number": linked_issue_number, + "linked_issue": linked_issue, + "linked_issue_comments": normalized_comments, + "human_linked_issue_comments": human_comments, + "issue_context_text": format_issue_context(linked_issue, normalized_comments), + } + + +def build_linked_issue_context( + repo: str, + pr_number: int, + *, + linked_issue_number: int | None, + linked_issue: dict | None, +) -> dict: + linked_issue_comments = ( + fetch_issue_comments(repo, linked_issue_number) + if linked_issue_number is not None + else [] + ) + return build_linked_issue_result( + pr_number=pr_number, + linked_issue_number=linked_issue_number, + linked_issue=linked_issue, + linked_issue_comments=linked_issue_comments, + ) + + +def build_context_result( + repo: str, + snapshot: dict, + comments: dict, + linked_issue_result: dict, +) -> dict: + return { + "repo": repo, + "pr_number": snapshot.get("number"), + "title": snapshot.get("title"), + "body": snapshot.get("body"), + "state": snapshot.get("state"), + "url": snapshot.get("url"), + "mergeable": snapshot.get("mergeable"), + "head_ref_name": snapshot.get("head_ref_name"), + "base_ref_name": snapshot.get("base_ref_name"), + "head_sha": snapshot.get("head_sha"), + "files": snapshot.get("files", []), + "commits": snapshot.get("commits", []), + "linked_issue_number": linked_issue_result.get("linked_issue_number"), + "linked_issue": linked_issue_result.get("linked_issue"), + "linked_issue_comments": linked_issue_result.get("linked_issue_comments", []), + "human_linked_issue_comments": linked_issue_result.get( + "human_linked_issue_comments", [] + ), + "issue_context_text": linked_issue_result.get( + "issue_context_text", "No linked issue found." + ), + "comments": comments, + "ci": snapshot.get("ci"), + "codecov": snapshot.get("codecov"), + "snapshot": snapshot, + } + + +def build_pr_context(repo: str, pr_number: int) -> dict: + snapshot = build_pr_snapshot(repo, pr_number) + comments = build_comments_summary(repo, pr_number) + linked_issue_result = build_linked_issue_context( + repo, + pr_number, + linked_issue_number=snapshot.get("linked_issue_number"), + linked_issue=snapshot.get("linked_issue"), + ) + return build_context_result(repo, snapshot, comments, linked_issue_result) + + +def wait_for_ci( + fetcher: Callable[[], dict], + *, + timeout_seconds: float, + interval_seconds: float, + monotonic_fn: Callable[[], float] = time.monotonic, + sleep_fn: Callable[[float], None] = time.sleep, +) -> dict: + start = monotonic_fn() + attempts = 0 + + while True: + attempts += 1 + summary = dict(fetcher()) + summary.setdefault("state", "pending") + summary["attempts"] = attempts + summary["elapsed_seconds"] = round(monotonic_fn() - start, 3) + + if summary["state"] != "pending": + summary["timed_out"] = False + return summary + + if monotonic_fn() + interval_seconds > start + timeout_seconds: + summary["state"] = "timeout" + summary["timed_out"] = True + return summary + + sleep_fn(interval_seconds) + + +def fetch_pr_data(repo: str, pr_number: int) -> dict: + return run_gh_json( + "pr", + "view", + str(pr_number), + "--repo", + repo, + "--json", + ( + "number,title,body,labels,files,additions,deletions,commits," + "headRefName,baseRefName,headRefOid,url,state,mergeable,author" + ), + ) + + +def fetch_current_repo() -> str: + data = run_gh_json("repo", "view", "--json", "nameWithOwner") + repo = data.get("nameWithOwner") + if not repo: + raise ValueError(f"Unexpected repo payload: {data!r}") + return repo + + +def fetch_current_pr_data() -> dict: + return run_gh_json("pr", "view", "--json", "number,title,headRefName,url") + + +def fetch_current_pr_data_for_repo(repo: str) -> dict: + return run_gh_json( + "pr", + "view", + "--repo", + repo, + "--json", + "number,title,headRefName,url", + ) + + +def fetch_issue_data(repo: str, issue_number: int) -> dict: + return run_gh_json( + "issue", + "view", + str(issue_number), + "--repo", + repo, + "--json", + "number,title,body,labels,state,url", + ) + + +def fetch_issue_comments(repo: str, issue_number: int) -> list[dict]: + data = run_gh_json("api", f"repos/{repo}/issues/{issue_number}/comments") + if not isinstance(data, list): + raise ValueError(f"Unexpected issue comments payload for #{issue_number}: {data!r}") + return data + + +def fetch_inline_comments(repo: str, pr_number: int) -> list[dict]: + data = run_gh_json("api", f"repos/{repo}/pulls/{pr_number}/comments") + if not isinstance(data, list): + raise ValueError(f"Unexpected inline comments payload for PR #{pr_number}: {data!r}") + return data + + +def fetch_reviews(repo: str, pr_number: int) -> list[dict]: + data = run_gh_json("api", f"repos/{repo}/pulls/{pr_number}/reviews") + if not isinstance(data, list): + raise ValueError(f"Unexpected reviews payload for PR #{pr_number}: {data!r}") + return data + + +def fetch_check_runs(repo: str, head_sha: str) -> dict: + return run_gh_json("api", f"repos/{repo}/commits/{head_sha}/check-runs") + + +def fetch_linked_issue_bundle(repo: str, pr_data: dict) -> tuple[int | None, dict | None]: + issue_number = extract_linked_issue_number(pr_data.get("title"), pr_data.get("body")) + if issue_number is None: + return None, None + return issue_number, fetch_issue_data(repo, issue_number) + + +def fetch_ci_summary(repo: str, pr_number: int, pr_data: dict | None = None) -> dict: + pr_data = pr_data or fetch_pr_data(repo, pr_number) + head_sha = pr_data.get("headRefOid") + if not head_sha: + raise ValueError(f"PR #{pr_number} is missing headRefOid") + + summary = summarize_check_runs(fetch_check_runs(repo, head_sha)) + summary["pr_number"] = pr_number + summary["head_sha"] = head_sha + return summary + + +def build_comments_summary(repo: str, pr_number: int, pr_data: dict | None = None) -> dict: + pr_data = pr_data or fetch_pr_data(repo, pr_number) + linked_issue_number = extract_linked_issue_number( + pr_data.get("title"), + pr_data.get("body"), + ) + + summary = summarize_comments( + inline_comments=fetch_inline_comments(repo, pr_number), + reviews=fetch_reviews(repo, pr_number), + issue_comments=fetch_issue_comments(repo, pr_number), + linked_issue_comments=( + fetch_issue_comments(repo, linked_issue_number) + if linked_issue_number is not None + else [] + ), + ) + summary["pr_number"] = pr_number + summary["linked_issue_number"] = linked_issue_number + return summary + + +def build_codecov_summary(repo: str, pr_number: int) -> dict: + summary = extract_codecov_summary(fetch_issue_comments(repo, pr_number)) + summary["pr_number"] = pr_number + return summary + + +def build_pr_snapshot(repo: str, pr_number: int) -> dict: + pr_data = fetch_pr_data(repo, pr_number) + linked_issue_number, linked_issue = fetch_linked_issue_bundle(repo, pr_data) + return build_snapshot( + pr_data, + linked_issue_number=linked_issue_number, + linked_issue=linked_issue, + ci_summary=fetch_ci_summary(repo, pr_number, pr_data), + codecov_summary=build_codecov_summary(repo, pr_number), + ) + + +def create_pr( + repo: str, + title: str, + body_file: str, + *, + base: str | None = None, + head: str | None = None, +) -> dict: + args = [ + "pr", + "create", + "--repo", + repo, + "--title", + title, + "--body-file", + body_file, + ] + if base: + args.extend(["--base", base]) + if head: + args.extend(["--head", head]) + run_gh_checked(*args) + return build_current_pr_context(repo, fetch_current_pr_data_for_repo(repo)) + + +def post_pr_comment(repo: str, pr_number: int, body_file: str) -> None: + run_gh_checked( + "pr", + "comment", + str(pr_number), + "--repo", + repo, + "--body-file", + body_file, + ) + + +def edit_pr_body(repo: str, pr_number: int, body_file: str) -> None: + run_gh_checked( + "pr", + "edit", + str(pr_number), + "--repo", + repo, + "--body-file", + body_file, + ) + + +def render_context_text(result: dict) -> str: + comments = result.get("comments") or {} + counts = comments.get("counts") or {} + ci = result.get("ci") or {} + codecov = result.get("codecov") or {} + + lines = [ + "# PR Context Packet", + "", + "## Selection", + f"- Repo: {result.get('repo', '')}", + f"- PR: #{result.get('pr_number')}", + ] + if result.get("title"): + lines.append(f"- Title: {result['title']}") + if result.get("url"): + lines.append(f"- URL: {result['url']}") + if result.get("head_sha"): + lines.append(f"- Head SHA: `{result['head_sha']}`") + if result.get("linked_issue_number") is not None: + lines.append(f"- Linked issue: #{result['linked_issue_number']}") + + lines.extend( + [ + "", + "## Comment Summary", + f"- Copilot inline comments: {counts.get('copilot_inline_comments', 0)}", + f"- Human inline comments: {counts.get('human_inline_comments', 0)}", + f"- Human PR issue comments: {counts.get('human_issue_comments', 0)}", + f"- Human linked-issue comments: {counts.get('human_linked_issue_comments', 0)}", + f"- Human review bodies: {counts.get('human_reviews', 0)}", + ] + ) + + lines.extend( + [ + "", + "## CI Summary", + f"- State: {ci.get('state', 'unknown')}", + ] + ) + if ci: + lines.append(f"- Failing checks: {ci.get('failing', 0)}") + lines.append(f"- Pending checks: {ci.get('pending', 0)}") + lines.append(f"- Succeeding checks: {ci.get('succeeding', 0)}") + + lines.extend(["", "## Codecov"]) + if codecov.get("found"): + if codecov.get("patch_coverage") is not None: + lines.append(f"- Patch coverage: {codecov['patch_coverage']}%") + if codecov.get("project_coverage") is not None: + lines.append(f"- Project coverage: {codecov['project_coverage']}%") + filepaths = codecov.get("filepaths") or [] + if filepaths: + lines.append("- Referenced files:") + lines.extend(f" - `{path}`" for path in filepaths) + else: + lines.append("- No Codecov comment found") + + if result.get("issue_context_text"): + lines.extend( + [ + "", + "## Linked Issue Context", + result["issue_context_text"], + ] + ) + + return "\n".join(lines) + "\n" + + +def emit_result(result: dict, fmt: str) -> None: + if fmt == "json": + print(json.dumps(result, indent=2, sort_keys=True)) + return + + if "pr_number" in result and "comments" in result: + print(render_context_text(result), end="") + return + + if "number" in result: + print(f"PR #{result['number']}: {result.get('title', '')}") + return + + print(json.dumps(result, indent=2, sort_keys=True)) + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="PR automation helpers.") + subparsers = parser.add_subparsers(dest="command", required=True) + + context = subparsers.add_parser("context") + context.add_argument("--repo") + context.add_argument("--pr", type=int) + context.add_argument("--current", action="store_true") + context.add_argument("--format", choices=["json", "text"], default="json") + + for name in [ + "current", + "snapshot", + "comments", + "ci", + "wait-ci", + "codecov", + "linked-issue", + "create", + "comment", + "edit-body", + ]: + command = subparsers.add_parser(name) + if name == "current": + command.add_argument("--format", choices=["json", "text"], default="json") + else: + command.add_argument("--repo", required=True) + if name != "create": + command.add_argument("--pr", required=True, type=int) + if name == "wait-ci": + command.add_argument("--timeout", type=float, default=900) + command.add_argument("--interval", type=float, default=30) + elif name == "create": + command.add_argument("--title", required=True) + command.add_argument("--body-file", required=True) + command.add_argument("--base") + command.add_argument("--head") + command.add_argument("--format", choices=["json", "text"], default="json") + elif name in {"comment", "edit-body"}: + command.add_argument("--body-file", required=True) + elif name != "current": + command.add_argument("--format", choices=["json", "text"], default="json") + + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + + if args.command == "context": + if args.current: + repo = fetch_current_repo() + pr_number = fetch_current_pr_data()["number"] + else: + if not args.repo or args.pr is None: + raise ValueError("context requires --current or both --repo and --pr") + repo = args.repo + pr_number = args.pr + emit_result(build_pr_context(repo, pr_number), args.format) + return 0 + + if args.command == "current": + emit_result( + build_current_pr_context(fetch_current_repo(), fetch_current_pr_data()), + args.format, + ) + return 0 + + if args.command == "snapshot": + emit_result(build_pr_snapshot(args.repo, args.pr), args.format) + return 0 + + if args.command == "comments": + emit_result(build_comments_summary(args.repo, args.pr), args.format) + return 0 + + if args.command == "ci": + emit_result(fetch_ci_summary(args.repo, args.pr), args.format) + return 0 + + if args.command == "wait-ci": + result = wait_for_ci( + lambda: fetch_ci_summary(args.repo, args.pr), + timeout_seconds=args.timeout, + interval_seconds=args.interval, + ) + emit_result(result, args.format) + return 0 + + if args.command == "codecov": + emit_result(build_codecov_summary(args.repo, args.pr), args.format) + return 0 + + if args.command == "linked-issue": + pr_data = fetch_pr_data(args.repo, args.pr) + issue_number, issue = fetch_linked_issue_bundle(args.repo, pr_data) + issue_comments = ( + fetch_issue_comments(args.repo, issue_number) + if issue_number is not None + else [] + ) + emit_result( + build_linked_issue_result( + pr_number=args.pr, + linked_issue_number=issue_number, + linked_issue=issue, + linked_issue_comments=issue_comments, + ), + args.format, + ) + return 0 + + if args.command == "create": + emit_result( + create_pr( + args.repo, + args.title, + args.body_file, + base=args.base, + head=args.head, + ), + args.format, + ) + return 0 + + if args.command == "comment": + post_pr_comment(args.repo, args.pr, args.body_file) + return 0 + + if args.command == "edit-body": + edit_pr_body(args.repo, args.pr, args.body_file) + return 0 + + raise AssertionError(f"Unhandled command: {args.command}") + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/pipeline_skill_context.py b/scripts/pipeline_skill_context.py new file mode 100644 index 000000000..7fb7be1b4 --- /dev/null +++ b/scripts/pipeline_skill_context.py @@ -0,0 +1,1365 @@ +#!/usr/bin/env python3 +"""Skill-scoped context bundle CLI skeleton.""" + +from __future__ import annotations + +import argparse +import json +import subprocess +import sys +from pathlib import Path +from typing import Callable + +import pipeline_board +import pipeline_checks +import pipeline_pr +import pipeline_worktree + + +DEFAULT_STATE_FILES = { + "review-pipeline": Path("/tmp/problemreductions-review-state.json"), + "final-review": Path("/tmp/problemreductions-final-review-state.json"), +} +PROJECT_BOARD_NUMBER = 8 +PROJECT_BOARD_LIMIT = 500 +DEFAULT_REPO = "CodingThrust/problem-reductions" + + +def build_status_result(skill: str, *, status: str, **fields: object) -> dict: + result = { + "skill": skill, + "status": status, + } + for key, value in fields.items(): + if value is not None: + result[key] = value + return result + + +def report_check_status(check: dict | None) -> str: + if not check: + return "unknown" + if check.get("skipped"): + return "skipped" + return "pass" if check.get("ok") else "fail" + + +def first_paragraph(text: str | None) -> str: + if not text: + return "" + paragraphs = [chunk.strip() for chunk in text.split("\n\n") if chunk.strip()] + if not paragraphs: + return "" + return " ".join(paragraphs[0].split()) + + +def scan_existing_problems(repo_root: str | Path) -> set[str]: + problem_names: set[str] = set() + models_root = Path(repo_root) / "src/models" + if not models_root.exists(): + return problem_names + + for path in sorted(models_root.rglob("*.rs")): + text = path.read_text() + for match in pipeline_checks.re.finditer( + r"\bpub\s+(?:struct|enum)\s+([A-Z][A-Za-z0-9_]*)\b", + text, + ): + problem_names.add(match.group(1)) + return problem_names + + +def review_pipeline_suggested_mode(result: dict) -> str: + status = result.get("status") + if status == "empty": + return "empty" + if status == "needs-user-choice": + return "needs-user-choice" + + merge_status = ((result.get("prep") or {}).get("merge") or {}).get("status") + if merge_status == "conflicted": + return "conflicted-fix" + if merge_status == "aborted": + return "manual-followup" + + ci_state = ((result.get("pr") or {}).get("ci") or {}).get("state") + if ci_state == "failure": + return "fix-ci" + return "normal-fix" + + +def review_pipeline_seed_items(result: dict) -> list[str]: + blockers: list[str] = [] + prep = result.get("prep") or {} + merge_status = (prep.get("merge") or {}).get("status") + if merge_status == "conflicted": + blockers.append("merge conflicts with main") + elif merge_status == "aborted": + blockers.append("merge prep aborted") + + pr = result.get("pr") or {} + ci_state = (pr.get("ci") or {}).get("state") + if ci_state == "failure": + blockers.append("CI is failing") + + comment_counts = (pr.get("comments") or {}).get("counts") or {} + copilot_count = int(comment_counts.get("copilot_inline_comments", 0)) + if copilot_count: + blockers.append(f"{copilot_count} Copilot inline comments to triage") + + human_count = sum( + int(comment_counts.get(key, 0)) + for key in [ + "human_inline_comments", + "human_issue_comments", + "human_linked_issue_comments", + "human_reviews", + ] + ) + if human_count: + blockers.append(f"{human_count} human review items to audit") + + deduped: list[str] = [] + for blocker in blockers: + if blocker not in deduped: + deduped.append(blocker) + return deduped + + +def render_review_pipeline_text(result: dict) -> str: + lines = [ + "# Review Pipeline Packet", + "", + "## Selection", + f"- Bundle status: {result.get('status')}", + ] + + if result.get("status") == "empty": + lines.append("- No eligible review-pipeline item is currently available.") + return "\n".join(lines) + "\n" + + if result.get("status") == "needs-user-choice": + lines.extend( + [ + "", + "## Ambiguous PR Options", + ] + ) + for option in result.get("options") or []: + lines.append( + f"- PR #{option.get('number')} [{option.get('state', 'UNKNOWN')}] {option.get('title') or ''}".rstrip() + ) + if result.get("recommendation") is not None: + lines.append(f"- Recommended PR: #{result['recommendation']}") + return "\n".join(lines) + "\n" + + selection = result.get("selection") or {} + pr = result.get("pr") or {} + prep = result.get("prep") or {} + comments = pr.get("comments") or {} + counts = comments.get("counts") or {} + ci = pr.get("ci") or {} + codecov = pr.get("codecov") or {} + checkout = prep.get("checkout") or {} + merge = prep.get("merge") or {} + + if selection.get("pr_number") is not None: + lines.append(f"- PR: #{selection['pr_number']}") + if selection.get("item_id"): + lines.append(f"- Board item: `{selection['item_id']}`") + if selection.get("issue_number") is not None: + lines.append(f"- Linked issue: #{selection['issue_number']}") + if pr.get("title") or selection.get("title"): + lines.append(f"- Title: {pr.get('title') or selection.get('title')}") + if pr.get("url"): + lines.append(f"- URL: {pr['url']}") + + lines.extend( + [ + "", + "## Recommendation Seed", + f"- Suggested mode: {review_pipeline_suggested_mode(result)}", + ] + ) + seed_items = review_pipeline_seed_items(result) + if seed_items: + lines.append("- Attention points:") + lines.extend(f" - {item}" for item in seed_items) + else: + lines.append("- Attention points: none from deterministic checks") + + lines.extend( + [ + "", + "## Comment Summary", + f"- Copilot inline comments: {counts.get('copilot_inline_comments', 0)}", + f"- Human inline comments: {counts.get('human_inline_comments', 0)}", + f"- Human PR issue comments: {counts.get('human_issue_comments', 0)}", + f"- Human linked-issue comments: {counts.get('human_linked_issue_comments', 0)}", + f"- Human review bodies: {counts.get('human_reviews', 0)}", + ] + ) + + lines.extend( + [ + "", + "## CI / Coverage", + f"- CI state: {ci.get('state', 'unknown')}", + ] + ) + if ci: + lines.append(f"- Failing checks: {ci.get('failing', 0)}") + lines.append(f"- Pending checks: {ci.get('pending', 0)}") + if codecov.get("found"): + lines.append(f"- Patch coverage: {codecov.get('patch_coverage')}%") + if codecov.get("project_coverage") is not None: + lines.append(f"- Project coverage: {codecov.get('project_coverage')}%") + + lines.extend( + [ + "", + "## Merge Prep", + f"- Ready: {str(prep.get('ready')).lower()}", + f"- Merge status: {merge.get('status', 'unknown')}", + ] + ) + if checkout.get("worktree_dir"): + lines.append(f"- Worktree: `{checkout['worktree_dir']}`") + conflicts = merge.get("conflicts") or [] + if conflicts: + lines.append("- Conflicts:") + lines.extend(f" - `{conflict}`" for conflict in conflicts) + + if pr.get("issue_context_text"): + lines.extend( + [ + "", + "## Linked Issue Context", + pr["issue_context_text"], + ] + ) + + return "\n".join(lines) + "\n" + + +def final_review_suggested_mode(result: dict) -> str: + status = result.get("status") + if status == "empty": + return "empty" + if status == "ready-with-warnings": + return "warning-fallback" + + merge_status = ((result.get("prep") or {}).get("merge") or {}).get("status") + if merge_status == "conflicted": + return "conflicted-review" + if merge_status == "aborted": + return "warning-fallback" + return "normal-review" + + +def final_review_seed_items(result: dict) -> list[str]: + review_context = result.get("review_context") or {} + prep = result.get("prep") or {} + warnings = list(result.get("warnings") or []) + blockers = list(warnings) + + merge_status = (prep.get("merge") or {}).get("status") + if merge_status == "conflicted": + blockers.append("merge conflicts with main") + elif merge_status == "aborted": + blockers.append("merge prep aborted") + + whitelist = review_context.get("whitelist") or {} + if whitelist and not whitelist.get("ok"): + blockers.append("files outside expected whitelist") + + completeness = review_context.get("completeness") or {} + for missing in completeness.get("missing", []): + blockers.append(f"missing completeness item: {missing}") + + comment_counts = ((result.get("pr") or {}).get("comments") or {}).get("counts") or {} + manual_comment_count = sum( + int(comment_counts.get(key, 0)) + for key in [ + "human_inline_comments", + "human_issue_comments", + "human_linked_issue_comments", + "human_reviews", + ] + ) + if manual_comment_count: + blockers.append( + f"manual comment audit required for {manual_comment_count} human review items" + ) + + deduped: list[str] = [] + for blocker in blockers: + if blocker not in deduped: + deduped.append(blocker) + return deduped + + +def render_final_review_text(result: dict) -> str: + selection = result.get("selection") or {} + pr = result.get("pr") or {} + prep = result.get("prep") or {} + review_context = result.get("review_context") or {} + subject = review_context.get("subject") or {} + comments = pr.get("comments") or {} + counts = comments.get("counts") or {} + checkout = prep.get("checkout") or {} + merge = prep.get("merge") or {} + + lines = [ + "# Final Review Packet", + "", + "## Selection", + f"- Bundle status: {result.get('status')}", + ] + if selection.get("pr_number") is not None: + lines.append(f"- PR: #{selection['pr_number']}") + if selection.get("item_id"): + lines.append(f"- Board item: `{selection['item_id']}`") + if selection.get("issue_number") is not None: + lines.append(f"- Linked issue: #{selection['issue_number']}") + if pr.get("title") or selection.get("title"): + lines.append(f"- Title: {pr.get('title') or selection.get('title')}") + if pr.get("url"): + lines.append(f"- URL: {pr['url']}") + + lines.extend( + [ + "", + "## Recommendation Seed", + f"- Suggested mode: {final_review_suggested_mode(result)}", + ] + ) + seed_items = final_review_seed_items(result) + if seed_items: + lines.append("- Review blockers / attention points:") + lines.extend(f" - {item}" for item in seed_items) + else: + lines.append("- Review blockers / attention points: none from deterministic checks") + + lines.extend( + [ + "", + "## Subject", + f"- Kind: {subject.get('kind', 'unknown')}", + ] + ) + if subject.get("name"): + lines.append(f"- Name: {subject['name']}") + if subject.get("source"): + lines.append(f"- Source: {subject['source']}") + if subject.get("target"): + lines.append(f"- Target: {subject['target']}") + + lines.extend( + [ + "", + "## Comment Summary", + f"- Human reviews: {counts.get('human_reviews', 0)}", + f"- Human inline comments: {counts.get('human_inline_comments', 0)}", + f"- Human PR issue comments: {counts.get('human_issue_comments', 0)}", + f"- Human linked-issue comments: {counts.get('human_linked_issue_comments', 0)}", + ] + ) + if pr.get("issue_context_text"): + lines.extend( + [ + "", + "### Linked Issue Context", + pr["issue_context_text"], + ] + ) + + lines.extend( + [ + "", + "## Merge Prep", + f"- Ready: {str(prep.get('ready')).lower()}", + f"- Merge status: {merge.get('status', 'unknown')}", + ] + ) + if checkout.get("worktree_dir"): + lines.append(f"- Worktree: `{checkout['worktree_dir']}`") + conflicts = merge.get("conflicts") or [] + if conflicts: + lines.append("- Conflicts:") + lines.extend(f" - `{conflict}`" for conflict in conflicts) + warnings = result.get("warnings") or [] + if warnings: + lines.append("- Warnings:") + lines.extend(f" - {warning}" for warning in warnings) + + lines.extend( + [ + "", + "## Deterministic Checks", + f"- Whitelist: {report_check_status(review_context.get('whitelist'))}", + f"- Completeness: {report_check_status(review_context.get('completeness'))}", + ] + ) + missing = (review_context.get("completeness") or {}).get("missing") or [] + if missing: + lines.append("- Missing items:") + lines.extend(f" - `{item}`" for item in missing) + + changed_files = review_context.get("changed_files") or [] + lines.extend(["", "## Changed Files"]) + if changed_files: + lines.extend(f"- `{path}`" for path in changed_files) + else: + lines.append("- None captured") + + diff_stat = review_context.get("diff_stat") + if diff_stat: + lines.extend(["", "## Diff Stat", "```text", diff_stat, "```"]) + + full_diff = review_context.get("full_diff") + if full_diff: + lines.extend(["", "## Full Diff", "```diff", full_diff, "```"]) + + pred_list = review_context.get("pred_list") + if pred_list: + lines.extend(["", "## Problem Catalog (`pred list`)", "```text", pred_list, "```"]) + + return "\n".join(lines) + "\n" + + +def render_review_implementation_text(result: dict) -> str: + git = result.get("git") or {} + review_context = result.get("review_context") or {} + scope = review_context.get("scope") or {} + subject = review_context.get("subject") or {} + current_pr = result.get("current_pr") or {} + + lines = [ + "# Review Implementation Packet", + "", + "## Review Range", + f"- Base SHA: `{git.get('base_sha', '')}`", + f"- Head SHA: `{git.get('head_sha', '')}`", + f"- Repo root: `{git.get('repo_root', '')}`", + "", + "## Scope", + f"- Review type: {scope.get('review_type', 'unknown')}", + f"- Subject kind: {subject.get('kind', 'unknown')}", + ] + if subject.get("name"): + lines.append(f"- Name: {subject['name']}") + if subject.get("source"): + lines.append(f"- Source: {subject['source']}") + if subject.get("target"): + lines.append(f"- Target: {subject['target']}") + + models = scope.get("models") or [] + if models: + lines.append("- Added models:") + lines.extend( + f" - {model.get('problem_name')} (`{model.get('path')}`)" + for model in models + ) + rules = scope.get("rules") or [] + if rules: + lines.append("- Added rules:") + lines.extend( + f" - {rule.get('rule_stem')} (`{rule.get('path')}`)" + for rule in rules + ) + + lines.extend( + [ + "", + "## Deterministic Checks", + f"- Whitelist: {report_check_status(review_context.get('whitelist'))}", + f"- Completeness: {report_check_status(review_context.get('completeness'))}", + ] + ) + missing = (review_context.get("completeness") or {}).get("missing") or [] + if missing: + lines.append("- Missing items:") + lines.extend(f" - `{item}`" for item in missing) + + changed_files = review_context.get("changed_files") or [] + lines.extend(["", "## Changed Files"]) + if changed_files: + lines.extend(f"- `{path}`" for path in changed_files) + else: + lines.append("- None captured") + + diff_stat = review_context.get("diff_stat") + if diff_stat: + lines.extend(["", "## Diff Stat", "```text", diff_stat, "```"]) + + lines.extend(["", "## Current PR"]) + if current_pr: + lines.append(f"- Repo: {current_pr.get('repo')}") + if current_pr.get("pr_number") is not None: + lines.append(f"- PR: #{current_pr['pr_number']}") + if current_pr.get("title"): + lines.append(f"- Title: {current_pr['title']}") + if current_pr.get("url"): + lines.append(f"- URL: {current_pr['url']}") + if current_pr.get("linked_issue_number") is not None: + lines.append(f"- Linked issue: #{current_pr['linked_issue_number']}") + else: + lines.append("- No current PR detected for this branch.") + + issue_context_text = current_pr.get("issue_context_text") + if issue_context_text: + lines.extend(["", "## Linked Issue Context", issue_context_text]) + + return "\n".join(lines) + "\n" + + +def render_project_pipeline_text(result: dict) -> str: + ready_issues = result.get("ready_issues") or [] + eligible = [issue for issue in ready_issues if issue.get("eligible")] + blocked = [issue for issue in ready_issues if not issue.get("eligible")] + in_progress = result.get("in_progress_issues") or [] + requested = result.get("requested_issue") + + lines = [ + "# Project Pipeline Packet", + "", + "## Queue Summary", + f"- Bundle status: {result.get('status')}", + f"- Ready issues: {len(ready_issues)}", + f"- Eligible ready issues: {len(eligible)}", + f"- Blocked ready issues: {len(blocked)}", + f"- In progress issues: {len(in_progress)}", + f"- Existing problems on main: {len(result.get('existing_problems') or [])}", + ] + + if requested is not None: + lines.extend( + [ + "", + "## Requested Issue", + f"- Issue: #{requested.get('issue_number')}", + f"- Title: {requested.get('title') or 'unknown'}", + f"- Eligible: {str(bool(requested.get('eligible'))).lower()}", + ] + ) + if requested.get("blocking_reason"): + lines.append(f"- Blocking reason: {requested['blocking_reason']}") + + lines.extend(["", "## Eligible Ready Issues"]) + if eligible: + for issue in eligible: + lines.append(f"- #{issue.get('issue_number')} {issue.get('title')}") + lines.append(f" - Kind: {issue.get('kind', 'unknown')}") + lines.append( + f" - Pending rules unblocked: {issue.get('pending_rule_count', 0)}" + ) + if issue.get("summary"): + lines.append(f" - Summary: {issue['summary']}") + else: + lines.append("- None") + + lines.extend(["", "## Blocked Ready Issues"]) + if blocked: + for issue in blocked: + lines.append(f"- #{issue.get('issue_number')} {issue.get('title')}") + lines.append(f" - Blocking reason: {issue.get('blocking_reason')}") + if issue.get("summary"): + lines.append(f" - Summary: {issue['summary']}") + else: + lines.append("- None") + + if in_progress: + lines.extend(["", "## In Progress Issues"]) + for issue in in_progress: + lines.append(f"- #{issue.get('issue_number')} {issue.get('title')}") + + return "\n".join(lines) + "\n" + + +def render_text(result: dict) -> str: + if result.get("skill") == "review-pipeline": + return render_review_pipeline_text(result) + if result.get("skill") == "final-review": + return render_final_review_text(result) + if result.get("skill") == "review-implementation": + return render_review_implementation_text(result) + if result.get("skill") == "project-pipeline": + return render_project_pipeline_text(result) + return json.dumps(result, indent=2, sort_keys=True) + "\n" + + +def emit_result(result: dict, fmt: str) -> None: + if fmt == "text": + print(render_text(result), end="") + return + print(json.dumps(result, indent=2, sort_keys=True)) + + +def fetch_review_candidates(repo: str) -> list[dict]: + owner = repo.split("/", 1)[0] + board_data = pipeline_board.fetch_board_items( + owner, + PROJECT_BOARD_NUMBER, + PROJECT_BOARD_LIMIT, + ) + return pipeline_board.review_candidates( + board_data, + repo, + pipeline_board.fetch_pr_reviews, + pipeline_board.resolve_issue_pr, + pipeline_board.fetch_pr_info, + ) + + +def claim_review_entry( + *, + repo: str, + state_file: Path, + pr_number: int | None, +) -> dict | None: + owner = repo.split("/", 1)[0] + board_data = pipeline_board.fetch_board_items( + owner, + PROJECT_BOARD_NUMBER, + PROJECT_BOARD_LIMIT, + ) + return pipeline_board.claim_next_entry( + "review", + board_data, + state_file, + repo=repo, + review_fetcher=pipeline_board.fetch_pr_reviews, + pr_resolver=pipeline_board.resolve_issue_pr, + pr_state_fetcher=pipeline_board.fetch_pr_state, + target_number=pr_number, + ) + + +def build_ready_result(*, skill: str, selection: dict, pr: dict, prep: dict) -> dict: + return build_status_result( + skill, + status="ready", + selection=selection, + pr=pr, + prep=prep, + ) + + +def build_ambiguous_selection(candidate: dict, *, pr_number: int) -> dict: + return { + "item_id": candidate["item_id"], + "number": pr_number, + "issue_number": candidate.get("issue_number"), + "pr_number": pr_number, + "status": candidate.get("status"), + "title": candidate.get("title"), + "claimed": True, + "claimed_status": pipeline_board.STATUS_UNDER_REVIEW, + } + + +def select_final_review_entry( + *, + repo: str, + state_file: Path, + pr_number: int | None, +) -> dict | None: + owner = repo.split("/", 1)[0] + board_data = pipeline_board.fetch_board_items( + owner, + PROJECT_BOARD_NUMBER, + PROJECT_BOARD_LIMIT, + ) + return pipeline_board.select_next_entry( + "final-review", + board_data, + state_file, + repo=repo, + pr_resolver=pipeline_board.resolve_issue_pr, + pr_state_fetcher=pipeline_board.fetch_pr_state, + target_number=pr_number, + ) + + +def _get_current_gh_user() -> str: + """Return the GitHub login of the currently authenticated user.""" + try: + output = subprocess.check_output( + ["gh", "api", "user", "--jq", ".login"], + text=True, + stderr=subprocess.DEVNULL, + ) + return output.strip() + except Exception: + return "" + + +def git_output_in(repo_root: str | Path, *args: str) -> list[str]: + output = subprocess.check_output( + ["git", "-C", str(repo_root), *args], + text=True, + ) + return [line for line in output.splitlines() if line] + + +def git_text_in(repo_root: str | Path, *args: str) -> str: + return subprocess.check_output( + ["git", "-C", str(repo_root), *args], + text=True, + ) + + +def infer_final_review_subject(scope: dict, pr_context: dict) -> dict: + subject = pipeline_checks.infer_review_subject(scope) + linked_issue = pr_context.get("linked_issue") or {} + linked_title = (linked_issue.get("title") or "").strip() + + rule_match = pipeline_checks.RULE_TITLE_RE.match(linked_title) + if rule_match: + subject["kind"] = "rule" + subject["source"] = rule_match.group("source") + subject["target"] = rule_match.group("target") + return subject + + model_match = pipeline_checks.MODEL_TITLE_RE.match(linked_title) + if model_match: + subject["kind"] = "model" + subject["name"] = subject.get("name") or model_match.group("name") + subject["source"] = None + subject["target"] = None + + return subject + + +def build_final_review_checks(*, prep: dict, pr_context: dict) -> dict: + checkout = prep.get("checkout") or {} + worktree_dir = checkout.get("worktree_dir") + base_sha = checkout.get("base_sha") + head_sha = checkout.get("head_sha") + if not worktree_dir or not base_sha or not head_sha: + raise ValueError("prepare-review output missing checkout diff range") + + diff_range = f"{base_sha}..{head_sha}" + changed_files = git_output_in(worktree_dir, "diff", "--name-only", diff_range) + added_files = git_output_in( + worktree_dir, + "diff", + "--name-only", + "--diff-filter=A", + diff_range, + ) + scope = pipeline_checks.detect_scope_from_paths( + added_files=added_files, + changed_files=changed_files, + ) + subject = infer_final_review_subject(scope, pr_context) + review_context = pipeline_checks.build_review_context( + worktree_dir, + diff_stat=git_text_in(worktree_dir, "diff", "--stat", diff_range), + scope=scope, + subject=subject, + ) + review_context["full_diff"] = git_text_in(worktree_dir, "diff", diff_range) + review_context["pred_list"] = _run_pred_list(worktree_dir) + return review_context + + +def _run_pred_list(worktree_dir: str | Path) -> str | None: + """Run ``pred list`` in *worktree_dir*, building the CLI first if needed.""" + pred_cmd = ["cargo", "run", "-p", "problemreductions-cli", "--bin", "pred", "--", "list"] + try: + return subprocess.check_output( + pred_cmd, cwd=str(worktree_dir), text=True, stderr=subprocess.DEVNULL, + ) + except (subprocess.CalledProcessError, FileNotFoundError): + pass + # Binary may not exist yet — build it and retry. + try: + subprocess.check_call( + ["make", "cli"], cwd=str(worktree_dir), + stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + return subprocess.check_output( + pred_cmd, cwd=str(worktree_dir), text=True, stderr=subprocess.DEVNULL, + ) + except Exception: + return None + + +def default_review_implementation_context_builder( + repo_root: str | Path, + *, + diff_stat: str, + changed_files: list[str], + added_files: list[str], + kind: str | None, + name: str | None, + source: str | None, + target: str | None, +) -> dict: + scope = pipeline_checks.detect_scope_from_paths( + added_files=added_files, + changed_files=changed_files, + ) + subject = pipeline_checks.infer_review_subject( + scope, + kind=kind, + name=name, + source=source, + target=target, + ) + return pipeline_checks.build_review_context( + repo_root, + diff_stat=diff_stat, + scope=scope, + subject=subject, + ) + + +def fetch_current_review_implementation_pr() -> dict | None: + try: + repo = pipeline_pr.fetch_current_repo() + current = pipeline_pr.fetch_current_pr_data_for_repo(repo) + pr_number = current.get("number") + if pr_number is None: + return None + pr_context = pipeline_pr.build_pr_context(repo, int(pr_number)) + return { + "repo": repo, + "pr_number": int(pr_number), + "title": pr_context.get("title"), + "url": pr_context.get("url"), + "head_ref_name": pr_context.get("head_ref_name"), + "linked_issue_number": pr_context.get("linked_issue_number"), + "issue_context_text": pr_context.get("issue_context_text"), + } + except Exception: + return None + + +def build_review_implementation_context( + *, + repo_root: Path, + kind: str | None, + name: str | None, + source: str | None, + target: str | None, + merge_base_getter: Callable[[Path], str] | None = None, + head_sha_getter: Callable[[Path], str] | None = None, + diff_stat_getter: Callable[[Path, str, str], str] | None = None, + changed_files_getter: Callable[[Path, str, str], list[str]] | None = None, + added_files_getter: Callable[[Path, str, str], list[str]] | None = None, + current_pr_fetcher: Callable[[], dict | None] | None = None, + review_context_builder: Callable[..., dict] | None = None, +) -> dict: + merge_base_getter = merge_base_getter or ( + lambda repo_root: git_text_in(repo_root, "merge-base", "main", "HEAD").strip() + ) + head_sha_getter = head_sha_getter or ( + lambda repo_root: git_text_in(repo_root, "rev-parse", "HEAD").strip() + ) + diff_stat_getter = diff_stat_getter or ( + lambda repo_root, base_sha, head_sha: git_text_in( + repo_root, + "diff", + "--stat", + f"{base_sha}..{head_sha}", + ) + ) + changed_files_getter = changed_files_getter or ( + lambda repo_root, base_sha, head_sha: git_output_in( + repo_root, + "diff", + "--name-only", + f"{base_sha}..{head_sha}", + ) + ) + added_files_getter = added_files_getter or ( + lambda repo_root, base_sha, head_sha: git_output_in( + repo_root, + "diff", + "--name-only", + "--diff-filter=A", + f"{base_sha}..{head_sha}", + ) + ) + current_pr_fetcher = current_pr_fetcher or fetch_current_review_implementation_pr + review_context_builder = review_context_builder or default_review_implementation_context_builder + + base_sha = merge_base_getter(repo_root) + head_sha = head_sha_getter(repo_root) + diff_stat = diff_stat_getter(repo_root, base_sha, head_sha) + changed_files = changed_files_getter(repo_root, base_sha, head_sha) + added_files = added_files_getter(repo_root, base_sha, head_sha) + current_pr = current_pr_fetcher() + review_context = review_context_builder( + repo_root, + diff_stat=diff_stat, + changed_files=changed_files, + added_files=added_files, + kind=kind, + name=name, + source=source, + target=target, + ) + + return { + "skill": "review-implementation", + "status": "ready", + "git": { + "repo_root": str(repo_root), + "base_sha": base_sha, + "head_sha": head_sha, + }, + "review_context": review_context, + "current_pr": current_pr, + } + + +def classify_project_issue( + entry: dict, + *, + issue: dict, + existing_problems: set[str], + pending_rule_counts: dict[str, int], +) -> dict: + kind, source_problem, target_problem = pipeline_checks.issue_kind_from_title( + entry.get("title") + ) + blocking_reason = None + eligible = True + if kind == "rule": + missing = [ + problem + for problem in [source_problem, target_problem] + if problem and problem not in existing_problems + ] + if missing: + eligible = False + blocking_reason = f'model "{missing[0]}" not yet implemented on main' + + issue_number = int(entry["issue_number"]) + return { + "item_id": entry.get("item_id"), + "issue_number": issue_number, + "title": entry.get("title"), + "kind": kind, + "source_problem": source_problem, + "target_problem": target_problem, + "eligible": eligible, + "blocking_reason": blocking_reason, + "pending_rule_count": pending_rule_counts.get(entry.get("title", ""), 0) + if kind == "rule" + else pending_rule_counts.get( + pipeline_checks.MODEL_TITLE_RE.match(entry.get("title", "")).group("name") + if pipeline_checks.MODEL_TITLE_RE.match(entry.get("title", "")) + else "", + 0, + ), + "summary": first_paragraph(issue.get("body")), + "issue": issue, + } + + +def build_pending_rule_counts( + ready_entries: list[dict], + in_progress_entries: list[dict], +) -> dict[str, int]: + counts: dict[str, int] = {} + for entry in [*ready_entries, *in_progress_entries]: + kind, source_problem, target_problem = pipeline_checks.issue_kind_from_title( + entry.get("title") + ) + if kind != "rule": + continue + for problem in [source_problem, target_problem]: + if not problem: + continue + counts[problem] = counts.get(problem, 0) + 1 + return counts + + +def fetch_project_board_data(repo: str) -> dict: + owner = repo.split("/", 1)[0] + return pipeline_board.fetch_board_items( + owner, + PROJECT_BOARD_NUMBER, + PROJECT_BOARD_LIMIT, + ) + + +def build_project_pipeline_context( + *, + repo: str, + issue_number: int | None, + repo_root: Path, + board_fetcher: Callable[[str], dict] | None = None, + issue_fetcher: Callable[[str, int], dict] | None = None, + existing_problem_finder: Callable[[Path], set[str]] | None = None, +) -> dict: + board_fetcher = board_fetcher or fetch_project_board_data + issue_fetcher = issue_fetcher or pipeline_checks.fetch_issue + existing_problem_finder = existing_problem_finder or scan_existing_problems + + board_data = board_fetcher(repo) + ready_entries = sorted( + pipeline_board.ready_entries(board_data).values(), + key=lambda entry: entry["issue_number"], + ) + in_progress_entries = pipeline_board.status_items( + board_data, + pipeline_board.STATUS_IN_PROGRESS, + ) + existing_problems = existing_problem_finder(repo_root) + pending_rule_counts = build_pending_rule_counts(ready_entries, in_progress_entries) + + ready_issues = [ + classify_project_issue( + dict(entry, item_id=item_id), + issue=issue_fetcher(repo, int(entry["issue_number"])), + existing_problems=existing_problems, + pending_rule_counts=pending_rule_counts, + ) + for item_id, entry in sorted( + pipeline_board.ready_entries(board_data).items(), + key=lambda pair: pair[1]["issue_number"], + ) + ] + + requested_issue = None + if issue_number is not None: + requested_issue = next( + ( + issue + for issue in ready_issues + if int(issue["issue_number"]) == issue_number + ), + None, + ) + + eligible_ready_issues = [issue for issue in ready_issues if issue.get("eligible")] + + if not ready_issues: + status = "empty" + elif issue_number is not None and requested_issue is None: + status = "requested-missing" + elif requested_issue is not None and not requested_issue.get("eligible"): + status = "requested-blocked" + elif not eligible_ready_issues: + status = "no-eligible-issues" + else: + status = "ready" + + return build_status_result( + "project-pipeline", + status=status, + repo=repo, + existing_problems=sorted(existing_problems), + ready_issues=ready_issues, + in_progress_issues=in_progress_entries, + requested_issue=requested_issue, + ) + + +def build_review_pipeline_context( + *, + repo: str, + pr_number: int | None, + state_file: Path, + review_candidate_fetcher: Callable[[str], list[dict]] | None = None, + claim_entry: Callable[..., dict | None] | None = None, + pr_context_builder: Callable[[str, int], dict] | None = None, + review_preparer: Callable[[str, int], dict] | None = None, + mover: Callable[[str, str], None] | None = None, +) -> dict: + review_candidate_fetcher = review_candidate_fetcher or fetch_review_candidates + claim_entry = claim_entry or claim_review_entry + pr_context_builder = pr_context_builder or pipeline_pr.build_pr_context + review_preparer = review_preparer or ( + lambda repo, pr_number: pipeline_worktree.prepare_review( + repo=repo, + pr_number=pr_number, + ) + ) + mover = mover or pipeline_board.move_item + + candidates = review_candidate_fetcher(repo) + if not candidates: + return build_status_result("review-pipeline", status="empty") + + if pr_number is None: + ambiguous = next( + ( + candidate + for candidate in candidates + if candidate.get("eligibility") == "ambiguous-linked-prs" + ), + None, + ) + if ambiguous is not None: + return build_status_result( + "review-pipeline", + status="needs-user-choice", + options=ambiguous.get("linked_repo_prs", []), + recommendation=ambiguous.get("recommendation"), + ) + + selection = claim_entry( + repo=repo, + state_file=state_file, + pr_number=None, + ) + if selection is None: + return build_status_result("review-pipeline", status="empty") + + selected_pr_number = int(selection["pr_number"]) + return build_ready_result( + skill="review-pipeline", + selection=selection, + pr=pr_context_builder(repo, selected_pr_number), + prep=review_preparer(repo, selected_pr_number), + ) + + matching_ambiguous = next( + ( + candidate + for candidate in candidates + if candidate.get("eligibility") == "ambiguous-linked-prs" + and any( + int(option["number"]) == pr_number + for option in candidate.get("linked_repo_prs", []) + ) + ), + None, + ) + if matching_ambiguous is not None: + mover(str(matching_ambiguous["item_id"]), pipeline_board.STATUS_UNDER_REVIEW) + selection = build_ambiguous_selection( + matching_ambiguous, + pr_number=pr_number, + ) + return build_ready_result( + skill="review-pipeline", + selection=selection, + pr=pr_context_builder(repo, pr_number), + prep=review_preparer(repo, pr_number), + ) + + matching_candidate = next( + ( + candidate + for candidate in candidates + if int(candidate.get("pr_number") or candidate.get("number") or -1) == pr_number + ), + None, + ) + if matching_candidate is None: + return build_status_result("review-pipeline", status="empty") + + if matching_candidate.get("eligibility") != "eligible": + return build_status_result("review-pipeline", status="empty") + + selection = claim_entry( + repo=repo, + state_file=state_file, + pr_number=pr_number, + ) + if selection is None: + return build_status_result("review-pipeline", status="empty") + + return build_ready_result( + skill="review-pipeline", + selection=selection, + pr=pr_context_builder(repo, pr_number), + prep=review_preparer(repo, pr_number), + ) + + +def build_final_review_context( + *, + repo: str, + pr_number: int | None, + state_file: Path, + selection_fetcher: Callable[..., dict | None] | None = None, + pr_context_builder: Callable[[str, int], dict] | None = None, + review_preparer: Callable[[str, int], dict] | None = None, + review_context_builder: Callable[..., dict] | None = None, +) -> dict: + selection_fetcher = selection_fetcher or select_final_review_entry + pr_context_builder = pr_context_builder or pipeline_pr.build_pr_context + review_preparer = review_preparer or ( + lambda repo, pr_number: pipeline_worktree.prepare_review( + repo=repo, + pr_number=pr_number, + ) + ) + review_context_builder = review_context_builder or build_final_review_checks + + selection = selection_fetcher( + repo=repo, + state_file=state_file, + pr_number=pr_number, + ) + if selection is None: + return build_status_result("final-review", status="empty") + + selected_pr_number = int(selection.get("pr_number") or selection["number"]) + pr_context = pr_context_builder(repo, selected_pr_number) + + # Self-review warning: flag if reviewer is the PR author (unless repo owner). + pr_author = (pr_context.get("author") or "").lower() + current_user = _get_current_gh_user().lower() + repo_owner = repo.split("/", 1)[0].lower() if "/" in repo else "" + self_review_warning = None + if pr_author and current_user and pr_author == current_user and current_user != repo_owner: + self_review_warning = f"Self-review: PR author '{pr_author}' is the current reviewer" + + prep: dict + try: + prep = review_preparer(repo, selected_pr_number) + except Exception as exc: + return { + "skill": "final-review", + "status": "ready-with-warnings", + "selection": selection, + "pr": pr_context, + "prep": { + "ready": False, + "error": str(exc), + }, + "review_context": None, + "warnings": [ + f"failed to prepare final-review worktree: {exc}", + ], + } + + try: + review_context = review_context_builder( + prep=prep, + pr_context=pr_context, + ) + except Exception as exc: + return { + "skill": "final-review", + "status": "ready-with-warnings", + "selection": selection, + "pr": pr_context, + "prep": prep, + "review_context": None, + "warnings": [ + f"failed to derive final-review review context: {exc}", + ], + } + + warnings = [self_review_warning] if self_review_warning else [] + return build_status_result( + "final-review", + status="ready", + selection=selection, + pr=pr_context, + prep=prep, + review_context=review_context, + warnings=warnings or None, + ) + + +def add_bundle_parser( + subparsers, + command: str, +) -> None: + parser = subparsers.add_parser(command) + parser.add_argument("--repo", required=True) + parser.add_argument("--pr", type=int) + parser.add_argument( + "--state-file", + type=Path, + default=DEFAULT_STATE_FILES[command], + ) + parser.add_argument("--format", choices=["json", "text"], default="json") + + +def add_review_implementation_parser(subparsers) -> None: + parser = subparsers.add_parser("review-implementation") + parser.add_argument("--repo-root", type=Path, default=Path(".")) + parser.add_argument("--kind", choices=["model", "rule", "generic"]) + parser.add_argument("--name") + parser.add_argument("--source") + parser.add_argument("--target") + parser.add_argument("--format", choices=["json", "text"], default="json") + + +def add_project_pipeline_parser(subparsers) -> None: + parser = subparsers.add_parser("project-pipeline") + parser.add_argument("--repo", default=DEFAULT_REPO) + parser.add_argument("--issue", type=int) + parser.add_argument("--repo-root", type=Path, default=Path(".")) + parser.add_argument("--format", choices=["json", "text"], default="json") + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="Skill-scoped pipeline context bundles.") + subparsers = parser.add_subparsers(dest="command", required=True) + + add_bundle_parser(subparsers, "review-pipeline") + add_bundle_parser(subparsers, "final-review") + add_review_implementation_parser(subparsers) + add_project_pipeline_parser(subparsers) + + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + + if args.command == "review-pipeline": + emit_result( + build_review_pipeline_context( + repo=args.repo, + pr_number=args.pr, + state_file=args.state_file, + ), + args.format, + ) + return 0 + + if args.command == "final-review": + emit_result( + build_final_review_context( + repo=args.repo, + pr_number=args.pr, + state_file=args.state_file, + ), + args.format, + ) + return 0 + + if args.command == "review-implementation": + emit_result( + build_review_implementation_context( + repo_root=args.repo_root, + kind=args.kind, + name=getattr(args, "name", None), + source=getattr(args, "source", None), + target=getattr(args, "target", None), + ), + args.format, + ) + return 0 + + if args.command == "project-pipeline": + emit_result( + build_project_pipeline_context( + repo=args.repo, + issue_number=args.issue, + repo_root=args.repo_root, + ), + args.format, + ) + return 0 + + raise AssertionError(f"Unhandled command: {args.command}") + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/pipeline_worktree.py b/scripts/pipeline_worktree.py new file mode 100644 index 000000000..cd6b88ab8 --- /dev/null +++ b/scripts/pipeline_worktree.py @@ -0,0 +1,383 @@ +#!/usr/bin/env python3 +"""Shared worktree helpers for issue and PR pipeline flows.""" + +from __future__ import annotations + +import argparse +import json +import re +import subprocess +import sys +from pathlib import Path + + +def sanitize_component(text: str) -> str: + normalized = re.sub(r"[^A-Za-z0-9]+", "-", text.strip().lower()).strip("-") + return normalized or "work" + + +def plan_issue_worktree( + repo_root: str | Path, + *, + issue_number: int, + slug: str, + base_ref: str = "origin/main", +) -> dict: + repo_root = str(Path(repo_root)) + branch = f"issue-{issue_number}-{sanitize_component(slug)}" + worktree_dir = str(Path(repo_root) / ".worktrees" / branch) + return { + "issue_number": issue_number, + "slug": slug, + "branch": branch, + "worktree_dir": worktree_dir, + "base_ref": base_ref, + } + + +def plan_pr_worktree( + repo_root: str | Path, + *, + pr_number: int, + head_ref_name: str, + base_sha: str, + head_sha: str, +) -> dict: + repo_root = str(Path(repo_root)) + local_branch = f"review-pr-{pr_number}-{sanitize_component(head_ref_name)}" + worktree_dir = str(Path(repo_root) / ".worktrees" / local_branch) + return { + "pr_number": pr_number, + "head_ref_name": head_ref_name, + "local_branch": local_branch, + "worktree_dir": worktree_dir, + "fetch_ref": f"pull/{pr_number}/head:{local_branch}", + "base_sha": base_sha, + "head_sha": head_sha, + } + + +def summarize_merge( + *, + worktree: str | Path, + exit_code: int, + conflicts: list[str], +) -> dict: + conflicts = sorted(conflicts) + if exit_code == 0: + status = "clean" + elif conflicts: + status = "conflicted" + else: + status = "aborted" + + likely_complex = len(conflicts) > 1 or any( + path.startswith(".claude/skills/add-model/") + or path.startswith(".claude/skills/add-rule/") + for path in conflicts + ) + + return { + "worktree": str(worktree), + "status": status, + "conflicts": conflicts, + "likely_complex": likely_complex, + } + + +def run_git(repo_root: str | Path, *args: str) -> str: + return subprocess.check_output(["git", "-C", str(repo_root), *args], text=True) + + +def run_git_checked(repo_root: str | Path, *args: str) -> None: + subprocess.check_call(["git", "-C", str(repo_root), *args]) + + +def run_gh_json(*args: str): + return json.loads(subprocess.check_output(["gh", *args], text=True)) + + +def repo_root_from(path: str | Path) -> Path: + return Path(run_git(path, "rev-parse", "--show-toplevel").strip()) + + +def branch_exists(repo_root: str | Path, branch: str) -> bool: + proc = subprocess.run( + ["git", "-C", str(repo_root), "rev-parse", "--verify", branch], + capture_output=True, + text=True, + ) + return proc.returncode == 0 + + +def prepare_issue_branch( + *, + issue_number: int, + slug: str, + base_ref: str = "main", + repo_root: str | Path | None = None, +) -> dict: + repo_root = Path(repo_root or repo_root_from(Path.cwd())).resolve() + plan = plan_issue_worktree( + repo_root, + issue_number=issue_number, + slug=slug, + base_ref=base_ref, + ) + + status_output = run_git(repo_root, "status", "--porcelain").strip() + if status_output: + raise RuntimeError("working tree is dirty; stash or commit changes before branching") + + run_git_checked(repo_root, "checkout", base_ref) + existing_branch = branch_exists(repo_root, plan["branch"]) + if existing_branch: + run_git_checked(repo_root, "checkout", plan["branch"]) + action = "checkout-existing" + else: + run_git_checked(repo_root, "checkout", "-b", plan["branch"]) + action = "create-branch" + + base_sha = run_git(repo_root, "rev-parse", base_ref).strip() + head_sha = run_git(repo_root, "rev-parse", "HEAD").strip() + return { + **plan, + "existing_branch": existing_branch, + "action": action, + "base_sha": base_sha, + "head_sha": head_sha, + } + + +def create_issue_worktree( + *, + issue_number: int, + slug: str, + base_ref: str = "origin/main", + repo_root: str | Path | None = None, +) -> dict: + repo_root = Path(repo_root or repo_root_from(Path.cwd())).resolve() + plan = plan_issue_worktree( + repo_root, + issue_number=issue_number, + slug=slug, + base_ref=base_ref, + ) + + Path(plan["worktree_dir"]).parent.mkdir(parents=True, exist_ok=True) + remote, _, branch_name = base_ref.partition("/") + if remote and branch_name: + run_git_checked(repo_root, "fetch", remote, branch_name) + run_git_checked( + repo_root, + "worktree", + "add", + plan["worktree_dir"], + "-b", + plan["branch"], + base_ref, + ) + + base_sha = run_git(repo_root, "rev-parse", base_ref).strip() + head_sha = run_git(plan["worktree_dir"], "rev-parse", "HEAD").strip() + return { + **plan, + "base_sha": base_sha, + "head_sha": head_sha, + } + + +def checkout_pr_worktree( + *, + repo: str, + pr_number: int, + repo_root: str | Path | None = None, +) -> dict: + repo_root = Path(repo_root or repo_root_from(Path.cwd())).resolve() + pr_data = run_gh_json( + "pr", + "view", + str(pr_number), + "--repo", + repo, + "--json", + "headRefName,headRefOid,baseRefOid", + ) + + plan = plan_pr_worktree( + repo_root, + pr_number=pr_number, + head_ref_name=pr_data["headRefName"], + base_sha=pr_data["baseRefOid"], + head_sha=pr_data["headRefOid"], + ) + + Path(plan["worktree_dir"]).parent.mkdir(parents=True, exist_ok=True) + run_git_checked(repo_root, "fetch", "origin", plan["fetch_ref"]) + run_git_checked(repo_root, "worktree", "add", plan["worktree_dir"], plan["local_branch"]) + return plan + + +def merge_main( + *, + worktree: str | Path, +) -> dict: + worktree = Path(worktree).resolve() + run_git_checked(worktree, "fetch", "origin", "main") + proc = subprocess.run( + ["git", "-C", str(worktree), "merge", "origin/main", "--no-edit"], + text=True, + capture_output=True, + ) + + conflict_output = run_git(worktree, "diff", "--name-only", "--diff-filter=U").strip() + conflicts = [line for line in conflict_output.splitlines() if line] + summary = summarize_merge(worktree=worktree, exit_code=proc.returncode, conflicts=conflicts) + summary["stdout"] = proc.stdout + summary["stderr"] = proc.stderr + return summary + + +def prepare_review( + *, + repo: str, + pr_number: int, + repo_root: str | Path | None = None, +) -> dict: + checkout = checkout_pr_worktree( + repo=repo, + pr_number=pr_number, + repo_root=repo_root, + ) + merge = merge_main(worktree=checkout["worktree_dir"]) + return { + "repo": repo, + "pr_number": pr_number, + "ready": merge["status"] == "clean", + "checkout": checkout, + "merge": merge, + } + + +def cleanup_worktree(*, worktree: str | Path) -> dict: + worktree = Path(worktree).resolve() + repo_root = repo_root_from(worktree) + subprocess.check_call(["git", "-C", str(repo_root), "worktree", "remove", str(worktree), "--force"]) + branch = run_git(repo_root, "branch", "--list", "--format=%(refname:short)").splitlines() + return { + "worktree": str(worktree), + "removed": not worktree.exists(), + "branch_still_exists": worktree.name in branch, + } + + +def emit_result(result: dict, fmt: str) -> None: + if fmt == "json": + print(json.dumps(result, indent=2, sort_keys=True)) + else: + print(json.dumps(result, indent=2, sort_keys=True)) + + +def parse_args(argv: list[str]) -> argparse.Namespace: + parser = argparse.ArgumentParser(description="Pipeline worktree helpers.") + subparsers = parser.add_subparsers(dest="command", required=True) + + create_issue = subparsers.add_parser("create-issue") + create_issue.add_argument("--issue", required=True, type=int) + create_issue.add_argument("--slug", required=True) + create_issue.add_argument("--base", default="origin/main") + create_issue.add_argument("--repo-root") + create_issue.add_argument("--format", choices=["json", "text"], default="json") + + prepare_issue = subparsers.add_parser("prepare-issue-branch") + prepare_issue.add_argument("--issue", required=True, type=int) + prepare_issue.add_argument("--slug", required=True) + prepare_issue.add_argument("--base", default="main") + prepare_issue.add_argument("--repo-root") + prepare_issue.add_argument("--format", choices=["json", "text"], default="json") + + checkout_pr = subparsers.add_parser("checkout-pr") + checkout_pr.add_argument("--repo", required=True) + checkout_pr.add_argument("--pr", required=True, type=int) + checkout_pr.add_argument("--repo-root") + checkout_pr.add_argument("--format", choices=["json", "text"], default="json") + + prepare_review = subparsers.add_parser("prepare-review") + prepare_review.add_argument("--repo", required=True) + prepare_review.add_argument("--pr", required=True, type=int) + prepare_review.add_argument("--repo-root") + prepare_review.add_argument("--format", choices=["json", "text"], default="json") + + merge_parser = subparsers.add_parser("merge-main") + merge_parser.add_argument("--worktree", required=True) + merge_parser.add_argument("--format", choices=["json", "text"], default="json") + + cleanup_parser = subparsers.add_parser("cleanup") + cleanup_parser.add_argument("--worktree", required=True) + cleanup_parser.add_argument("--format", choices=["json", "text"], default="json") + + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv or sys.argv[1:]) + + if args.command == "create-issue": + emit_result( + create_issue_worktree( + issue_number=args.issue, + slug=args.slug, + base_ref=args.base, + repo_root=args.repo_root, + ), + args.format, + ) + return 0 + + if args.command == "prepare-issue-branch": + emit_result( + prepare_issue_branch( + issue_number=args.issue, + slug=args.slug, + base_ref=args.base, + repo_root=args.repo_root, + ), + args.format, + ) + return 0 + + if args.command == "checkout-pr": + emit_result( + checkout_pr_worktree( + repo=args.repo, + pr_number=args.pr, + repo_root=args.repo_root, + ), + args.format, + ) + return 0 + + if args.command == "prepare-review": + emit_result( + prepare_review( + repo=args.repo, + pr_number=args.pr, + repo_root=args.repo_root, + ), + args.format, + ) + return 0 + + if args.command == "merge-main": + emit_result(merge_main(worktree=args.worktree), args.format) + return 0 + + if args.command == "cleanup": + emit_result(cleanup_worktree(worktree=args.worktree), args.format) + return 0 + + raise AssertionError(f"Unhandled command: {args.command}") + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/project_board_poll.py b/scripts/project_board_poll.py index 6df5ebac4..92c628334 100644 --- a/scripts/project_board_poll.py +++ b/scripts/project_board_poll.py @@ -1,59 +1,23 @@ #!/usr/bin/env python3 -"""Track eligible project-board items and expose a retryable pending queue.""" +"""Compatibility wrapper for the board poller CLI.""" from __future__ import annotations import argparse -import json import subprocess import sys from pathlib import Path -from typing import Callable -COPILOT_REVIEWER = "copilot-pull-request-reviewer[bot]" - - -def item_identity(item: dict) -> str: - item_id = item.get("id") - if item_id is not None: - return str(item_id) - - content = item.get("content") or {} - number = content.get("number") - item_type = content.get("type", "item") - if number is not None: - return f"{item_type}:{number}" - - title = item.get("title") - if title: - return str(title) - - raise ValueError(f"Board item has no stable identity: {item!r}") - - -def load_state(state_file: Path) -> dict: - if not state_file.exists(): - return {"visible": {}, "pending": []} - - raw = state_file.read_text().strip() - if not raw: - return {"visible": {}, "pending": []} - - data = json.loads(raw) - if not isinstance(data, dict): - raise ValueError(f"State file must contain a JSON object: {state_file}") +import pipeline_board - visible = data.get("visible", {}) - pending = data.get("pending", []) - if not isinstance(visible, dict) or not isinstance(pending, list): - raise ValueError(f"Invalid poll state format: {state_file}") +item_identity = pipeline_board.item_identity +load_state = pipeline_board.load_state +save_state = pipeline_board.save_state +has_copilot_review = pipeline_board.has_copilot_review +ready_entries = pipeline_board.ready_entries +ack_item = pipeline_board.ack_item - return {"visible": visible, "pending": [str(item_id) for item_id in pending]} - - -def save_state(state_file: Path, state: dict) -> None: - state_file.parent.mkdir(parents=True, exist_ok=True) - state_file.write_text(json.dumps(state, indent=2, sort_keys=True) + "\n") +COPILOT_REVIEWER = "copilot-pull-request-reviewer[bot]" def fetch_pr_reviews(repo: str, pr_number: int) -> list[dict]: @@ -61,12 +25,30 @@ def fetch_pr_reviews(repo: str, pr_number: int) -> list[dict]: ["gh", "api", f"repos/{repo}/pulls/{pr_number}/reviews"], text=True, ) - data = json.loads(output) + data = pipeline_board.json.loads(output) if not isinstance(data, list): raise ValueError(f"Unexpected PR review payload for #{pr_number}: {data!r}") return data +def fetch_pr_state(repo: str, pr_number: int) -> str: + return subprocess.check_output( + [ + "gh", + "pr", + "view", + str(pr_number), + "--repo", + repo, + "--json", + "state", + "--jq", + ".state", + ], + text=True, + ).strip() + + def resolve_issue_pr(repo: str, issue_number: int) -> int | None: output = subprocess.check_output( [ @@ -84,81 +66,48 @@ def resolve_issue_pr(repo: str, issue_number: int) -> int | None: ], text=True, ) - data = json.loads(output) + data = pipeline_board.json.loads(output) if not data: return None return int(data[0]["number"]) -def has_copilot_review(reviews: list[dict]) -> bool: - return any( - review.get("user", {}).get("login") == COPILOT_REVIEWER for review in reviews - ) - - -def ready_entries(board_data: dict) -> dict[str, dict]: - entries = {} - for item in board_data.get("items", []): - if item.get("status") != "Ready": - continue - - content = item.get("content") or {} - number = content.get("number") - if number is None: - continue - - entries[item_identity(item)] = {"number": int(number)} - return entries +def linked_repo_pr_numbers(item: dict, repo: str) -> list[int]: + return pipeline_board.linked_repo_pr_numbers(item, repo) def review_entries( board_data: dict, repo: str, - review_fetcher: Callable[[str, int], list[dict]] = fetch_pr_reviews, - pr_resolver: Callable[[str, int], int | None] = resolve_issue_pr, + review_fetcher=fetch_pr_reviews, + pr_resolver=resolve_issue_pr, + pr_state_fetcher=fetch_pr_state, ) -> dict[str, dict]: - entries = {} - for item in board_data.get("items", []): - if item.get("status") != "Review pool": - continue - - content = item.get("content") or {} - item_type = content.get("type") - number = content.get("number") - if number is None: - continue - - pr_number: int | None - if item_type == "PullRequest": - pr_number = int(number) - elif item_type == "Issue": - pr_number = pr_resolver(repo, int(number)) - else: - pr_number = None - - if pr_number is None: - continue - - reviews = review_fetcher(repo, pr_number) - if has_copilot_review(reviews): - entries[item_identity(item)] = {"number": pr_number} - return entries + return pipeline_board.review_entries( + board_data, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, + ) def current_entries( mode: str, board_data: dict, repo: str | None = None, - review_fetcher: Callable[[str, int], list[dict]] = fetch_pr_reviews, - pr_resolver: Callable[[str, int], int | None] = resolve_issue_pr, + review_fetcher=fetch_pr_reviews, + pr_resolver=resolve_issue_pr, + pr_state_fetcher=fetch_pr_state, ) -> dict[str, dict]: - if mode == "ready": - return ready_entries(board_data) - if mode == "review": - if repo is None: - raise ValueError("repo is required in review mode") - return review_entries(board_data, repo, review_fetcher, pr_resolver) - raise ValueError(f"Unsupported mode: {mode}") + return pipeline_board.current_entries( + mode, + board_data, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, + ) def process_snapshot( @@ -166,39 +115,19 @@ def process_snapshot( board_data: dict, state_file: Path, repo: str | None = None, - review_fetcher: Callable[[str, int], list[dict]] = fetch_pr_reviews, - pr_resolver: Callable[[str, int], int | None] = resolve_issue_pr, + review_fetcher=fetch_pr_reviews, + pr_resolver=resolve_issue_pr, + pr_state_fetcher=fetch_pr_state, ) -> tuple[str, int] | None: - state = load_state(state_file) - previous_visible = state["visible"] - current_visible = current_entries(mode, board_data, repo, review_fetcher, pr_resolver) - - pending = [item_id for item_id in state["pending"] if item_id in current_visible] - entered = sorted( - (item_id for item_id in current_visible if item_id not in previous_visible), - key=lambda item_id: (current_visible[item_id]["number"], item_id), + return pipeline_board.process_snapshot( + mode, + board_data, + state_file, + repo, + review_fetcher, + pr_resolver, + pr_state_fetcher, ) - for item_id in entered: - if item_id not in pending: - pending.append(item_id) - - state["visible"] = current_visible - state["pending"] = pending - save_state(state_file, state) - - if not pending: - return None - - item_id = pending[0] - return item_id, int(current_visible[item_id]["number"]) - - -def ack_item(state_file: Path, item_id: str) -> None: - state = load_state(state_file) - state["pending"] = [ - pending_id for pending_id in state["pending"] if pending_id != item_id - ] - save_state(state_file, state) def parse_args(argv: list[str]) -> argparse.Namespace: @@ -229,7 +158,7 @@ def main(argv: list[str] | None = None) -> int: if args.mode == "review" and not args.repo: raise SystemExit("--repo is required in review mode") - board_data = json.load(sys.stdin) + board_data = pipeline_board.json.load(sys.stdin) next_item = process_snapshot( args.mode, board_data, diff --git a/scripts/project_board_recover.py b/scripts/project_board_recover.py index c77b27692..c0eb74ae6 100644 --- a/scripts/project_board_recover.py +++ b/scripts/project_board_recover.py @@ -1,38 +1,37 @@ #!/usr/bin/env python3 -"""Recover GitHub Project board statuses after the Status field was recreated.""" +"""Compatibility wrapper for project-board status recovery.""" from __future__ import annotations import argparse -import json import subprocess import sys -from collections import Counter -from datetime import datetime, timezone from pathlib import Path +import pipeline_board + PROJECT_ID = "PVT_kwDOBrtarc4BRNVy" STATUS_FIELD_ID = "PVTSSF_lADOBrtarc4BRNVyzg_GmQc" -STATUS_BACKLOG = "Backlog" -STATUS_READY = "Ready" -STATUS_REVIEW_POOL = "Review pool" -STATUS_FINAL_REVIEW = "Final review" -STATUS_DONE = "Done" - -STATUS_OPTION_IDS = { - STATUS_BACKLOG: "ab337660", - STATUS_READY: "f37d0d80", - STATUS_REVIEW_POOL: "7082ed60", - STATUS_FINAL_REVIEW: "51a3d8bb", - STATUS_DONE: "6aca54fa", -} - -FAILURE_LABELS = {"PoorWritten", "Wrong", "Trivial", "Useless"} -COPILOT_REVIEWERS = { - "copilot-pull-request-reviewer", - "copilot-pull-request-reviewer[bot]", -} +STATUS_BACKLOG = pipeline_board.STATUS_BACKLOG +STATUS_READY = pipeline_board.STATUS_READY +STATUS_REVIEW_POOL = pipeline_board.STATUS_REVIEW_POOL +STATUS_FINAL_REVIEW = pipeline_board.STATUS_FINAL_REVIEW +STATUS_DONE = pipeline_board.STATUS_DONE +STATUS_OPTION_IDS = pipeline_board.STATUS_OPTION_IDS +FAILURE_LABELS = pipeline_board.FAILURE_LABELS +COPILOT_REVIEWERS = pipeline_board.COPILOT_REVIEWERS +label_names = pipeline_board.label_names +linked_pr_numbers = pipeline_board.linked_pr_numbers +is_tracked_issue_title = pipeline_board.is_tracked_issue_title +has_copilot_review = pipeline_board.has_copilot_review +all_checks_green = pipeline_board.all_checks_green +infer_issue_status = pipeline_board.infer_issue_status +build_recovery_plan = pipeline_board.build_recovery_plan +apply_plan = pipeline_board.apply_plan +save_backup = pipeline_board.save_backup +print_summary = pipeline_board.print_summary +print_examples = pipeline_board.print_examples def run_gh(*args: str) -> str: @@ -40,7 +39,7 @@ def run_gh(*args: str) -> str: def fetch_board_items(owner: str, project_number: int, limit: int) -> dict: - return json.loads( + return pipeline_board.json.loads( run_gh( "project", "item-list", @@ -56,7 +55,7 @@ def fetch_board_items(owner: str, project_number: int, limit: int) -> dict: def fetch_issues(repo: str, limit: int) -> list[dict]: - return json.loads( + return pipeline_board.json.loads( run_gh( "issue", "list", @@ -73,7 +72,7 @@ def fetch_issues(repo: str, limit: int) -> list[dict]: def fetch_prs(repo: str, limit: int) -> list[dict]: - return json.loads( + return pipeline_board.json.loads( run_gh( "pr", "list", @@ -90,224 +89,12 @@ def fetch_prs(repo: str, limit: int) -> list[dict]: def fetch_pr_reviews(repo: str, pr_number: int) -> list[dict]: - data = json.loads( + data = pipeline_board.json.loads( run_gh("pr", "view", str(pr_number), "-R", repo, "--json", "reviews") ) return data.get("reviews", []) - - -def label_names(issue: dict) -> set[str]: - return {label["name"] for label in issue.get("labels", [])} - - -def linked_pr_numbers(item: dict) -> list[int]: - urls = item.get("linked pull requests") or [] - numbers = [] - for url in urls: - try: - numbers.append(int(url.rstrip("/").split("/")[-1])) - except ValueError: - continue - return numbers - - -def is_tracked_issue_title(title: str | None) -> bool: - if not title: - return False - return title.startswith("[Model]") or title.startswith("[Rule]") - - -def has_copilot_review(reviews: list[dict]) -> bool: - for review in reviews: - author = review.get("author") or review.get("user") or {} - if author.get("login") in COPILOT_REVIEWERS: - return True - return False - - -def all_checks_green(pr: dict) -> bool: - statuses = pr.get("statusCheckRollup") or [] - if not statuses: - return False - - for status in statuses: - typename = status.get("__typename") - if typename == "CheckRun": - if status.get("status") != "COMPLETED": - return False - if status.get("conclusion") not in {"SUCCESS", "SKIPPED", "NEUTRAL"}: - return False - elif typename == "StatusContext": - if status.get("state") != "SUCCESS": - return False - return True - - -def infer_issue_status( - issue: dict, - linked_prs: list[dict], - pr_reviews: dict[int, list[dict]], -) -> tuple[str, str]: - labels = label_names(issue) - merged_prs = [pr for pr in linked_prs if pr.get("mergedAt")] - open_prs = [pr for pr in linked_prs if pr.get("state") == "OPEN"] - - if merged_prs: - pr_numbers = ", ".join(f"#{pr['number']}" for pr in merged_prs) - return STATUS_DONE, f"linked merged PR {pr_numbers}" - - if issue.get("state") == "CLOSED": - return STATUS_DONE, "issue itself is closed" - - if open_prs: - waiting_for_copilot = [ - pr - for pr in open_prs - if not has_copilot_review(pr_reviews.get(int(pr["number"]), [])) - ] - if waiting_for_copilot: - pr_numbers = ", ".join(f"#{pr['number']}" for pr in waiting_for_copilot) - return STATUS_REVIEW_POOL, f"open PR {pr_numbers} waiting for Copilot review" - - green_prs = [pr for pr in open_prs if all_checks_green(pr)] - if len(green_prs) == len(open_prs): - pr_numbers = ", ".join(f"#{pr['number']}" for pr in open_prs) - return STATUS_FINAL_REVIEW, f"Copilot reviewed green open PR {pr_numbers}" - - pr_numbers = ", ".join(f"#{pr['number']}" for pr in open_prs) - return STATUS_REVIEW_POOL, f"open PR {pr_numbers} still implementing or fixing review" - - if "Good" in labels: - return STATUS_READY, 'label "Good" present and no linked PR' - - if labels & FAILURE_LABELS: - bad = ", ".join(sorted(labels & FAILURE_LABELS)) - return STATUS_BACKLOG, f"failure labels present: {bad}" - - return STATUS_BACKLOG, "default backlog: no linked PR and no Ready signal" - - -def build_recovery_plan( - board_data: dict, - issues: list[dict], - prs: list[dict], - pr_reviews: dict[int, list[dict]], -) -> list[dict]: - issues_by_number = {issue["number"]: issue for issue in issues} - prs_by_number = {pr["number"]: pr for pr in prs} - - plan = [] - for item in board_data.get("items", []): - content = item.get("content") or {} - issue_number = content.get("number") - if issue_number is None: - continue - - issue = issues_by_number.get(issue_number) - if issue is None: - continue - - title = content.get("title") or issue.get("title") - if not is_tracked_issue_title(title): - continue - - linked_prs = [ - prs_by_number[pr_number] - for pr_number in linked_pr_numbers(item) - if pr_number in prs_by_number - ] - status_name, reason = infer_issue_status(issue, linked_prs, pr_reviews) - plan.append( - { - "item_id": item["id"], - "issue_number": issue_number, - "title": title, - "current_status": item.get("status"), - "proposed_status": status_name, - "option_id": STATUS_OPTION_IDS[status_name], - "reason": reason, - } - ) - - return sorted(plan, key=lambda entry: entry["issue_number"]) - - -def save_backup( - backup_file: Path, - *, - board_data: dict, - issues: list[dict], - prs: list[dict], - pr_reviews: dict[int, list[dict]], - plan: list[dict], -) -> None: - backup_file.parent.mkdir(parents=True, exist_ok=True) - payload = { - "generated_at": datetime.now(timezone.utc).isoformat(), - "board_data": board_data, - "issues": issues, - "prs": prs, - "pr_reviews": pr_reviews, - "plan": plan, - } - backup_file.write_text(json.dumps(payload, indent=2, sort_keys=True) + "\n") - - -def apply_plan( - plan: list[dict], - *, - project_id: str, - field_id: str, -) -> int: - changed = 0 - for entry in plan: - if entry["current_status"] == entry["proposed_status"]: - continue - subprocess.check_call( - [ - "gh", - "project", - "item-edit", - "--project-id", - project_id, - "--id", - entry["item_id"], - "--field-id", - field_id, - "--single-select-option-id", - entry["option_id"], - ] - ) - changed += 1 - return changed - - def default_backup_path(project_number: int) -> Path: - stamp = datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ") - return Path("/tmp") / f"project-{project_number}-status-recovery-{stamp}.json" - - -def print_summary(plan: list[dict]) -> None: - counts = Counter(entry["proposed_status"] for entry in plan) - print("Proposed status counts:") - for status_name in [ - STATUS_BACKLOG, - STATUS_READY, - STATUS_REVIEW_POOL, - STATUS_FINAL_REVIEW, - STATUS_DONE, - ]: - print(f" {status_name}: {counts.get(status_name, 0)}") - - -def print_examples(plan: list[dict], limit: int = 20) -> None: - print("") - print(f"First {min(limit, len(plan))} assignments:") - for entry in plan[:limit]: - print( - f" #{entry['issue_number']:<4} {entry['proposed_status']:<13} " - f"{entry['reason']} | {entry['title']}" - ) + return pipeline_board.default_backup_path(project_number) def parse_args(argv: list[str]) -> argparse.Namespace: @@ -351,7 +138,9 @@ def main(argv: list[str] | None = None) -> int: plan = build_recovery_plan(board_data, issues, prs, pr_reviews) if args.plan_file is not None: args.plan_file.parent.mkdir(parents=True, exist_ok=True) - args.plan_file.write_text(json.dumps(plan, indent=2, sort_keys=True) + "\n") + args.plan_file.write_text( + pipeline_board.json.dumps(plan, indent=2, sort_keys=True) + "\n" + ) print_summary(plan) if not args.no_examples: diff --git a/scripts/test_make_helpers.py b/scripts/test_make_helpers.py index 5b3f7a224..1089235cc 100644 --- a/scripts/test_make_helpers.py +++ b/scripts/test_make_helpers.py @@ -55,6 +55,558 @@ def test_run_agent_enables_multi_agent_for_codex(self) -> None: ], ) + def test_skill_prompt_with_context_appends_json_for_codex(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "RUNNER=codex " + "skill_prompt_with_context review-pipeline '/review-pipeline 570' " + "'process PR #570' 'Selected queue item' '{\"pr_number\":570}'" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn("Use the repo-local skill", proc.stdout) + self.assertIn("Selected queue item", proc.stdout) + self.assertIn('{"pr_number":570}', proc.stdout) + + def test_skill_prompt_with_context_keeps_claude_slash_command_clean(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "RUNNER=claude " + "skill_prompt_with_context review-pipeline '/review-pipeline 570' " + "'process PR #570' 'Selected queue item' '{\"pr_number\":570}'" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual(proc.stdout.strip(), "/review-pipeline 570") + + def test_poll_project_items_uses_pipeline_board_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "poll_project_items ready /tmp/state.json" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_board.py", + "next", + "ready", + "/tmp/state.json", + "--format", + "text", + ], + ) + + def test_move_board_item_uses_pipeline_board_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "move_board_item PVTI_demo final-review" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_board.py", + "move", + "PVTI_demo", + "final-review", + ], + ) + + def test_claim_project_items_uses_pipeline_board_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "claim_project_items ready /tmp/state.json" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_board.py", + "claim-next", + "ready", + "/tmp/state.json", + "--format", + "json", + ], + ) + + def test_make_board_next_final_review_passes_repo(self) -> None: + proc = subprocess.run( + [ + "make", + "-n", + "board-next", + "MODE=final-review", + "REPO=CodingThrust/problem-reductions", + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn( + 'poll_project_items "final-review" "$state_file" "$repo"', + proc.stdout, + ) + + def test_make_board_next_review_forwards_number_and_format(self) -> None: + proc = subprocess.run( + [ + "make", + "-n", + "board-next", + "MODE=review", + "REPO=CodingThrust/problem-reductions", + "NUMBER=570", + "FORMAT=json", + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn( + 'poll_project_items "review" "$state_file" "$repo" "570" "json"', + proc.stdout, + ) + + def test_make_board_claim_review_forwards_repo_number_and_format(self) -> None: + proc = subprocess.run( + [ + "make", + "-n", + "board-claim", + "MODE=review", + "REPO=CodingThrust/problem-reductions", + "NUMBER=570", + "FORMAT=json", + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn( + 'claim_project_items "review" "$state_file" "$repo" "570" "json"', + proc.stdout, + ) + + def test_board_next_json_uses_scripted_json_poll(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "board_next_json review CodingThrust/problem-reductions 570 /tmp/review.json" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_board.py", + "next", + "review", + "/tmp/review.json", + "--format", + "json", + "--repo", + "CodingThrust/problem-reductions", + "--number", + "570", + ], + ) + + def test_board_claim_json_uses_scripted_json_claim(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "board_claim_json review CodingThrust/problem-reductions 570 /tmp/review.json" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_board.py", + "claim-next", + "review", + "/tmp/review.json", + "--format", + "json", + "--repo", + "CodingThrust/problem-reductions", + "--number", + "570", + ], + ) + + def test_review_pipeline_context_uses_skill_bundle_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "review_pipeline_context CodingThrust/problem-reductions 570 /tmp/review.json" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_skill_context.py", + "review-pipeline", + "--repo", + "CodingThrust/problem-reductions", + "--state-file", + "/tmp/review.json", + "--format", + "json", + "--pr", + "570", + ], + ) + + def test_make_run_review_uses_skill_bundle_context(self) -> None: + proc = subprocess.run( + ["make", "-n", "run-review"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn('review_pipeline_context "$repo"', proc.stdout) + self.assertIn('skill_prompt_with_context review-pipeline', proc.stdout) + + def test_make_run_pipeline_uses_scripted_board_selection(self) -> None: + proc = subprocess.run( + ["make", "-n", "run-pipeline"], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn('board_next_json ready "" "" "$state_file"', proc.stdout) + self.assertIn('skill_prompt_with_context project-pipeline', proc.stdout) + + def test_pr_snapshot_uses_pipeline_pr_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "pr_snapshot CodingThrust/problem-reductions 570" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_pr.py", + "snapshot", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--format", + "json", + ], + ) + + def test_issue_guards_uses_pipeline_checks_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "issue_guards CodingThrust/problem-reductions 117 /tmp/repo" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_checks.py", + "issue-guards", + "--repo", + "CodingThrust/problem-reductions", + "--issue", + "117", + "--repo-root", + "/tmp/repo", + "--format", + "json", + ], + ) + + def test_pr_wait_ci_uses_pipeline_pr_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "pr_wait_ci CodingThrust/problem-reductions 570 1200 15" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_pr.py", + "wait-ci", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--timeout", + "1200", + "--interval", + "15", + "--format", + "json", + ], + ) + + def test_issue_context_uses_pipeline_checks_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "issue_context CodingThrust/problem-reductions 117 /tmp/repo" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_checks.py", + "issue-context", + "--repo", + "CodingThrust/problem-reductions", + "--issue", + "117", + "--repo-root", + "/tmp/repo", + "--format", + "json", + ], + ) + + def test_make_issue_context_uses_shared_helper(self) -> None: + proc = subprocess.run( + [ + "make", + "-n", + "issue-context", + "ISSUE=117", + "REPO=CodingThrust/problem-reductions", + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertIn( + 'issue_context "$repo" "117"', + proc.stdout, + ) + + def test_create_issue_worktree_uses_pipeline_worktree_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "create_issue_worktree 117 graph-partitioning origin/main" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_worktree.py", + "create-issue", + "--issue", + "117", + "--slug", + "graph-partitioning", + "--base", + "origin/main", + "--format", + "json", + ], + ) + + def test_checkout_pr_worktree_uses_pipeline_worktree_cli(self) -> None: + if shutil.which("dash") is None: + self.skipTest("dash is not installed") + + proc = subprocess.run( + [ + "dash", + "-c", + ( + ". scripts/make_helpers.sh; " + "python3() { printf '%s\\n' \"$@\"; }; " + "checkout_pr_worktree CodingThrust/problem-reductions 570" + ), + ], + cwd=REPO_ROOT, + capture_output=True, + text=True, + ) + self.assertEqual(proc.returncode, 0, proc.stderr) + self.assertEqual( + proc.stdout.splitlines(), + [ + "scripts/pipeline_worktree.py", + "checkout-pr", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--format", + "json", + ], + ) + if __name__ == "__main__": unittest.main() diff --git a/scripts/test_pipeline_board.py b/scripts/test_pipeline_board.py new file mode 100644 index 000000000..1e69fe8ee --- /dev/null +++ b/scripts/test_pipeline_board.py @@ -0,0 +1,702 @@ +#!/usr/bin/env python3 +import io +import json +import tempfile +import unittest +from contextlib import redirect_stdout +from pathlib import Path + +from pipeline_board import ( + STATUS_DONE, + STATUS_FINAL_REVIEW, + STATUS_IN_PROGRESS, + STATUS_ON_HOLD, + STATUS_READY, + STATUS_REVIEW_POOL, + STATUS_UNDER_REVIEW, + ack_item, + claim_next_entry, + build_recovery_plan, + normalize_status_name, + print_next_item, + process_snapshot, + review_candidates, + select_next_entry, + status_items, +) + + +def make_issue_item( + item_id: str, + number: int, + *, + status: str = "Ready", + title: str | None = None, + linked_prs: list[int] | None = None, +) -> dict: + item = { + "id": item_id, + "status": status, + "content": { + "type": "Issue", + "number": number, + "title": title or f"[Model] Issue {number}", + }, + "title": title or f"[Model] Issue {number}", + } + if linked_prs is not None: + item["linked pull requests"] = [ + f"https://github.com/CodingThrust/problem-reductions/pull/{pr_number}" + for pr_number in linked_prs + ] + return item + + +def make_pr_item(item_id: str, number: int, status: str = "Review pool") -> dict: + return { + "id": item_id, + "status": status, + "content": {"type": "PullRequest", "number": number}, + } + + +def make_issue(number: int, *, state: str = "OPEN", labels: list[str] | None = None) -> dict: + return { + "number": number, + "state": state, + "title": f"[Model] Issue {number}", + "labels": [{"name": label} for label in (labels or [])], + } + + +def make_pr( + number: int, + *, + state: str = "OPEN", + merged: bool = False, + checks: list[dict] | None = None, +) -> dict: + return { + "number": number, + "state": state, + "mergedAt": "2026-03-15T00:00:00Z" if merged else None, + "statusCheckRollup": checks or [], + } + + +def success_check(name: str = "ci") -> dict: + return { + "__typename": "CheckRun", + "name": name, + "status": "COMPLETED", + "conclusion": "SUCCESS", + } + + +class PipelineBoardPollTests(unittest.TestCase): + def test_ready_queue_retries_same_item_until_ack(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "ready-state.json" + snapshot = { + "items": [ + make_issue_item("PVTI_1", 101), + make_issue_item("PVTI_2", 102), + ] + } + + item_id, number = process_snapshot("ready", snapshot, state_file) + self.assertEqual((item_id, number), ("PVTI_1", 101)) + + item_id, number = process_snapshot("ready", snapshot, state_file) + self.assertEqual((item_id, number), ("PVTI_1", 101)) + + ack_item(state_file, "PVTI_1") + item_id, number = process_snapshot("ready", snapshot, state_file) + self.assertEqual((item_id, number), ("PVTI_2", 102)) + + def test_review_queue_resolves_issue_cards_to_prs(self) -> None: + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + return 570 if issue_number == 117 else None + + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + self.assertEqual(repo, "CodingThrust/problem-reductions") + if pr_number == 570: + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + return [] + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "OPEN" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + item_id, number = process_snapshot( + "review", + {"items": [make_issue_item("PVTI_10", 117, status="Review pool")]}, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertEqual((item_id, number), ("PVTI_10", 570)) + + def test_review_queue_skips_closed_pr_cards(self) -> None: + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "CLOSED" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + no_item = process_snapshot( + "review", + {"items": [make_pr_item("PVTI_10", 570)]}, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertIsNone(no_item) + + def test_final_review_queue_resolves_issue_cards_to_open_prs(self) -> None: + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + return 615 if issue_number == 101 else None + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 615) + return "OPEN" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "final-review-state.json" + item_id, number = process_snapshot( + "final-review", + {"items": [make_issue_item("PVTI_20", 101, status="Final review")]}, + state_file, + repo="CodingThrust/problem-reductions", + pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertEqual((item_id, number), ("PVTI_20", 615)) + + def test_final_review_queue_skips_closed_pr_cards(self) -> None: + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 621) + return "CLOSED" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "final-review-state.json" + no_item = process_snapshot( + "final-review", + {"items": [make_pr_item("PVTI_21", 621, status="Final review")]}, + state_file, + repo="CodingThrust/problem-reductions", + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertIsNone(no_item) + + +class PipelineBoardRecoveryTests(unittest.TestCase): + def test_recovery_plan_marks_merged_pr_items_done(self) -> None: + board_data = { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="Review pool", + title="[Model] MinimumFeedbackVertexSet", + linked_prs=[615], + ) + ] + } + issues = [make_issue(101, labels=["Good"])] + prs = [make_pr(615, state="MERGED", merged=True)] + + plan = build_recovery_plan(board_data, issues, prs, pr_reviews={}) + + self.assertEqual(len(plan), 1) + self.assertEqual(plan[0]["proposed_status"], STATUS_DONE) + + def test_recovery_plan_marks_green_copilot_reviewed_prs_final_review(self) -> None: + board_data = { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="Review pool", + title="[Model] HamiltonianPath", + linked_prs=[621], + ) + ] + } + issues = [make_issue(101, labels=["Good"])] + prs = [make_pr(621, checks=[success_check()])] + pr_reviews = {621: [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}]} + + plan = build_recovery_plan(board_data, issues, prs, pr_reviews=pr_reviews) + + self.assertEqual(plan[0]["proposed_status"], STATUS_FINAL_REVIEW) + + def test_recovery_plan_marks_open_pr_without_copilot_review_review_pool(self) -> None: + board_data = { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="In progress", + title="[Model] SteinerTree", + linked_prs=[192], + ) + ] + } + issues = [make_issue(101, labels=["Good"])] + prs = [make_pr(192, checks=[success_check()])] + + plan = build_recovery_plan(board_data, issues, prs, pr_reviews={192: []}) + + self.assertEqual(plan[0]["proposed_status"], STATUS_REVIEW_POOL) + + def test_recovery_plan_marks_good_issue_without_pr_ready(self) -> None: + board_data = { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="Backlog", + title="[Model] ExactCoverBy3Sets", + ) + ] + } + issues = [make_issue(101, labels=["Good"])] + + plan = build_recovery_plan(board_data, issues, prs=[], pr_reviews={}) + + self.assertEqual(plan[0]["proposed_status"], STATUS_READY) + + +class PipelineBoardStatusTests(unittest.TestCase): + def test_normalize_status_name_accepts_pipeline_aliases(self) -> None: + self.assertEqual(normalize_status_name("ready"), STATUS_READY) + self.assertEqual(normalize_status_name("review-pool"), STATUS_REVIEW_POOL) + self.assertEqual(normalize_status_name("in-progress"), STATUS_IN_PROGRESS) + self.assertEqual(normalize_status_name("under review"), STATUS_UNDER_REVIEW) + self.assertEqual(normalize_status_name("on-hold"), STATUS_ON_HOLD) + self.assertEqual(normalize_status_name("done"), STATUS_DONE) + + +class PipelineBoardOutputTests(unittest.TestCase): + def test_claim_next_ready_moves_selected_item_to_in_progress(self) -> None: + moves: list[tuple[str, str]] = [] + + def fake_mover(item_id: str, status: str) -> None: + moves.append((item_id, status)) + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "ready-state.json" + result = claim_next_entry( + "ready", + { + "items": [ + make_issue_item( + "PVTI_1", + 101, + title="[Model] ExactCoverBy3Sets", + ) + ] + }, + state_file, + mover=fake_mover, + ) + + self.assertEqual(moves, [("PVTI_1", STATUS_IN_PROGRESS)]) + self.assertEqual( + result, + { + "item_id": "PVTI_1", + "number": 101, + "issue_number": 101, + "pr_number": None, + "status": STATUS_READY, + "title": "[Model] ExactCoverBy3Sets", + "claimed": True, + "claimed_status": STATUS_IN_PROGRESS, + }, + ) + + def test_claim_next_review_moves_selected_item_to_under_review(self) -> None: + moves: list[tuple[str, str]] = [] + + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(issue_number, 117) + return 570 + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "OPEN" + + def fake_mover(item_id: str, status: str) -> None: + moves.append((item_id, status)) + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + result = claim_next_entry( + "review", + { + "items": [ + make_issue_item( + "PVTI_10", + 117, + status="Review pool", + title="[Model] GraphPartitioning", + ) + ] + }, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, + mover=fake_mover, + ) + + self.assertEqual(moves, [("PVTI_10", STATUS_UNDER_REVIEW)]) + self.assertEqual( + result, + { + "item_id": "PVTI_10", + "number": 570, + "issue_number": 117, + "pr_number": 570, + "status": STATUS_REVIEW_POOL, + "title": "[Model] GraphPartitioning", + "claimed": True, + "claimed_status": STATUS_UNDER_REVIEW, + }, + ) + + def test_select_next_entry_honors_requested_number(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "ready-state.json" + entry = select_next_entry( + "ready", + { + "items": [ + make_issue_item("PVTI_1", 101, title="[Model] A"), + make_issue_item("PVTI_2", 102, title="[Model] B"), + ] + }, + state_file, + target_number=102, + ) + self.assertEqual( + entry, + { + "item_id": "PVTI_2", + "number": 102, + "issue_number": 102, + "pr_number": None, + "status": STATUS_READY, + "title": "[Model] B", + }, + ) + + def test_select_next_entry_includes_ready_issue_metadata(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "ready-state.json" + entry = select_next_entry( + "ready", + { + "items": [ + make_issue_item( + "PVTI_1", + 101, + title="[Model] ExactCoverBy3Sets", + ) + ] + }, + state_file, + ) + self.assertEqual( + entry, + { + "item_id": "PVTI_1", + "number": 101, + "issue_number": 101, + "pr_number": None, + "status": STATUS_READY, + "title": "[Model] ExactCoverBy3Sets", + }, + ) + + def test_select_next_entry_includes_review_metadata(self) -> None: + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(issue_number, 117) + return 570 + + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "OPEN" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + entry = select_next_entry( + "review", + { + "items": [ + make_issue_item( + "PVTI_10", + 117, + status="Review pool", + title="[Model] GraphPartitioning", + ) + ] + }, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertEqual( + entry, + { + "item_id": "PVTI_10", + "number": 570, + "issue_number": 117, + "pr_number": 570, + "status": STATUS_REVIEW_POOL, + "title": "[Model] GraphPartitioning", + }, + ) + + def test_print_next_item_json_emits_rich_payload(self) -> None: + buffer = io.StringIO() + with redirect_stdout(buffer): + rc = print_next_item( + { + "item_id": "PVTI_20", + "number": 615, + "issue_number": 101, + "pr_number": 615, + "status": STATUS_FINAL_REVIEW, + "title": "[Model] MinimumFeedbackVertexSet", + }, + mode="final-review", + fmt="json", + ) + + self.assertEqual(rc, 0) + self.assertEqual( + json.loads(buffer.getvalue()), + { + "mode": "final-review", + "item_id": "PVTI_20", + "number": 615, + "issue_number": 101, + "pr_number": 615, + "status": STATUS_FINAL_REVIEW, + "title": "[Model] MinimumFeedbackVertexSet", + }, + ) + + +class PipelineBoardReviewCandidateTests(unittest.TestCase): + def test_review_candidates_report_ambiguous_issue_cards(self) -> None: + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + raise AssertionError("ambiguous cards should not fetch reviews") + + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + raise AssertionError("ambiguous cards should not resolve by issue search") + + def fake_pr_info_fetcher(repo: str, pr_number: int) -> dict: + self.assertEqual(repo, "CodingThrust/problem-reductions") + return { + 170: {"number": 170, "state": "CLOSED", "title": "Superseded LCS model"}, + 173: { + "number": 173, + "state": "OPEN", + "title": "Fix #109: Add LCS reduction", + }, + }[pr_number] + + candidates = review_candidates( + { + "items": [ + make_issue_item( + "PVTI_10", + 108, + status="Review pool", + title="[Model] LongestCommonSubsequence", + linked_prs=[170, 173], + ) + ] + }, + "CodingThrust/problem-reductions", + fake_review_fetcher, + fake_pr_resolver, + fake_pr_info_fetcher, + ) + + self.assertEqual(len(candidates), 1) + self.assertEqual( + candidates[0], + { + "item_id": "PVTI_10", + "number": 173, + "issue_number": 108, + "pr_number": 173, + "status": STATUS_REVIEW_POOL, + "title": "[Model] LongestCommonSubsequence", + "eligibility": "ambiguous-linked-prs", + "reason": "multiple linked repo PRs require confirmation", + "recommendation": 173, + "linked_repo_prs": [ + { + "number": 170, + "state": "CLOSED", + "title": "Superseded LCS model", + }, + { + "number": 173, + "state": "OPEN", + "title": "Fix #109: Add LCS reduction", + }, + ], + }, + ) + + def test_review_candidates_report_waiting_for_copilot(self) -> None: + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return [] + + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(issue_number, 117) + return 570 + + def fake_pr_info_fetcher(repo: str, pr_number: int) -> dict: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return { + "number": 570, + "state": "OPEN", + "title": "Fix #117: [Model] GraphPartitioning", + } + + candidates = review_candidates( + { + "items": [ + make_issue_item( + "PVTI_11", + 117, + status="Review pool", + title="[Model] GraphPartitioning", + ) + ] + }, + "CodingThrust/problem-reductions", + fake_review_fetcher, + fake_pr_resolver, + fake_pr_info_fetcher, + ) + + self.assertEqual(candidates[0]["eligibility"], "waiting-for-copilot") + self.assertEqual(candidates[0]["reason"], "open PR #570 waiting for Copilot review") + + +class PipelineBoardStatusListTests(unittest.TestCase): + def test_status_items_list_ready_issues(self) -> None: + items = status_items( + { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="Ready", + title="[Model] ExactCoverBy3Sets", + ), + make_issue_item( + "PVTI_2", + 102, + status="In progress", + title="[Rule] A to B", + ), + ] + }, + STATUS_READY, + ) + self.assertEqual( + items, + [ + { + "item_id": "PVTI_1", + "number": 101, + "issue_number": 101, + "pr_number": None, + "status": STATUS_READY, + "title": "[Model] ExactCoverBy3Sets", + } + ], + ) + + def test_status_items_list_in_progress_issues(self) -> None: + items = status_items( + { + "items": [ + make_issue_item( + "PVTI_1", + 101, + status="Ready", + title="[Model] ExactCoverBy3Sets", + ), + make_issue_item( + "PVTI_2", + 102, + status="In progress", + title="[Rule] A to B", + ), + ] + }, + STATUS_IN_PROGRESS, + ) + self.assertEqual( + items, + [ + { + "item_id": "PVTI_2", + "number": 102, + "issue_number": 102, + "pr_number": None, + "status": STATUS_IN_PROGRESS, + "title": "[Rule] A to B", + } + ], + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_pipeline_checks.py b/scripts/test_pipeline_checks.py new file mode 100644 index 000000000..4c240086e --- /dev/null +++ b/scripts/test_pipeline_checks.py @@ -0,0 +1,521 @@ +#!/usr/bin/env python3 +import tempfile +import unittest +from pathlib import Path + +import pipeline_checks +from pipeline_checks import ( + build_review_context, + completeness_check, + detect_scope_from_paths, + file_whitelist_check, + infer_review_subject, + issue_guard_check, + parse_args, +) + + +class PipelineChecksTests(unittest.TestCase): + def test_detect_scope_reports_model_review_for_new_model_file(self) -> None: + scope = detect_scope_from_paths( + added_files=["src/models/graph/graph_partitioning.rs"], + changed_files=[ + "src/models/graph/graph_partitioning.rs", + "src/unit_tests/models/graph/graph_partitioning.rs", + ], + ) + + self.assertEqual(scope["review_type"], "model") + self.assertEqual(scope["models"][0]["category"], "graph") + self.assertEqual(scope["models"][0]["file_stem"], "graph_partitioning") + self.assertEqual(scope["models"][0]["problem_name"], "GraphPartitioning") + + def test_detect_scope_reports_rule_review_for_new_rule_file(self) -> None: + scope = detect_scope_from_paths( + added_files=["src/rules/binpacking_ilp.rs"], + changed_files=["src/rules/binpacking_ilp.rs"], + ) + + self.assertEqual(scope["review_type"], "rule") + self.assertEqual(scope["rules"][0]["rule_stem"], "binpacking_ilp") + + def test_detect_scope_reports_generic_when_no_new_model_or_rule_files(self) -> None: + scope = detect_scope_from_paths( + added_files=[], + changed_files=["src/lib.rs", "docs/paper/reductions.typ"], + ) + + self.assertEqual(scope["review_type"], "generic") + self.assertEqual(scope["models"], []) + self.assertEqual(scope["rules"], []) + + def test_infer_review_subject_prefers_explicit_rule_metadata(self) -> None: + scope = { + "review_type": "rule", + "models": [], + "rules": [{"rule_stem": "binpacking_ilp"}], + "changed_files": ["src/rules/binpacking_ilp.rs"], + } + + subject = infer_review_subject( + scope, + kind="rule", + name="binpacking_ilp", + source="BinPacking", + target="ILP", + ) + + self.assertEqual(subject["kind"], "rule") + self.assertEqual(subject["name"], "binpacking_ilp") + self.assertEqual(subject["source"], "BinPacking") + self.assertEqual(subject["target"], "ILP") + self.assertFalse(subject["inferred"]) + + def test_infer_review_subject_infers_model_from_scope(self) -> None: + scope = { + "review_type": "model", + "models": [{"problem_name": "GraphPartitioning"}], + "rules": [], + "changed_files": ["src/models/graph/graph_partitioning.rs"], + } + + subject = infer_review_subject(scope) + + self.assertEqual(subject["kind"], "model") + self.assertEqual(subject["name"], "GraphPartitioning") + self.assertTrue(subject["inferred"]) + + def test_file_whitelist_accepts_expected_model_files(self) -> None: + report = file_whitelist_check( + "model", + [ + "src/models/graph/graph_partitioning.rs", + "src/unit_tests/models/graph/graph_partitioning.rs", + "src/example_db/model_builders.rs", + "docs/paper/reductions.typ", + "docs/src/reductions/problem_schemas.json", + "tests/suites/trait_consistency.rs", + ], + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["violations"], []) + + def test_file_whitelist_flags_unexpected_model_files(self) -> None: + report = file_whitelist_check( + "model", + [ + "src/models/graph/graph_partitioning.rs", + "Cargo.toml", + ], + ) + + self.assertFalse(report["ok"]) + self.assertEqual(report["violations"][0]["path"], "Cargo.toml") + + def test_file_whitelist_accepts_expected_rule_files(self) -> None: + report = file_whitelist_check( + "rule", + [ + "src/rules/binpacking_ilp.rs", + "src/rules/mod.rs", + "src/unit_tests/rules/binpacking_ilp.rs", + "src/example_db/rule_builders.rs", + "src/models/graph/bin_packing.rs", + "docs/paper/reductions.typ", + "docs/src/reductions/reduction_graph.json", + ], + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["violations"], []) + + def test_model_completeness_reports_all_required_components(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/models/graph/graph_partitioning.rs", + """ + inventory::submit! { ProblemSchemaEntry { name: "GraphPartitioning" } } + impl OptimizationProblem for GraphPartitioning<SimpleGraph> {} + crate::declare_variants! { opt GraphPartitioning<SimpleGraph> => "1.2^n" } + pub(crate) fn canonical_model_example_specs() -> Vec<ModelExampleSpec> { vec![] } + """, + ) + self._write( + repo / "src/unit_tests/models/graph/graph_partitioning.rs", + "#[test]\nfn test_graph_partitioning_basic() {}\n", + ) + self._write( + repo / "src/unit_tests/trait_consistency.rs", + """ + fn test_all_problems_implement_trait_correctly() { + check_problem_trait(&GraphPartitioning::new(), "GraphPartitioning"); + } + fn test_direction() { + let _ = GraphPartitioning::new().direction(); + } + """, + ) + self._write(repo / "src/example_db/model_builders.rs", "pub fn build_model_examples() {}\n") + self._write( + repo / "docs/paper/reductions.typ", + """ + #let display-name = ( + "GraphPartitioning": [Graph Partitioning], + ) + #problem-def("GraphPartitioning")[body][proof] + """, + ) + + report = completeness_check("model", repo, name="GraphPartitioning") + + self.assertTrue(report["ok"]) + self.assertEqual(report["missing"], []) + self.assertEqual(report["checks"]["paper_display_name"]["status"], "pass") + self.assertEqual(report["checks"]["trait_direction"]["status"], "pass") + + def test_model_completeness_flags_missing_paper_and_trait_entries(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/models/graph/graph_partitioning.rs", + """ + inventory::submit! { ProblemSchemaEntry { name: "GraphPartitioning" } } + impl OptimizationProblem for GraphPartitioning<SimpleGraph> {} + crate::declare_variants! { opt GraphPartitioning<SimpleGraph> => "1.2^n" } + pub(crate) fn canonical_model_example_specs() -> Vec<ModelExampleSpec> { vec![] } + """, + ) + self._write( + repo / "src/unit_tests/models/graph/graph_partitioning.rs", + "#[test]\nfn test_graph_partitioning_basic() {}\n", + ) + self._write(repo / "src/unit_tests/trait_consistency.rs", "fn test_direction() {}\n") + self._write(repo / "src/example_db/model_builders.rs", "pub fn build_model_examples() {}\n") + self._write(repo / "docs/paper/reductions.typ", "#let display-name = ()\n") + + report = completeness_check("model", repo, name="GraphPartitioning") + + self.assertFalse(report["ok"]) + self.assertIn("paper_definition", report["missing"]) + self.assertIn("paper_display_name", report["missing"]) + self.assertIn("trait_consistency", report["missing"]) + + def test_rule_completeness_reports_all_required_components(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/rules/binpacking_ilp.rs", + """ + #[reduction(overhead = { num_vars = "num_items" })] + impl ReduceTo<ILP> for BinPacking {} + pub(crate) fn canonical_rule_example_specs() -> Vec<RuleExampleSpec> { vec![] } + """, + ) + self._write(repo / "src/rules/mod.rs", "mod binpacking_ilp;\n") + self._write( + repo / "src/unit_tests/rules/binpacking_ilp.rs", + "#[test]\nfn test_binpacking_to_ilp_closed_loop() {}\n", + ) + self._write(repo / "src/example_db/rule_builders.rs", "pub fn build_rule_examples() {}\n") + self._write( + repo / "docs/paper/reductions.typ", + '#reduction-rule("BinPacking", "ILP")[rule][proof]\n', + ) + + report = completeness_check( + "rule", + repo, + name="binpacking_ilp", + source="BinPacking", + target="ILP", + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["checks"]["module_registration"]["status"], "pass") + self.assertEqual(report["checks"]["paper_rule"]["status"], "pass") + + def test_rule_completeness_flags_missing_overhead_and_paper(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/rules/binpacking_ilp.rs", + """ + #[reduction] + impl ReduceTo<ILP> for BinPacking {} + """, + ) + self._write(repo / "src/rules/mod.rs", "") + self._write( + repo / "src/unit_tests/rules/binpacking_ilp.rs", + "#[test]\nfn test_binpacking_to_ilp_closed_loop() {}\n", + ) + self._write(repo / "src/example_db/rule_builders.rs", "pub fn build_rule_examples() {}\n") + self._write(repo / "docs/paper/reductions.typ", "") + + report = completeness_check( + "rule", + repo, + name="binpacking_ilp", + source="BinPacking", + target="ILP", + ) + + self.assertFalse(report["ok"]) + self.assertIn("overhead_form", report["missing"]) + self.assertIn("paper_rule", report["missing"]) + self.assertIn("module_registration", report["missing"]) + + def test_issue_guards_pass_for_good_model_issue_without_existing_pr(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + report = issue_guard_check( + repo, + issue={ + "number": 117, + "title": "[Model] GraphPartitioning", + "body": "Implement the model.", + "state": "OPEN", + "url": "https://example.test/issues/117", + "labels": [{"name": "Good"}], + "comments": [ + { + "author": {"login": "maintainer"}, + "body": "Use the paper notation.", + } + ], + }, + existing_prs=[], + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["kind"], "model") + self.assertEqual(report["checks"]["good_label"]["status"], "pass") + self.assertEqual(report["checks"]["source_model"]["status"], "skip") + self.assertEqual(report["comments"][0]["author"], "maintainer") + self.assertEqual(report["action"], "create-pr") + + def test_issue_guards_fail_when_good_label_missing(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + report = issue_guard_check( + repo, + issue={ + "number": 118, + "title": "[Model] GraphPartitioning", + "body": "Implement the model.", + "state": "OPEN", + "url": "https://example.test/issues/118", + "labels": [{"name": "NeedsCheck"}], + "comments": [], + }, + existing_prs=[], + ) + + self.assertFalse(report["ok"]) + self.assertIn("good_label", report["missing"]) + self.assertEqual(report["checks"]["good_label"]["status"], "fail") + + def test_issue_guards_fail_rule_issue_when_target_model_missing(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/models/misc/bin_packing.rs", + "pub struct BinPacking;\n", + ) + + report = issue_guard_check( + repo, + issue={ + "number": 119, + "title": "[Rule] BinPacking to ILP", + "body": "Implement the reduction.", + "state": "OPEN", + "url": "https://example.test/issues/119", + "labels": [{"name": "Good"}], + "comments": [], + }, + existing_prs=[], + ) + + self.assertFalse(report["ok"]) + self.assertEqual(report["kind"], "rule") + self.assertEqual(report["source_problem"], "BinPacking") + self.assertEqual(report["target_problem"], "ILP") + self.assertEqual(report["checks"]["source_model"]["status"], "pass") + self.assertEqual(report["checks"]["target_model"]["status"], "fail") + self.assertIn("target_model", report["missing"]) + + def test_issue_guards_report_existing_open_pr_for_resume(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + report = issue_guard_check( + repo, + issue={ + "number": 120, + "title": "[Model] GraphPartitioning", + "body": "Implement the model.", + "state": "OPEN", + "url": "https://example.test/issues/120", + "labels": [{"name": "Good"}], + "comments": [], + }, + existing_prs=[ + { + "number": 650, + "headRefName": "issue-120-graph-partitioning", + "url": "https://example.test/pull/650", + } + ], + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["action"], "resume-pr") + self.assertEqual(report["resume_pr"]["number"], 650) + self.assertEqual(report["resume_pr"]["head_ref_name"], "issue-120-graph-partitioning") + + def test_issue_context_alias_matches_issue_guard_contract(self) -> None: + issue_context_check = getattr(pipeline_checks, "issue_context_check", None) + self.assertIsNotNone(issue_context_check) + + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + report = issue_context_check( + repo, + issue={ + "number": 121, + "title": "[Model] ExactCoverBy3Sets", + "body": "Implement the model.", + "state": "OPEN", + "url": "https://example.test/issues/121", + "labels": [{"name": "Good"}], + "comments": [], + }, + existing_prs=[], + ) + + self.assertTrue(report["ok"]) + self.assertEqual(report["issue_number"], 121) + self.assertEqual(report["kind"], "model") + self.assertEqual(report["action"], "create-pr") + + def test_build_review_context_reports_model_bundle(self) -> None: + with tempfile.TemporaryDirectory() as tmpdir: + repo = Path(tmpdir) + self._write( + repo / "src/models/graph/graph_partitioning.rs", + """ + inventory::submit! { ProblemSchemaEntry { name: "GraphPartitioning" } } + impl OptimizationProblem for GraphPartitioning<SimpleGraph> {} + crate::declare_variants! { opt GraphPartitioning<SimpleGraph> => "1.2^n" } + pub(crate) fn canonical_model_example_specs() -> Vec<ModelExampleSpec> { vec![] } + """, + ) + self._write( + repo / "src/unit_tests/models/graph/graph_partitioning.rs", + "#[test]\nfn test_graph_partitioning_basic() {}\n", + ) + self._write( + repo / "src/unit_tests/trait_consistency.rs", + """ + fn test_all_problems_implement_trait_correctly() { + check_problem_trait(&GraphPartitioning::new(), "GraphPartitioning"); + } + fn test_direction() { + let _ = GraphPartitioning::new().direction(); + } + """, + ) + self._write( + repo / "docs/paper/reductions.typ", + """ + #let display-name = ( + "GraphPartitioning": [Graph Partitioning], + ) + #problem-def("GraphPartitioning")[body][proof] + """, + ) + scope = detect_scope_from_paths( + added_files=["src/models/graph/graph_partitioning.rs"], + changed_files=[ + "src/models/graph/graph_partitioning.rs", + "src/unit_tests/models/graph/graph_partitioning.rs", + "docs/paper/reductions.typ", + ], + ) + + context = build_review_context( + repo, + diff_stat=" 3 files changed, 20 insertions(+)\n", + scope=scope, + subject=infer_review_subject(scope), + ) + + self.assertEqual(context["subject"]["kind"], "model") + self.assertFalse(context["whitelist"]["skipped"]) + self.assertTrue(context["whitelist"]["ok"]) + self.assertFalse(context["completeness"]["skipped"]) + self.assertTrue(context["completeness"]["ok"]) + self.assertIn("GraphPartitioning", context["subject"]["name"]) + + def test_build_review_context_skips_checks_for_generic_scope(self) -> None: + scope = detect_scope_from_paths( + added_files=[], + changed_files=["src/lib.rs", "README.md"], + ) + + context = build_review_context( + ".", + diff_stat=" 2 files changed, 3 insertions(+)\n", + scope=scope, + subject=infer_review_subject(scope), + ) + + self.assertEqual(context["subject"]["kind"], "generic") + self.assertTrue(context["whitelist"]["skipped"]) + self.assertTrue(context["completeness"]["skipped"]) + + def test_parse_args_accepts_review_context(self) -> None: + args = parse_args( + [ + "review-context", + "--repo-root", + ".", + "--base", + "abc123", + "--head", + "def456", + "--format", + "json", + ] + ) + + self.assertEqual(args.command, "review-context") + self.assertEqual(args.base, "abc123") + self.assertEqual(args.head, "def456") + + def test_parse_args_accepts_issue_context(self) -> None: + args = parse_args( + [ + "issue-context", + "--repo", + "CodingThrust/problem-reductions", + "--issue", + "117", + "--format", + "json", + ] + ) + + self.assertEqual(args.command, "issue-context") + self.assertEqual(args.repo, "CodingThrust/problem-reductions") + self.assertEqual(args.issue, 117) + + def _write(self, path: Path, content: str) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content.strip() + "\n") + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_pipeline_pr.py b/scripts/test_pipeline_pr.py new file mode 100644 index 000000000..73d0c10ad --- /dev/null +++ b/scripts/test_pipeline_pr.py @@ -0,0 +1,614 @@ +#!/usr/bin/env python3 +import io +import unittest +from contextlib import redirect_stdout +from unittest import mock + +from pipeline_pr import ( + build_current_pr_context, + build_context_result, + build_pr_context, + build_linked_issue_result, + build_snapshot, + create_pr, + emit_result, + edit_pr_body, + extract_codecov_summary, + extract_linked_issue_number, + format_issue_context, + post_pr_comment, + parse_args, + summarize_check_runs, + summarize_comments, + wait_for_ci, +) + + +def make_inline_comment( + login: str, + *, + body: str = "comment", + path: str = "src/lib.rs", + line: int = 7, +) -> dict: + return { + "user": {"login": login}, + "body": body, + "path": path, + "line": line, + "original_line": line, + } + + +def make_review( + login: str, + *, + body: str = "review body", + state: str = "COMMENTED", +) -> dict: + return { + "user": {"login": login}, + "body": body, + "state": state, + } + + +def make_issue_comment(login: str, *, body: str = "discussion") -> dict: + return { + "user": {"login": login}, + "body": body, + } + + +def make_check_run( + name: str, + *, + status: str = "completed", + conclusion: str | None = "success", +) -> dict: + return { + "name": name, + "status": status, + "conclusion": conclusion, + } + + +class PipelinePrHelpersTests(unittest.TestCase): + def test_extract_linked_issue_number_prefers_body_over_title(self) -> None: + linked_issue = extract_linked_issue_number( + "Fix #117: Add GraphPartitioning model", + "This supersedes older work and closes #42.", + ) + self.assertEqual(linked_issue, 42) + + def test_extract_linked_issue_number_falls_back_to_title(self) -> None: + linked_issue = extract_linked_issue_number( + "Fix #117: Add GraphPartitioning model", + "No explicit closing keyword here.", + ) + self.assertEqual(linked_issue, 117) + + def test_summarize_comments_splits_human_copilot_and_codecov_sources(self) -> None: + summary = summarize_comments( + inline_comments=[ + make_inline_comment("copilot-pull-request-reviewer[bot]"), + make_inline_comment("alice"), + ], + reviews=[ + make_review("bob", body="Please add tests"), + make_review("copilot-pull-request-reviewer[bot]", body="bot review"), + make_review("carol", body=""), + ], + issue_comments=[ + make_issue_comment("dave", body="Please update docs"), + make_issue_comment("codecov[bot]", body="## [Codecov] Patch coverage is 82%"), + ], + linked_issue_comments=[ + make_issue_comment("erin", body="The literature citation is important"), + make_issue_comment("deploy-bot[bot]", body="automated deployment note"), + ], + ) + + self.assertEqual(summary["counts"]["inline_comments"], 2) + self.assertEqual(summary["counts"]["copilot_inline_comments"], 1) + self.assertEqual(summary["counts"]["human_inline_comments"], 1) + self.assertEqual(summary["counts"]["human_reviews"], 1) + self.assertEqual(summary["counts"]["human_issue_comments"], 1) + self.assertEqual(summary["counts"]["human_linked_issue_comments"], 1) + self.assertEqual(summary["counts"]["codecov_comments"], 1) + + def test_extract_codecov_summary_parses_latest_comment_and_filepaths(self) -> None: + summary = extract_codecov_summary( + [ + make_issue_comment("alice", body="looks fine"), + make_issue_comment( + "codecov[bot]", + body=( + "## [Codecov]\n" + "Patch coverage: `84.21%`\n" + "Project coverage is `91.30%`\n" + "https://codecov.io/gh/CodingThrust/problem-reductions?" + "filepath=src%2Fmodels%2Fgraph%2Ffoo.rs&line=17\n" + "https://codecov.io/gh/CodingThrust/problem-reductions?" + "filepath=src%2Fmodels%2Fgraph%2Ffoo.rs&line=21\n" + "https://codecov.io/gh/CodingThrust/problem-reductions?" + "filepath=src%2Frules%2Ffoo_bar.rs&line=8\n" + ), + ), + ] + ) + + self.assertTrue(summary["found"]) + self.assertEqual(summary["patch_coverage"], 84.21) + self.assertEqual(summary["project_coverage"], 91.30) + self.assertEqual( + summary["filepaths"], + [ + "src/models/graph/foo.rs", + "src/rules/foo_bar.rs", + ], + ) + + def test_summarize_check_runs_reports_overall_state(self) -> None: + self.assertEqual( + summarize_check_runs({"check_runs": [make_check_run("test", status="queued", conclusion=None)]})["state"], + "pending", + ) + self.assertEqual( + summarize_check_runs({"check_runs": [make_check_run("test", conclusion="failure")]})["state"], + "failure", + ) + self.assertEqual( + summarize_check_runs( + { + "check_runs": [ + make_check_run("fmt"), + make_check_run("coverage", conclusion="neutral"), + ] + } + )["state"], + "success", + ) + + def test_build_snapshot_includes_linked_issue_ci_and_codecov(self) -> None: + snapshot = build_snapshot( + { + "number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "body": "Closes #117", + "state": "OPEN", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + "headRefName": "feature/graph-partitioning", + "baseRefName": "main", + "mergeable": "MERGEABLE", + "headRefOid": "abc123", + "labels": [{"name": "model"}], + "files": [{"path": "src/models/graph/graph_partitioning.rs"}], + "commits": [{"oid": "abc123"}, {"oid": "def456"}], + "additions": 120, + "deletions": 7, + }, + linked_issue_number=117, + linked_issue={ + "number": 117, + "title": "[Model] GraphPartitioning", + "state": "OPEN", + }, + ci_summary={"state": "success", "total": 3, "failing": 0, "pending": 0}, + codecov_summary={"found": True, "patch_coverage": 84.21, "filepaths": ["src/models/graph/graph_partitioning.rs"]}, + ) + + self.assertEqual(snapshot["number"], 570) + self.assertEqual(snapshot["linked_issue_number"], 117) + self.assertEqual(snapshot["linked_issue"]["title"], "[Model] GraphPartitioning") + self.assertEqual(snapshot["ci"]["state"], "success") + self.assertEqual(snapshot["codecov"]["patch_coverage"], 84.21) + self.assertEqual(snapshot["counts"]["files"], 1) + self.assertEqual(snapshot["counts"]["commits"], 2) + + def test_build_current_pr_context_includes_repo_and_pr_fields(self) -> None: + current = build_current_pr_context( + "CodingThrust/problem-reductions", + { + "number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "headRefName": "feature/graph-partitioning", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + }, + ) + + self.assertEqual(current["repo"], "CodingThrust/problem-reductions") + self.assertEqual(current["pr_number"], 570) + self.assertEqual(current["title"], "Fix #117: Add GraphPartitioning model") + self.assertEqual(current["head_ref_name"], "feature/graph-partitioning") + + def test_format_issue_context_includes_title_body_and_comments(self) -> None: + issue_context = format_issue_context( + { + "number": 117, + "title": "[Model] GraphPartitioning", + "body": "Implement the model.", + "state": "OPEN", + }, + [ + { + "author": "maintainer", + "body": "Use the paper notation.", + "created_at": "2026-03-15T09:00:00Z", + "is_bot": False, + } + ], + ) + + self.assertIn("# [Model] GraphPartitioning", issue_context) + self.assertIn("Implement the model.", issue_context) + self.assertIn("## Comments", issue_context) + self.assertIn("**maintainer** (2026-03-15T09:00:00Z):", issue_context) + + def test_build_linked_issue_result_includes_normalized_comments_and_context(self) -> None: + result = build_linked_issue_result( + pr_number=570, + linked_issue_number=117, + linked_issue={ + "number": 117, + "title": "[Model] GraphPartitioning", + "body": "Implement the model.", + "state": "OPEN", + "url": "https://github.com/CodingThrust/problem-reductions/issues/117", + }, + linked_issue_comments=[ + { + "user": {"login": "maintainer"}, + "body": "Use the paper notation.", + "created_at": "2026-03-15T09:00:00Z", + }, + { + "user": {"login": "deploy-bot[bot]"}, + "body": "Automated message.", + "created_at": "2026-03-15T09:05:00Z", + }, + ], + ) + + self.assertEqual(result["pr_number"], 570) + self.assertEqual(result["linked_issue_number"], 117) + self.assertEqual(len(result["linked_issue_comments"]), 2) + self.assertEqual(result["human_linked_issue_comments"][0]["author"], "maintainer") + self.assertIn("# [Model] GraphPartitioning", result["issue_context_text"]) + self.assertIn("**maintainer** (2026-03-15T09:00:00Z):", result["issue_context_text"]) + + def test_build_context_result_merges_snapshot_comments_and_issue_context(self) -> None: + snapshot = { + "number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "body": "Closes #117", + "state": "OPEN", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + "mergeable": "MERGEABLE", + "head_ref_name": "feature/graph-partitioning", + "base_ref_name": "main", + "head_sha": "abc123", + "linked_issue_number": 117, + "linked_issue": {"number": 117, "title": "[Model] GraphPartitioning"}, + "files": ["src/models/graph/graph_partitioning.rs"], + "commits": ["abc123", "def456"], + "ci": {"state": "success"}, + "codecov": {"found": True, "patch_coverage": 92.0}, + "counts": {"files": 1, "commits": 2}, + } + comments = { + "inline_comments": [{"user": "alice", "body": "nit"}], + "reviews": [{"user": "bob", "body": "looks good", "state": "COMMENTED"}], + "human_issue_comments": [{"user": "carol", "body": "please update docs"}], + "counts": {"inline_comments": 1}, + } + linked_issue_result = { + "linked_issue_number": 117, + "linked_issue": {"number": 117, "title": "[Model] GraphPartitioning"}, + "linked_issue_comments": [{"author": "maintainer", "body": "Use paper notation"}], + "human_linked_issue_comments": [{"author": "maintainer", "body": "Use paper notation"}], + "issue_context_text": "# [Model] GraphPartitioning\n\nImplement the model.", + } + + context = build_context_result( + "CodingThrust/problem-reductions", + snapshot, + comments, + linked_issue_result, + ) + + self.assertEqual(context["repo"], "CodingThrust/problem-reductions") + self.assertEqual(context["pr_number"], 570) + self.assertEqual(context["title"], "Fix #117: Add GraphPartitioning model") + self.assertEqual(context["comments"]["inline_comments"][0]["user"], "alice") + self.assertEqual(context["ci"]["state"], "success") + self.assertEqual(context["codecov"]["patch_coverage"], 92.0) + self.assertEqual(context["linked_issue_number"], 117) + self.assertIn("GraphPartitioning", context["issue_context_text"]) + + def test_emit_result_prints_context_text_report(self) -> None: + context = { + "repo": "CodingThrust/problem-reductions", + "pr_number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + "head_sha": "abc123", + "comments": { + "counts": { + "copilot_inline_comments": 2, + "human_inline_comments": 1, + "human_issue_comments": 1, + "human_linked_issue_comments": 1, + "human_reviews": 1, + } + }, + "ci": {"state": "failure", "failing": 1, "pending": 0}, + "codecov": { + "found": True, + "patch_coverage": 84.21, + "project_coverage": 91.3, + "filepaths": ["src/models/graph/graph_partitioning.rs"], + }, + "linked_issue_number": 117, + "issue_context_text": "# [Model] GraphPartitioning\n\nImplement the model.", + } + + stdout = io.StringIO() + with redirect_stdout(stdout): + emit_result(context, "text") + + rendered = stdout.getvalue() + self.assertIn("# PR Context Packet", rendered) + self.assertIn("- PR: #570", rendered) + self.assertIn("- Repo: CodingThrust/problem-reductions", rendered) + self.assertIn("- Head SHA: `abc123`", rendered) + self.assertIn("## Comment Summary", rendered) + self.assertIn("- Copilot inline comments: 2", rendered) + self.assertIn("## CI Summary", rendered) + self.assertIn("- State: failure", rendered) + self.assertIn("## Codecov", rendered) + self.assertIn("- Patch coverage: 84.21%", rendered) + self.assertIn("## Linked Issue Context", rendered) + + @mock.patch("pipeline_pr.build_linked_issue_context") + @mock.patch("pipeline_pr.build_comments_summary") + @mock.patch("pipeline_pr.build_pr_snapshot") + def test_build_pr_context_assembles_existing_helper_results( + self, + build_pr_snapshot: mock.Mock, + build_comments_summary: mock.Mock, + build_linked_issue_context: mock.Mock, + ) -> None: + build_pr_snapshot.return_value = { + "number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "body": "Closes #117", + "state": "OPEN", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + "mergeable": "MERGEABLE", + "head_ref_name": "feature/graph-partitioning", + "base_ref_name": "main", + "head_sha": "abc123", + "linked_issue_number": 117, + "linked_issue": {"number": 117, "title": "[Model] GraphPartitioning"}, + "files": ["src/models/graph/graph_partitioning.rs"], + "commits": ["abc123"], + "ci": {"state": "success"}, + "codecov": {"found": True}, + "counts": {"files": 1, "commits": 1}, + } + build_comments_summary.return_value = {"inline_comments": [], "counts": {"inline_comments": 0}} + build_linked_issue_context.return_value = { + "linked_issue_number": 117, + "linked_issue": {"number": 117, "title": "[Model] GraphPartitioning"}, + "linked_issue_comments": [], + "human_linked_issue_comments": [], + "issue_context_text": "# [Model] GraphPartitioning", + } + + context = build_pr_context("CodingThrust/problem-reductions", 570) + + build_pr_snapshot.assert_called_once_with("CodingThrust/problem-reductions", 570) + build_comments_summary.assert_called_once_with("CodingThrust/problem-reductions", 570) + build_linked_issue_context.assert_called_once() + self.assertEqual(context["pr_number"], 570) + self.assertEqual(context["issue_context_text"], "# [Model] GraphPartitioning") + + def test_wait_for_ci_polls_until_terminal_state(self) -> None: + summaries = [ + {"state": "pending", "total": 2, "pending": 1, "failing": 0}, + {"state": "pending", "total": 2, "pending": 1, "failing": 0}, + {"state": "success", "total": 2, "pending": 0, "failing": 0}, + ] + sleeps: list[float] = [] + now = [0.0] + + def fake_fetcher() -> dict: + return summaries.pop(0) + + def fake_monotonic() -> float: + return now[0] + + def fake_sleep(seconds: float) -> None: + sleeps.append(seconds) + now[0] += seconds + + result = wait_for_ci( + fake_fetcher, + timeout_seconds=30, + interval_seconds=5, + monotonic_fn=fake_monotonic, + sleep_fn=fake_sleep, + ) + + self.assertEqual(result["state"], "success") + self.assertEqual(sleeps, [5, 5]) + + @mock.patch("pipeline_pr.subprocess.check_call") + def test_post_pr_comment_uses_gh_pr_comment_with_body_file(self, check_call: mock.Mock) -> None: + post_pr_comment( + "CodingThrust/problem-reductions", + 570, + "/tmp/comment.md", + ) + + check_call.assert_called_once_with( + [ + "gh", + "pr", + "comment", + "570", + "--repo", + "CodingThrust/problem-reductions", + "--body-file", + "/tmp/comment.md", + ] + ) + + @mock.patch("pipeline_pr.subprocess.check_call") + def test_edit_pr_body_uses_gh_pr_edit_with_body_file(self, check_call: mock.Mock) -> None: + edit_pr_body( + "CodingThrust/problem-reductions", + 570, + "/tmp/body.md", + ) + + check_call.assert_called_once_with( + [ + "gh", + "pr", + "edit", + "570", + "--repo", + "CodingThrust/problem-reductions", + "--body-file", + "/tmp/body.md", + ] + ) + + @mock.patch("pipeline_pr.fetch_current_pr_data_for_repo") + @mock.patch("pipeline_pr.run_gh_checked") + def test_create_pr_uses_gh_pr_create_and_returns_current_context( + self, + run_gh_checked: mock.Mock, + fetch_current_pr_data_for_repo: mock.Mock, + ) -> None: + fetch_current_pr_data_for_repo.return_value = { + "number": 570, + "title": "Fix #117: Add GraphPartitioning model", + "headRefName": "issue-117-graph-partitioning", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + } + + result = create_pr( + "CodingThrust/problem-reductions", + "Fix #117: Add GraphPartitioning model", + "/tmp/pr-body.md", + base="main", + head="issue-117-graph-partitioning", + ) + + run_gh_checked.assert_called_once_with( + "pr", + "create", + "--repo", + "CodingThrust/problem-reductions", + "--title", + "Fix #117: Add GraphPartitioning model", + "--body-file", + "/tmp/pr-body.md", + "--base", + "main", + "--head", + "issue-117-graph-partitioning", + ) + fetch_current_pr_data_for_repo.assert_called_once_with("CodingThrust/problem-reductions") + self.assertEqual(result["pr_number"], 570) + self.assertEqual(result["repo"], "CodingThrust/problem-reductions") + + def test_parse_args_accepts_comment_and_edit_body_commands(self) -> None: + comment_args = parse_args( + [ + "comment", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--body-file", + "/tmp/comment.md", + ] + ) + self.assertEqual(comment_args.command, "comment") + self.assertEqual(comment_args.body_file, "/tmp/comment.md") + + edit_args = parse_args( + [ + "edit-body", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--body-file", + "/tmp/body.md", + ] + ) + self.assertEqual(edit_args.command, "edit-body") + self.assertEqual(edit_args.body_file, "/tmp/body.md") + + current_args = parse_args(["current", "--format", "json"]) + self.assertEqual(current_args.command, "current") + + linked_issue_args = parse_args( + [ + "linked-issue", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--format", + "json", + ] + ) + self.assertEqual(linked_issue_args.command, "linked-issue") + + create_args = parse_args( + [ + "create", + "--repo", + "CodingThrust/problem-reductions", + "--title", + "Fix #117: Add GraphPartitioning model", + "--body-file", + "/tmp/pr-body.md", + "--base", + "main", + "--head", + "issue-117-graph-partitioning", + "--format", + "json", + ] + ) + self.assertEqual(create_args.command, "create") + self.assertEqual(create_args.body_file, "/tmp/pr-body.md") + + context_args = parse_args( + [ + "context", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "570", + "--format", + "json", + ] + ) + self.assertEqual(context_args.command, "context") + self.assertEqual(context_args.pr, 570) + + current_context_args = parse_args(["context", "--current", "--format", "json"]) + self.assertEqual(current_context_args.command, "context") + self.assertTrue(current_context_args.current) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_pipeline_skill_context.py b/scripts/test_pipeline_skill_context.py new file mode 100644 index 000000000..8aafe8039 --- /dev/null +++ b/scripts/test_pipeline_skill_context.py @@ -0,0 +1,870 @@ +#!/usr/bin/env python3 +import io +import json +import unittest +from contextlib import redirect_stdout +from pathlib import Path +from unittest import mock + +import pipeline_skill_context + + +class PipelineSkillContextTests(unittest.TestCase): + def test_parse_args_review_pipeline_defaults(self) -> None: + args = pipeline_skill_context.parse_args( + [ + "review-pipeline", + "--repo", + "CodingThrust/problem-reductions", + ] + ) + + self.assertEqual(args.command, "review-pipeline") + self.assertEqual(args.repo, "CodingThrust/problem-reductions") + self.assertIsNone(args.pr) + self.assertEqual( + args.state_file, + Path("/tmp/problemreductions-review-state.json"), + ) + self.assertEqual(args.format, "json") + + def test_parse_args_final_review_with_explicit_values(self) -> None: + args = pipeline_skill_context.parse_args( + [ + "final-review", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "615", + "--state-file", + "/tmp/custom-final-review-state.json", + "--format", + "text", + ] + ) + + self.assertEqual(args.command, "final-review") + self.assertEqual(args.repo, "CodingThrust/problem-reductions") + self.assertEqual(args.pr, 615) + self.assertEqual( + args.state_file, + Path("/tmp/custom-final-review-state.json"), + ) + self.assertEqual(args.format, "text") + + def test_parse_args_review_implementation_defaults(self) -> None: + args = pipeline_skill_context.parse_args( + [ + "review-implementation", + "--format", + "text", + ] + ) + + self.assertEqual(args.command, "review-implementation") + self.assertEqual(args.repo_root, Path(".")) + self.assertIsNone(args.kind) + self.assertEqual(args.format, "text") + + def test_parse_args_project_pipeline_with_explicit_values(self) -> None: + args = pipeline_skill_context.parse_args( + [ + "project-pipeline", + "--repo", + "CodingThrust/problem-reductions", + "--issue", + "117", + "--repo-root", + "/tmp/repo", + "--format", + "text", + ] + ) + + self.assertEqual(args.command, "project-pipeline") + self.assertEqual(args.repo, "CodingThrust/problem-reductions") + self.assertEqual(args.issue, 117) + self.assertEqual(args.repo_root, Path("/tmp/repo")) + self.assertEqual(args.format, "text") + + def test_emit_result_prints_sorted_json_for_all_formats(self) -> None: + expected_output = '{\n "a": 2,\n "b": 1\n}\n' + + for fmt in ["json"]: + with self.subTest(fmt=fmt): + stdout = io.StringIO() + with redirect_stdout(stdout): + pipeline_skill_context.emit_result({"b": 1, "a": 2}, fmt) + self.assertEqual(stdout.getvalue(), expected_output) + + def test_emit_result_prints_final_review_text_report(self) -> None: + result = { + "skill": "final-review", + "status": "ready", + "selection": { + "item_id": "PVTI_22", + "pr_number": 615, + "issue_number": 117, + "title": "[Model] GraphPartitioning", + }, + "pr": { + "number": 615, + "title": "Fix #117: [Model] GraphPartitioning", + "url": "https://github.com/CodingThrust/problem-reductions/pull/615", + "comments": { + "counts": { + "human_inline_comments": 1, + "human_issue_comments": 2, + "human_linked_issue_comments": 1, + "human_reviews": 1, + } + }, + "issue_context_text": "Issue #117: Add GraphPartitioning model", + }, + "prep": { + "ready": False, + "checkout": {"worktree_dir": "/tmp/final-pr-615"}, + "merge": {"status": "conflicted", "conflicts": ["src/models/graph_partitioning.rs"]}, + }, + "review_context": { + "subject": {"kind": "model", "name": "GraphPartitioning"}, + "whitelist": {"ok": True, "violations": []}, + "completeness": {"ok": False, "missing": ["paper_display_name"]}, + "changed_files": ["src/models/graph_partitioning.rs", "docs/paper/reductions.typ"], + "diff_stat": "2 files changed, 30 insertions(+), 2 deletions(-)", + }, + } + + stdout = io.StringIO() + with redirect_stdout(stdout): + pipeline_skill_context.emit_result(result, "text") + + rendered = stdout.getvalue() + self.assertIn("# Final Review Packet", rendered) + self.assertIn("- PR: #615", rendered) + self.assertIn("- Board item: `PVTI_22`", rendered) + self.assertIn("## Recommendation Seed", rendered) + self.assertIn("- Suggested mode: conflicted-review", rendered) + self.assertIn("## Deterministic Checks", rendered) + self.assertIn("- Completeness: fail", rendered) + self.assertIn("- `paper_display_name`", rendered) + self.assertIn("## Changed Files", rendered) + + def test_emit_result_prints_review_pipeline_text_report(self) -> None: + result = { + "skill": "review-pipeline", + "status": "ready", + "selection": { + "item_id": "PVTI_11", + "pr_number": 570, + "issue_number": 117, + "title": "[Model] GraphPartitioning", + }, + "pr": { + "number": 570, + "title": "Fix #117: [Model] GraphPartitioning", + "url": "https://github.com/CodingThrust/problem-reductions/pull/570", + "comments": { + "counts": { + "copilot_inline_comments": 2, + "human_inline_comments": 1, + "human_issue_comments": 1, + "human_linked_issue_comments": 1, + } + }, + "issue_context_text": "Issue #117: Add GraphPartitioning model", + "ci": {"state": "failure", "failing": 1, "pending": 0}, + "codecov": {"found": True, "patch_coverage": 84.21}, + }, + "prep": { + "ready": False, + "checkout": {"worktree_dir": "/tmp/review-pr-570"}, + "merge": {"status": "conflicted", "conflicts": ["src/models/graph_partitioning.rs"]}, + }, + } + + stdout = io.StringIO() + with redirect_stdout(stdout): + pipeline_skill_context.emit_result(result, "text") + + rendered = stdout.getvalue() + self.assertIn("# Review Pipeline Packet", rendered) + self.assertIn("- PR: #570", rendered) + self.assertIn("- Board item: `PVTI_11`", rendered) + self.assertIn("## Recommendation Seed", rendered) + self.assertIn("- Suggested mode: conflicted-fix", rendered) + self.assertIn("- Copilot inline comments: 2", rendered) + self.assertIn("- CI state: failure", rendered) + self.assertIn("## Merge Prep", rendered) + self.assertIn("- Worktree: `/tmp/review-pr-570`", rendered) + self.assertIn("## Linked Issue Context", rendered) + + def test_emit_result_prints_review_implementation_text_report(self) -> None: + result = { + "skill": "review-implementation", + "status": "ready", + "git": { + "repo_root": "/tmp/repo", + "base_sha": "abc123", + "head_sha": "def456", + }, + "review_context": { + "scope": { + "review_type": "model", + "models": [ + { + "path": "src/models/graph/graph_partitioning.rs", + "problem_name": "GraphPartitioning", + } + ], + "rules": [], + "changed_files": [ + "src/models/graph/graph_partitioning.rs", + "src/unit_tests/models/graph/graph_partitioning.rs", + ], + }, + "subject": {"kind": "model", "name": "GraphPartitioning"}, + "changed_files": [ + "src/models/graph/graph_partitioning.rs", + "src/unit_tests/models/graph/graph_partitioning.rs", + ], + "diff_stat": "2 files changed, 40 insertions(+)", + "whitelist": {"ok": True, "skipped": False}, + "completeness": {"ok": False, "skipped": False, "missing": ["paper_display_name"]}, + }, + "current_pr": { + "repo": "CodingThrust/problem-reductions", + "pr_number": 615, + "title": "Fix #117: [Model] GraphPartitioning", + "linked_issue_number": 117, + "issue_context_text": "# Add GraphPartitioning\n\nNeed canonical example.", + }, + } + + stdout = io.StringIO() + with redirect_stdout(stdout): + pipeline_skill_context.emit_result(result, "text") + + rendered = stdout.getvalue() + self.assertIn("# Review Implementation Packet", rendered) + self.assertIn("- Base SHA: `abc123`", rendered) + self.assertIn("- Review type: model", rendered) + self.assertIn("- Name: GraphPartitioning", rendered) + self.assertIn("- PR: #615", rendered) + self.assertIn("- Linked issue: #117", rendered) + self.assertIn("## Deterministic Checks", rendered) + self.assertIn("- Completeness: fail", rendered) + self.assertIn("## Linked Issue Context", rendered) + + def test_emit_result_prints_project_pipeline_text_report(self) -> None: + result = { + "skill": "project-pipeline", + "status": "ready", + "repo": "CodingThrust/problem-reductions", + "existing_problems": ["BinPacking", "ILP", "GraphColoring"], + "requested_issue": None, + "ready_issues": [ + { + "item_id": "PVTI_1", + "issue_number": 117, + "title": "[Model] GraphPartitioning", + "kind": "model", + "eligible": True, + "blocking_reason": None, + "pending_rule_count": 2, + "summary": "Partition graph vertices into balanced groups.", + }, + { + "item_id": "PVTI_2", + "issue_number": 130, + "title": "[Rule] MultivariateQuadratic to ILP", + "kind": "rule", + "eligible": False, + "blocking_reason": 'model "MultivariateQuadratic" not yet implemented on main', + "pending_rule_count": 0, + "source_problem": "MultivariateQuadratic", + "target_problem": "ILP", + "summary": "Linearize quadratic constraints.", + }, + ], + "in_progress_issues": [ + { + "issue_number": 129, + "title": "[Model] MultivariateQuadratic", + } + ], + } + + stdout = io.StringIO() + with redirect_stdout(stdout): + pipeline_skill_context.emit_result(result, "text") + + rendered = stdout.getvalue() + self.assertIn("# Project Pipeline Packet", rendered) + self.assertIn("- Bundle status: ready", rendered) + self.assertIn("- Ready issues: 2", rendered) + self.assertIn("- In progress issues: 1", rendered) + self.assertIn("## Eligible Ready Issues", rendered) + self.assertIn("- #117 [Model] GraphPartitioning", rendered) + self.assertIn("- Pending rules unblocked: 2", rendered) + self.assertIn("## Blocked Ready Issues", rendered) + self.assertIn('model "MultivariateQuadratic" not yet implemented on main', rendered) + + def test_build_status_result_normalizes_empty_state(self) -> None: + self.assertEqual( + pipeline_skill_context.build_status_result( + "review-pipeline", + status="empty", + ), + { + "skill": "review-pipeline", + "status": "empty", + }, + ) + + def test_build_status_result_normalizes_manual_choice_state(self) -> None: + options = [{"item_id": "PVTI_1", "pr_number": 173}] + + self.assertEqual( + pipeline_skill_context.build_status_result( + "review-pipeline", + status="needs-user-choice", + options=options, + recommendation=173, + ), + { + "skill": "review-pipeline", + "status": "needs-user-choice", + "options": options, + "recommendation": 173, + }, + ) + + def test_main_review_pipeline_emits_ready_bundle_shape(self) -> None: + result = { + "skill": "review-pipeline", + "status": "ready", + "selection": {"item_id": "PVTI_1", "pr_number": 173}, + "prep": {"ready": True}, + "pr": {"number": 173}, + } + + with mock.patch.object( + pipeline_skill_context, + "build_review_pipeline_context", + return_value=result, + ) as builder: + stdout = io.StringIO() + with redirect_stdout(stdout): + exit_code = pipeline_skill_context.main( + [ + "review-pipeline", + "--repo", + "CodingThrust/problem-reductions", + ] + ) + + builder.assert_called_once_with( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-review-state.json"), + ) + self.assertEqual(exit_code, 0) + self.assertEqual(json.loads(stdout.getvalue()), result) + + def test_main_final_review_emits_ready_bundle_shape(self) -> None: + result = { + "skill": "final-review", + "status": "ready", + "selection": {"item_id": "PVTI_2", "pr_number": 615}, + "prep": {"ready": True}, + "pr": {"number": 615}, + "review_context": {"files": ["src/lib.rs"]}, + } + + with mock.patch.object( + pipeline_skill_context, + "build_final_review_context", + return_value=result, + ) as builder: + stdout = io.StringIO() + with redirect_stdout(stdout): + exit_code = pipeline_skill_context.main( + [ + "final-review", + "--repo", + "CodingThrust/problem-reductions", + "--pr", + "615", + ] + ) + + builder.assert_called_once_with( + repo="CodingThrust/problem-reductions", + pr_number=615, + state_file=Path("/tmp/problemreductions-final-review-state.json"), + ) + self.assertEqual(exit_code, 0) + self.assertEqual(json.loads(stdout.getvalue()), result) + + def test_main_review_implementation_emits_ready_bundle_shape(self) -> None: + result = { + "skill": "review-implementation", + "status": "ready", + "git": {"base_sha": "abc123", "head_sha": "def456"}, + "review_context": {"subject": {"kind": "generic"}}, + "current_pr": None, + } + + with mock.patch.object( + pipeline_skill_context, + "build_review_implementation_context", + return_value=result, + ) as builder: + stdout = io.StringIO() + with redirect_stdout(stdout): + exit_code = pipeline_skill_context.main( + [ + "review-implementation", + "--repo-root", + ".", + ] + ) + + builder.assert_called_once_with( + repo_root=Path("."), + kind=None, + name=None, + source=None, + target=None, + ) + self.assertEqual(exit_code, 0) + self.assertEqual(json.loads(stdout.getvalue()), result) + + def test_main_project_pipeline_emits_ready_bundle_shape(self) -> None: + result = { + "skill": "project-pipeline", + "status": "ready", + "ready_issues": [{"issue_number": 117}], + } + + with mock.patch.object( + pipeline_skill_context, + "build_project_pipeline_context", + return_value=result, + ) as builder: + stdout = io.StringIO() + with redirect_stdout(stdout): + exit_code = pipeline_skill_context.main( + [ + "project-pipeline", + "--repo", + "CodingThrust/problem-reductions", + "--issue", + "117", + ] + ) + + builder.assert_called_once_with( + repo="CodingThrust/problem-reductions", + issue_number=117, + repo_root=Path("."), + ) + self.assertEqual(exit_code, 0) + self.assertEqual(json.loads(stdout.getvalue()), result) + + def test_build_review_pipeline_context_reports_empty_queue(self) -> None: + result = pipeline_skill_context.build_review_pipeline_context( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-review-state.json"), + review_candidate_fetcher=lambda repo: [], + ) + + self.assertEqual( + result, + { + "skill": "review-pipeline", + "status": "empty", + }, + ) + + def test_build_review_pipeline_context_reports_manual_choice_for_ambiguous_card(self) -> None: + result = pipeline_skill_context.build_review_pipeline_context( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-review-state.json"), + review_candidate_fetcher=lambda repo: [ + { + "item_id": "PVTI_10", + "issue_number": 108, + "pr_number": 173, + "status": "Review pool", + "title": "[Model] LongestCommonSubsequence", + "eligibility": "ambiguous-linked-prs", + "recommendation": 173, + "linked_repo_prs": [ + {"number": 170, "state": "CLOSED", "title": "Superseded LCS model"}, + {"number": 173, "state": "OPEN", "title": "Fix #109: Add LCS reduction"}, + ], + } + ], + ) + + self.assertEqual( + result, + { + "skill": "review-pipeline", + "status": "needs-user-choice", + "options": [ + {"number": 170, "state": "CLOSED", "title": "Superseded LCS model"}, + {"number": 173, "state": "OPEN", "title": "Fix #109: Add LCS reduction"}, + ], + "recommendation": 173, + }, + ) + + def test_build_review_pipeline_context_disambiguates_explicit_pr_choice(self) -> None: + moves: list[tuple[str, str]] = [] + + result = pipeline_skill_context.build_review_pipeline_context( + repo="CodingThrust/problem-reductions", + pr_number=173, + state_file=Path("/tmp/problemreductions-review-state.json"), + review_candidate_fetcher=lambda repo: [ + { + "item_id": "PVTI_10", + "issue_number": 108, + "pr_number": 173, + "status": "Review pool", + "title": "[Model] LongestCommonSubsequence", + "eligibility": "ambiguous-linked-prs", + "recommendation": 173, + "linked_repo_prs": [ + {"number": 170, "state": "CLOSED", "title": "Superseded LCS model"}, + {"number": 173, "state": "OPEN", "title": "Fix #109: Add LCS reduction"}, + ], + } + ], + mover=lambda item_id, status: moves.append((item_id, status)), + pr_context_builder=lambda repo, pr_number: { + "number": pr_number, + "title": "Fix #109: Add LCS reduction", + }, + review_preparer=lambda repo, pr_number: { + "ready": True, + "checkout": {"worktree_dir": "/tmp/review-pr-173"}, + }, + ) + + self.assertEqual(moves, [("PVTI_10", "Under review")]) + self.assertEqual( + result, + { + "skill": "review-pipeline", + "status": "ready", + "selection": { + "item_id": "PVTI_10", + "number": 173, + "issue_number": 108, + "pr_number": 173, + "status": "Review pool", + "title": "[Model] LongestCommonSubsequence", + "claimed": True, + "claimed_status": "Under review", + }, + "pr": { + "number": 173, + "title": "Fix #109: Add LCS reduction", + }, + "prep": { + "ready": True, + "checkout": {"worktree_dir": "/tmp/review-pr-173"}, + }, + }, + ) + + def test_build_review_pipeline_context_returns_ready_bundle_for_eligible_pr(self) -> None: + result = pipeline_skill_context.build_review_pipeline_context( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-review-state.json"), + review_candidate_fetcher=lambda repo: [ + { + "item_id": "PVTI_11", + "issue_number": 117, + "pr_number": 570, + "status": "Review pool", + "title": "[Model] GraphPartitioning", + "eligibility": "eligible", + "reason": "copilot reviewed", + } + ], + claim_entry=lambda **kwargs: { + "item_id": "PVTI_11", + "number": 570, + "issue_number": 117, + "pr_number": 570, + "status": "Review pool", + "title": "[Model] GraphPartitioning", + "claimed": True, + "claimed_status": "Under review", + }, + pr_context_builder=lambda repo, pr_number: { + "number": pr_number, + "comments": {"counts": {"copilot_inline_comments": 1}}, + }, + review_preparer=lambda repo, pr_number: { + "ready": True, + "checkout": {"worktree_dir": "/tmp/review-pr-570"}, + }, + ) + + self.assertEqual( + result, + { + "skill": "review-pipeline", + "status": "ready", + "selection": { + "item_id": "PVTI_11", + "number": 570, + "issue_number": 117, + "pr_number": 570, + "status": "Review pool", + "title": "[Model] GraphPartitioning", + "claimed": True, + "claimed_status": "Under review", + }, + "pr": { + "number": 570, + "comments": {"counts": {"copilot_inline_comments": 1}}, + }, + "prep": { + "ready": True, + "checkout": {"worktree_dir": "/tmp/review-pr-570"}, + }, + }, + ) + + def test_build_final_review_context_reports_empty_queue(self) -> None: + result = pipeline_skill_context.build_final_review_context( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-final-review-state.json"), + selection_fetcher=lambda **kwargs: None, + ) + + self.assertEqual( + result, + { + "skill": "final-review", + "status": "empty", + }, + ) + + def test_build_final_review_context_returns_ready_bundle_for_clean_prep(self) -> None: + selection = { + "item_id": "PVTI_22", + "number": 615, + "issue_number": 117, + "pr_number": 615, + "status": "Final review", + "title": "[Model] GraphPartitioning", + } + prep = { + "ready": True, + "checkout": { + "worktree_dir": "/tmp/final-pr-615", + "base_sha": "abc123", + "head_sha": "def456", + }, + "merge": {"status": "clean", "conflicts": [], "likely_complex": False}, + } + pr_context = { + "number": 615, + "title": "Fix #117: [Model] GraphPartitioning", + } + review_context = { + "subject": {"kind": "model", "name": "GraphPartitioning"}, + "whitelist": {"ok": True}, + "completeness": {"ok": True}, + } + + result = pipeline_skill_context.build_final_review_context( + repo="CodingThrust/problem-reductions", + pr_number=None, + state_file=Path("/tmp/problemreductions-final-review-state.json"), + selection_fetcher=lambda **kwargs: selection, + pr_context_builder=lambda repo, pr_number: pr_context, + review_preparer=lambda repo, pr_number: prep, + review_context_builder=lambda *, prep, pr_context: review_context, + ) + + self.assertEqual( + result, + { + "skill": "final-review", + "status": "ready", + "selection": selection, + "pr": pr_context, + "prep": prep, + "review_context": review_context, + }, + ) + + def test_build_final_review_context_keeps_review_context_on_conflicted_prep(self) -> None: + selection = { + "item_id": "PVTI_23", + "number": 620, + "issue_number": 118, + "pr_number": 620, + "status": "Final review", + "title": "[Rule] BinPacking to ILP", + } + prep = { + "ready": False, + "checkout": { + "worktree_dir": "/tmp/final-pr-620", + "base_sha": "abc123", + "head_sha": "def456", + }, + "merge": { + "status": "conflicted", + "conflicts": ["src/rules/binpacking_ilp.rs"], + "likely_complex": False, + }, + } + review_context = { + "subject": {"kind": "rule", "name": "binpacking_ilp"}, + "whitelist": {"ok": True}, + "completeness": {"ok": True}, + } + + result = pipeline_skill_context.build_final_review_context( + repo="CodingThrust/problem-reductions", + pr_number=620, + state_file=Path("/tmp/problemreductions-final-review-state.json"), + selection_fetcher=lambda **kwargs: selection, + pr_context_builder=lambda repo, pr_number: {"number": pr_number}, + review_preparer=lambda repo, pr_number: prep, + review_context_builder=lambda *, prep, pr_context: review_context, + ) + + self.assertEqual(result["status"], "ready") + self.assertEqual(result["prep"]["merge"]["status"], "conflicted") + self.assertEqual(result["review_context"], review_context) + + def test_build_final_review_context_returns_warning_state_on_prep_failure(self) -> None: + selection = { + "item_id": "PVTI_24", + "number": 621, + "issue_number": 119, + "pr_number": 621, + "status": "Final review", + "title": "[Model] FlowShopScheduling", + } + + def fail_prepare(repo: str, pr_number: int) -> dict: + raise RuntimeError("checkout failed") + + result = pipeline_skill_context.build_final_review_context( + repo="CodingThrust/problem-reductions", + pr_number=621, + state_file=Path("/tmp/problemreductions-final-review-state.json"), + selection_fetcher=lambda **kwargs: selection, + pr_context_builder=lambda repo, pr_number: {"number": pr_number}, + review_preparer=fail_prepare, + ) + + self.assertEqual( + result, + { + "skill": "final-review", + "status": "ready-with-warnings", + "selection": selection, + "pr": {"number": 621}, + "prep": {"ready": False, "error": "checkout failed"}, + "review_context": None, + "warnings": [ + "failed to prepare final-review worktree: checkout failed", + ], + }, + ) + + def test_build_review_implementation_context_without_current_pr(self) -> None: + result = pipeline_skill_context.build_review_implementation_context( + repo_root=Path("/tmp/repo"), + kind=None, + name=None, + source=None, + target=None, + merge_base_getter=lambda repo_root: "abc123", + head_sha_getter=lambda repo_root: "def456", + diff_stat_getter=lambda repo_root, base_sha, head_sha: "2 files changed", + changed_files_getter=lambda repo_root, base_sha, head_sha: [ + "src/lib.rs", + "src/unit_tests/lib.rs", + ], + added_files_getter=lambda repo_root, base_sha, head_sha: [], + current_pr_fetcher=lambda: None, + review_context_builder=lambda repo_root, **kwargs: { + "scope": {"review_type": "generic", "models": [], "rules": [], "changed_files": kwargs["changed_files"]}, + "subject": {"kind": "generic"}, + "changed_files": kwargs["changed_files"], + "diff_stat": kwargs["diff_stat"], + "whitelist": {"ok": True, "skipped": True}, + "completeness": {"ok": True, "skipped": True, "missing": []}, + }, + ) + + self.assertEqual(result["skill"], "review-implementation") + self.assertEqual(result["status"], "ready") + self.assertEqual(result["git"]["base_sha"], "abc123") + self.assertEqual(result["current_pr"], None) + self.assertEqual(result["review_context"]["subject"]["kind"], "generic") + + def test_build_project_pipeline_context_reports_requested_blocked_issue(self) -> None: + board_data = { + "items": [ + { + "id": "PVTI_2", + "status": "Ready", + "content": { + "type": "Issue", + "number": 130, + "title": "[Rule] MultivariateQuadratic to ILP", + }, + } + ] + } + issue_data = { + 130: { + "number": 130, + "title": "[Rule] MultivariateQuadratic to ILP", + "body": "Linearize quadratic constraints.", + "comments": [], + "labels": [], + "url": "https://github.com/CodingThrust/problem-reductions/issues/130", + } + } + + result = pipeline_skill_context.build_project_pipeline_context( + repo="CodingThrust/problem-reductions", + issue_number=130, + repo_root=Path("/tmp/repo"), + board_fetcher=lambda repo: board_data, + issue_fetcher=lambda repo, issue_number: issue_data[issue_number], + existing_problem_finder=lambda repo_root: {"ILP"}, + ) + + self.assertEqual(result["skill"], "project-pipeline") + self.assertEqual(result["status"], "requested-blocked") + self.assertEqual(result["requested_issue"]["issue_number"], 130) + self.assertEqual( + result["requested_issue"]["blocking_reason"], + 'model "MultivariateQuadratic" not yet implemented on main', + ) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_pipeline_worktree.py b/scripts/test_pipeline_worktree.py new file mode 100644 index 000000000..cf2428b43 --- /dev/null +++ b/scripts/test_pipeline_worktree.py @@ -0,0 +1,245 @@ +#!/usr/bin/env python3 +import unittest +from unittest import mock + +import pipeline_worktree +from pipeline_worktree import ( + prepare_issue_branch, + plan_issue_worktree, + plan_pr_worktree, + summarize_merge, +) + + +class PipelineWorktreeTests(unittest.TestCase): + def test_plan_issue_worktree_sanitizes_slug_and_uses_worktrees_dir(self) -> None: + plan = plan_issue_worktree( + "/tmp/problemreductions", + issue_number=117, + slug="Graph Partitioning / Exact", + base_ref="origin/main", + ) + + self.assertEqual(plan["branch"], "issue-117-graph-partitioning-exact") + self.assertEqual( + plan["worktree_dir"], + "/tmp/problemreductions/.worktrees/issue-117-graph-partitioning-exact", + ) + self.assertEqual(plan["base_ref"], "origin/main") + + def test_plan_pr_worktree_uses_pull_ref_and_sanitized_local_branch(self) -> None: + plan = plan_pr_worktree( + "/tmp/problemreductions", + pr_number=570, + head_ref_name="feature/lcs cleanup", + base_sha="base123", + head_sha="head456", + ) + + self.assertEqual(plan["local_branch"], "review-pr-570-feature-lcs-cleanup") + self.assertEqual( + plan["worktree_dir"], + "/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup", + ) + self.assertEqual( + plan["fetch_ref"], + "pull/570/head:review-pr-570-feature-lcs-cleanup", + ) + self.assertEqual(plan["base_sha"], "base123") + self.assertEqual(plan["head_sha"], "head456") + + def test_summarize_merge_clean_result(self) -> None: + summary = summarize_merge( + worktree="/tmp/problemreductions/.worktrees/review-pr-570", + exit_code=0, + conflicts=[], + ) + + self.assertEqual(summary["status"], "clean") + self.assertFalse(summary["likely_complex"]) + self.assertEqual(summary["conflicts"], []) + + def test_summarize_merge_conflicted_result_marks_complex_skill_conflicts(self) -> None: + summary = summarize_merge( + worktree="/tmp/problemreductions/.worktrees/review-pr-570", + exit_code=1, + conflicts=[ + ".claude/skills/add-model/SKILL.md", + "src/models/graph/graph_partitioning.rs", + ], + ) + + self.assertEqual(summary["status"], "conflicted") + self.assertTrue(summary["likely_complex"]) + self.assertEqual( + summary["conflicts"], + [ + ".claude/skills/add-model/SKILL.md", + "src/models/graph/graph_partitioning.rs", + ], + ) + + def test_summarize_merge_without_conflicts_is_aborted(self) -> None: + summary = summarize_merge( + worktree="/tmp/problemreductions/.worktrees/review-pr-570", + exit_code=128, + conflicts=[], + ) + + self.assertEqual(summary["status"], "aborted") + self.assertFalse(summary["likely_complex"]) + + @mock.patch("pipeline_worktree.merge_main") + @mock.patch("pipeline_worktree.checkout_pr_worktree") + def test_prepare_review_bundles_checkout_and_clean_merge( + self, + checkout_pr_worktree: mock.Mock, + merge_main: mock.Mock, + ) -> None: + prepare_review = getattr(pipeline_worktree, "prepare_review", None) + self.assertIsNotNone(prepare_review) + + checkout_payload = { + "pr_number": 570, + "head_ref_name": "feature/lcs cleanup", + "local_branch": "review-pr-570-feature-lcs-cleanup", + "worktree_dir": "/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup", + "fetch_ref": "pull/570/head:review-pr-570-feature-lcs-cleanup", + "base_sha": "base123", + "head_sha": "head456", + } + merge_payload = { + "worktree": "/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup", + "status": "clean", + "conflicts": [], + "likely_complex": False, + "stdout": "Already up to date.\n", + "stderr": "", + } + checkout_pr_worktree.return_value = checkout_payload + merge_main.return_value = merge_payload + + result = prepare_review( + repo="CodingThrust/problem-reductions", + pr_number=570, + repo_root="/tmp/problemreductions", + ) + + self.assertEqual(result["checkout"], checkout_payload) + self.assertEqual(result["merge"], merge_payload) + self.assertTrue(result["ready"]) + checkout_pr_worktree.assert_called_once_with( + repo="CodingThrust/problem-reductions", + pr_number=570, + repo_root="/tmp/problemreductions", + ) + merge_main.assert_called_once_with( + worktree="/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup" + ) + + @mock.patch("pipeline_worktree.merge_main") + @mock.patch("pipeline_worktree.checkout_pr_worktree") + def test_prepare_review_marks_conflicted_merge_not_ready( + self, + checkout_pr_worktree: mock.Mock, + merge_main: mock.Mock, + ) -> None: + prepare_review = getattr(pipeline_worktree, "prepare_review", None) + self.assertIsNotNone(prepare_review) + + checkout_pr_worktree.return_value = { + "pr_number": 570, + "head_ref_name": "feature/lcs cleanup", + "local_branch": "review-pr-570-feature-lcs-cleanup", + "worktree_dir": "/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup", + "fetch_ref": "pull/570/head:review-pr-570-feature-lcs-cleanup", + "base_sha": "base123", + "head_sha": "head456", + } + merge_main.return_value = { + "worktree": "/tmp/problemreductions/.worktrees/review-pr-570-feature-lcs-cleanup", + "status": "conflicted", + "conflicts": [ + ".claude/skills/add-model/SKILL.md", + "src/models/graph/graph_partitioning.rs", + ], + "likely_complex": True, + "stdout": "Auto-merging ...\n", + "stderr": "CONFLICT (content): Merge conflict in .claude/skills/add-model/SKILL.md\n", + } + + result = prepare_review( + repo="CodingThrust/problem-reductions", + pr_number=570, + repo_root="/tmp/problemreductions", + ) + + self.assertFalse(result["ready"]) + self.assertEqual( + result["merge"]["conflicts"], + [ + ".claude/skills/add-model/SKILL.md", + "src/models/graph/graph_partitioning.rs", + ], + ) + self.assertTrue(result["merge"]["likely_complex"]) + + @mock.patch("pipeline_worktree.run_git_checked") + @mock.patch("pipeline_worktree.run_git") + def test_prepare_issue_branch_creates_new_branch_when_missing( + self, + run_git: mock.Mock, + run_git_checked: mock.Mock, + ) -> None: + run_git.side_effect = [ + "", # git status --porcelain + "abc123\n", # rev-parse main + "def456\n", # rev-parse HEAD + ] + + with mock.patch("pipeline_worktree.branch_exists", return_value=False): + result = prepare_issue_branch( + issue_number=117, + slug="Graph Partitioning", + base_ref="main", + repo_root="/tmp/problemreductions", + ) + + self.assertEqual(result["branch"], "issue-117-graph-partitioning") + self.assertEqual(result["action"], "create-branch") + self.assertFalse(result["existing_branch"]) + tails = [call.args[1:] for call in run_git_checked.call_args_list] + self.assertIn(("checkout", "main"), tails) + self.assertIn(("checkout", "-b", "issue-117-graph-partitioning"), tails) + + @mock.patch("pipeline_worktree.run_git_checked") + @mock.patch("pipeline_worktree.run_git") + def test_prepare_issue_branch_reuses_existing_branch( + self, + run_git: mock.Mock, + run_git_checked: mock.Mock, + ) -> None: + run_git.side_effect = [ + "", # git status --porcelain + "abc123\n", # rev-parse main + "def456\n", # rev-parse HEAD + ] + + with mock.patch("pipeline_worktree.branch_exists", return_value=True): + result = prepare_issue_branch( + issue_number=117, + slug="Graph Partitioning", + base_ref="main", + repo_root="/tmp/problemreductions", + ) + + self.assertEqual(result["branch"], "issue-117-graph-partitioning") + self.assertEqual(result["action"], "checkout-existing") + self.assertTrue(result["existing_branch"]) + tails = [call.args[1:] for call in run_git_checked.call_args_list] + self.assertIn(("checkout", "main"), tails) + self.assertIn(("checkout", "issue-117-graph-partitioning"), tails) + + +if __name__ == "__main__": + unittest.main() diff --git a/scripts/test_project_board_poll.py b/scripts/test_project_board_poll.py index 10451a61d..44b925e1c 100644 --- a/scripts/test_project_board_poll.py +++ b/scripts/test_project_board_poll.py @@ -26,6 +26,15 @@ def make_pr_item(item_id: str, number: int, status: str = "Review pool") -> dict } +def with_linked_prs(item: dict, *pr_numbers: int) -> dict: + updated = dict(item) + updated["linked pull requests"] = [ + f"https://github.com/CodingThrust/problem-reductions/pull/{number}" + for number in pr_numbers + ] + return updated + + class ProjectBoardPollTests(unittest.TestCase): def test_ready_queue_retries_same_item_until_ack(self) -> None: with tempfile.TemporaryDirectory() as tmpdir: @@ -92,6 +101,11 @@ def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] return [] + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "OPEN" + with tempfile.TemporaryDirectory() as tmpdir: state_file = Path(tmpdir) / "review-state.json" item_id, number = process_snapshot( @@ -101,6 +115,7 @@ def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: repo="CodingThrust/problem-reductions", review_fetcher=fake_review_fetcher, pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, ) self.assertEqual((item_id, number), ("PVTI_10", 570)) @@ -108,6 +123,11 @@ def test_review_fetch_errors_are_not_suppressed(self) -> None: def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: raise subprocess.CalledProcessError(42, ["gh", "api"]) + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "OPEN" + with tempfile.TemporaryDirectory() as tmpdir: state_file = Path(tmpdir) / "review-state.json" with self.assertRaises(subprocess.CalledProcessError): @@ -117,8 +137,66 @@ def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: state_file, repo="CodingThrust/problem-reductions", review_fetcher=fake_review_fetcher, + pr_state_fetcher=fake_pr_state_fetcher, ) + def test_review_queue_skips_closed_pr_cards(self) -> None: + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 570) + return "CLOSED" + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + no_item = process_snapshot( + "review", + {"items": [make_pr_item("PVTI_10", 570)]}, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertIsNone(no_item) + + def test_review_queue_skips_issue_cards_with_mixed_linked_pr_states(self) -> None: + def fake_pr_resolver(repo: str, issue_number: int) -> int | None: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(issue_number, 108) + return 173 + + def fake_review_fetcher(repo: str, pr_number: int) -> list[dict]: + self.assertEqual(repo, "CodingThrust/problem-reductions") + self.assertEqual(pr_number, 173) + return [{"user": {"login": "copilot-pull-request-reviewer[bot]"}}] + + def fake_pr_state_fetcher(repo: str, pr_number: int) -> str: + self.assertEqual(repo, "CodingThrust/problem-reductions") + return {170: "CLOSED", 173: "OPEN"}[pr_number] + + with tempfile.TemporaryDirectory() as tmpdir: + state_file = Path(tmpdir) / "review-state.json" + no_item = process_snapshot( + "review", + { + "items": [ + with_linked_prs( + make_issue_item("PVTI_10", 108, status="Review pool"), + 170, + 173, + ) + ] + }, + state_file, + repo="CodingThrust/problem-reductions", + review_fetcher=fake_review_fetcher, + pr_resolver=fake_pr_resolver, + pr_state_fetcher=fake_pr_state_fetcher, + ) + self.assertIsNone(no_item) + if __name__ == "__main__": unittest.main()