cleanup: simplify agent mode providers

[kalil0321]

Summary

• Agent mode now only exposes auto (Playwright MCP) and chrome-mcp (Chrome DevTools MCP); two legacy provider options have been retired
• Drops the legacy AgentBrowser path and its helpers from browser.py; run_agent_capture in cli.py becomes a thin delegate to run_auto_capture
• Trims DEFAULT_CONFIG and silently migrates retired agent_provider values to auto so existing user configs keep working
• Drops the now-empty [agent] optional-dependency extra and updates README + adds an Unreleased CHANGELOG entry

Net diff: +32 / −872 across 8 files.

Test plan

• uv run pytest tests/test_config.py (18/18 pass, including the two new provider-fallback migration tests)
• uv run pytest full suite (685 pass; 4 pre-existing failures reproduce on main, unrelated to this change)
• uv run ruff check on touched files: 1 fewer warning than main, no new issues
• uv run reverse-api-engineer --help still resolves all subcommands
• Smoke-test agent mode end-to-end (auto + chrome-mcp) before merge

🤖 Generated with Claude Code

---

Summary by cubic

Simplifies agent mode to two providers — auto (Playwright MCP) and chrome-mcp (Chrome DevTools MCP) — and removes the inline tag system in favor of clear CLI flags. Also fixes engineer prompt handling and removes dead code and the [agent] extra.

• Refactors

  • Retired browser-use and stagehand; run_agent_capture now delegates directly to auto and the ignored --reverse-engineer/--no-engineer flag is removed.
  • Removed tag system (@record-only, @codegen, @id, @docs, @help); deleted action_recorder.py, playwright_codegen.py, and related utils/tests; dropped stagehand from uv.lock; updated README and CHANGELOG.

• Migration

  • Configs with agent_provider set to browser-use or stagehand now load as auto; obsolete keys are ignored.
  • Tag replacements: @record-only → manual --no-engineer; @id <run_id> → engineer <run_id>; @id <run_id> <prompt> → engineer <run_id> --prompt "..."; @id <run_id> --fresh → engineer <run_id> --fresh. Additionally, engineer <run_id> --prompt now keeps the original goal unless --fresh is set.

^{Written for commit 19311d8. Summary will update on new commits.}

Greptile Summary

This PR retires the browser-use and stagehand agent providers, leaving only auto (Playwright MCP) and chrome-mcp (Chrome DevTools MCP). Existing configs with the old provider values are silently migrated to auto, and the associated [agent] optional-dependency extra is dropped.

• browser.py: ~560 lines removed — AgentBrowser, run_agent_browser, and the model/API-key helper functions are all gone.
• cli.py: run_agent_capture is now a thin delegate to run_auto_capture; the browser-use/stagehand settings menu entries are removed. The --reverse-engineer/--no-engineer CLI flag on the agent command is accepted but not forwarded to run_auto_capture (pre-existing issue, flagged in a prior review).
• config.py + tests: Migration logic correctly resets retired provider values to auto; the valid_config filter strips obsolete keys (browser_use_model, stagehand_model) on the next save. Two new migration tests cover the fallback paths.

Confidence Score: 5/5

Safe to merge — this is a focused dead-code removal with correct migration logic and full test coverage for the new paths.

The change deletes legacy code that is no longer reachable, adds a straightforward migration guard in ConfigManager.load, and updates all supporting tests. No new functional paths are introduced. The one existing gap (the --reverse-engineer flag being ignored on the agent command) predates this PR and was already noted in a previous review.

No files require special attention; cli.py carries a pre-existing cosmetic gap with the unused --reverse-engineer flag, but that is unrelated to this diff.

Important Files Changed

Filename	Overview
src/reverse_api/browser.py	Removes ~560 lines: AgentBrowser, run_agent_browser, parse_agent_model, get_required_api_key, validate_api_key, _suppress_stagehand_logs, and associated imports. No remaining references to deleted symbols.
src/reverse_api/cli.py	Removes browser-use/stagehand settings UI entries and simplifies run_agent_capture to delegate directly to run_auto_capture. The --reverse-engineer/--no-engineer flag on the agent command is now silently ignored (noted in previous review).
src/reverse_api/config.py	Drops browser_use_model and stagehand_model from DEFAULT_CONFIG; adds migration to reset legacy agent_provider values to auto. Logic is correct — the valid_config filter naturally purges unknown keys on next save.
tests/test_config.py	Replaces four old migration tests with two focused provider-fallback tests; adds copilot_model to the expected-keys set, which was missing before. All existing test logic stays intact.
pyproject.toml	Removes the now-empty [agent] optional-dependency extra; no other dependency changes.
uv.lock	Removes stagehand package entry and all its transitive references; provides-extras updated to drop agent. Lock file is consistent with pyproject.toml.
CHANGELOG.md	Adds an [Unreleased] section documenting the removal of browser-use and stagehand providers and the silent migration behaviour.
README.md	Updates installation instructions, feature bullets, and agent-provider documentation to reflect the two remaining providers (auto, chrome-mcp); removes browser-use/stagehand sections and related API-key guidance.

Comments Outside Diff (1)

1. src/reverse_api/cli.py, line 1171-1185 (link)

   --reverse-engineer / --no-engineer flag is silently ignored

   The agent command still exposes --reverse-engineer/--no-engineer (default True) and passes it into run_agent_capture, but the new simplified body of run_agent_capture never forwards that value to run_auto_capture. A user who passes --no-engineer expecting to suppress engineering will get no effect. Consider either (a) removing the flag from the agent command now that the legacy paths that respected it are gone, or (b) documenting in the help text that engineering always runs in auto mode.

   Prompt To Fix With AI

   This is a comment left during a code review.
   Path: src/reverse_api/cli.py
   Line: 1171-1185
   
   Comment:
   **`--reverse-engineer` / `--no-engineer` flag is silently ignored**
   
   The `agent` command still exposes `--reverse-engineer/--no-engineer` (default `True`) and passes it into `run_agent_capture`, but the new simplified body of `run_agent_capture` never forwards that value to `run_auto_capture`. A user who passes `--no-engineer` expecting to suppress engineering will get no effect. Consider either (a) removing the flag from the `agent` command now that the legacy paths that respected it are gone, or (b) documenting in the help text that engineering always runs in auto mode.
   
   How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (2): Last reviewed commit: "cleanup: simplify agent mode providers" | Re-trigger Greptile}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8b94c847a5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Honor agent no-engineer/record-only controls

run_agent_capture now unconditionally delegates to run_auto_capture and never uses the parsed reverse_engineer/@record-only state, so agent --no-engineer and @record-only still run full auto engineering. After this commit removed the legacy providers, that means there is no remaining agent path that can skip engineering, which is a functional regression and can trigger unexpected model usage/cost for users expecting capture-only behavior.

Useful? React with 👍 / 👎.

[cubic-dev-ai]

1 issue found across 8 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/reverse_api/cli.py">

<violation number="1" location="src/reverse_api/cli.py:1289">
P1: Agent-mode reverse-engineering controls are ignored: `--no-engineer` and `@record-only` no longer disable generation after this unconditional delegate to `run_auto_capture`.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[kind-agent]

Test Results

🛡️ 4/6

Results

#	Test	Status	Details
1	Config migration for retired providers	✅ passed	uv run pytest tests/test_config.py -q passed (18 tests). Legacy agent_provider values browser-use and stagehand now fall back to auto without breaking config loading.
2	Full test suite regression check	✅ passed	uv run pytest completed with 4 failures, all outside the changed provider/config path (test_opencode_ui::test_sync_flash, two test_sync cases around SyncHandler._is_temporary_file, and test_tui::test_thinking_truncation). I did not find a new failure attributable to this PR.
3	CLI exposure cleanup	✅ passed	uv run reverse-api-engineer agent --help still works, and code/help inspection shows only the supported provider flow remains exposed. Removed settings like browser_use_model and stagehand_model are no longer surfaced.
4	Agent capture delegation	⚠️ inconclusive	I verified in code that run_agent_capture() now delegates directly to run_auto_capture() with the configured provider, but I could not complete a deterministic end-to-end live MCP/browser session in this CLI-only sandbox without risking a hung interactive run.
5	chrome-mcp provider smoke test	⚠️ inconclusive	chrome-mcp is still threaded through auto_engineer.py, but I could not run a reliable live Chrome/MCP capture session in this environment.
6	Docs and packaging cleanup	✅ passed	README install docs no longer mention the removed [agent] extra, CHANGELOG documents the removal, the Keep a Changelog link returned 200, and installed metadata/pyproject.toml now expose only collector, copilot, and pricing extras.
7	Ruff on touched files	❌ failed	uv run ruff check src/reverse_api/config.py src/reverse_api/cli.py src/reverse_api/browser.py tests/test_config.py does not pass cleanly because src/reverse_api/browser.py still has whitespace lint violations (W293) on unchanged lines.

Issues Found

Ruff on touched files: The requested lint validation does not pass because src/reverse_api/browser.py still contains whitespace-only-line warnings (W293). The provider-cleanup behavior itself tested cleanly, but the branch is not fully lint-clean under the requested touched-file check.

Live provider smoke coverage: I could confirm the refactor structurally (config migration, removed options, delegation to run_auto_capture, and retained chrome-mcp branching), but I could not fully verify live auto and chrome-mcp capture sessions in this CLI-only run. That leaves some residual risk around runtime MCP/browser integration.

Summary

The cleanup itself looks functionally sound: config migration works, legacy provider values are safely reset to auto, removed options disappeared from the CLI/settings surface, and packaging/docs are consistent with the simplified provider story. My main blockers to a clean pass were the failing touched-file ruff check and the lack of a reliable noninteractive harness here to complete real end-to-end auto and chrome-mcp capture sessions.

⚠️ Note: This is a large PR — the diff was truncated for analysis. Test coverage may not reflect all changes. Consider splitting large PRs for more thorough testing.

---

View full run details · Tested by Kind

_{I tested 8b94c84 (cleanup: simplify agent mode providers)}

📹 View browser recording

[kalil0321]

@cubic-dev-ai — No ACK on this PR (issue is pre-existing)

P1: Agent-mode reverse-engineering controls are ignored: --no-engineer and @record-only no longer disable generation after this unconditional delegate to run_auto_capture.

I checked main before applying this cleanup. The auto / chrome-mcp branch of run_agent_capture was already an unconditional delegate that didn't propagate reverse_engineer or is_record_only:

# main, src/reverse_api/cli.py:1378
if agent_provider in ("auto", "chrome-mcp"):
    return run_auto_capture(
        prompt=prompt,
        url=url,
        model=model,
        output_dir=output_dir,
        agent_provider=agent_provider,
    )

So --no-engineer and @record-only were already a no-op for the default agent_provider="auto" path. The only path that did honor those flags was the legacy browser-use / stagehand branch — which is exactly what this PR removes. So this isn't a regression introduced here; it's a pre-existing UX gap of the auto provider that this PR makes more visible (since auto is now the only path).

@kind-agent — partial Ack

• Tests 1–6 passing: noted, thanks.
• Test 7 (ruff on touched files) failed on browser.py W293: No ACK as a regression. Those whitespace warnings are on lines this PR did not touch — they reproduce on main (verified via stash + run). The ruff baseline for changed files is actually 1 warning lower on this branch than on main.
• Tests 4 & 5 (live MCP / chrome-mcp sessions inconclusive): agreed; couldn't smoke-test live captures here either. Will validate manually before merge.

Follow-up

I've filed #62 (agent friendliness) which tracks the broader "make the agent CLI honor its flags reliably" theme — the --no-engineer / @record-only behavior on the auto path fits there. Happy to take it as a follow-up PR, separate from this cleanup.

[cubic-dev-ai]

@cubic-dev-ai — No ACK on this PR (issue is pre-existing)

P1: Agent-mode reverse-engineering controls are ignored: --no-engineer and @record-only no longer disable generation after this unconditional delegate to run_auto_capture.

I checked main before applying this cleanup. The auto / chrome-mcp branch of run_agent_capture was already an unconditional delegate that didn't propagate reverse_engineer or is_record_only:
...

@kalil0321 You're right on both counts — thanks for digging into main to verify.

--no-engineer / @record-only: Fair catch. The auto path was already an unconditional delegate on main, so this PR doesn't introduce a regression there — it just makes the gap more visible by removing the only path that did honor those flags. No ACK from me for the original P1 framing. Filing #62 as a follow-up is the right call.

Ruff W293: Also correct — those warnings are pre-existing on main. The cleaner diff is actually +32/−872 with one fewer warning than the baseline. My test report should have checked against main rather than treating this as a regression in the PR. Apologies for the noise there.

The rest of the analysis (config migration, test suite, docs cleanup) stands. Your reasoning is solid across the board.

[kind-agent]

Thanks — that matches the evidence available from the previous run.

I did not verify the main branch comparison myself in that run, so I can’t independently confirm the historical behavior from execution. But based on what I already observed:

• the current PR’s run_agent_capture() does unconditionally delegate to run_auto_capture() with only prompt, url, model, output_dir, and agent_provider
• I did observe that reverse_engineer / is_record_only were parsed in run_agent_capture() and then not forwarded into that delegate call
• so the flagged behavior is present on this PR head, but from my prior run alone I cannot support calling it a regression introduced by this PR

Likewise on ruff:

• I did observe the touched-file command fail on src/reverse_api/browser.py with W293
• I did not compare that lint state against main during the prior run
• so I agree it should not be treated as a PR-introduced regression based on my evidence alone

Given your note that both the SHA I tested and the current head SHA are identical, and since your follow-up is clarifying interpretation rather than asking for new execution, I’m treating this as:

• No regression established for this PR on the --no-engineer / @record-only auto-path behavior
• No regression established for this PR on the reported browser.py whitespace lint warnings
• live auto / chrome-mcp runtime smoke coverage remains inconclusive from my prior run

Thanks for the clarification and for splitting the agent-flag behavior into follow-up issue #62.

[kalil0321]

@greptile review