feat: agent-friendly CLI surface (--json, --no-interactive)#61
Conversation
Add scriptable / non-interactive flags so other agents can drive the CLI for reverse engineering without TTY prompts. - `agent` gains `--no-interactive` and `--json`; --json suppresses Rich output to stderr and emits a single stable JSON object on stdout (schema_version, status, run_id, prompt, url, mode, har_path, script_path, usage, error). Missing --prompt under --json/--no-interactive errors out with exit code 2 instead of opening questionary - `run_auto_capture` now returns a normalized dict so the JSON payload has a stable shape regardless of the underlying engineer SDK - `list --json` emits `[]` on empty history (was a human-readable "No runs found." line); `show <id> --json` emits a structured error and exits 1 when the run can't be found - `run` gains `--no-interactive` (fail instead of opening a script picker) and `--auto-install` (install missing deps without confirm) - Documented scripted usage and exit-code contract in the README - Added tests/test_cli_agent_json.py covering the JSON payload shape and CLI-level edge cases (10 new tests) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| finally: | ||
| sys.stdout = real_stdout | ||
| for c, original_inner in redirected_consoles: | ||
| c._file = original_inner |
There was a problem hiding this comment.
candidate.file via the public property setter (which can trigger side-effects like _detect_color_system() in some Rich versions) but restores with c._file = original_inner, bypassing those same side-effects entirely. This inconsistency could silently leave consoles in a wrong color/capability state after the context exits, and may break silently when Rich updates its internals. Prefer the public file setter for both directions.
| finally: | |
| sys.stdout = real_stdout | |
| for c, original_inner in redirected_consoles: | |
| c._file = original_inner | |
| finally: | |
| sys.stdout = real_stdout | |
| for c, original_inner in redirected_consoles: | |
| c.file = original_inner |
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/reverse_api/cli.py
Line: 89-92
Comment:
The context manager sets `candidate.file` via the public property setter (which can trigger side-effects like `_detect_color_system()` in some Rich versions) but restores with `c._file = original_inner`, bypassing those same side-effects entirely. This inconsistency could silently leave consoles in a wrong color/capability state after the context exits, and may break silently when Rich updates its internals. Prefer the public `file` setter for both directions.
```suggestion
finally:
sys.stdout = real_stdout
for c, original_inner in redirected_consoles:
c.file = original_inner
```
How can I resolve this? If you propose a fix, please make it concise.| def test_no_interactive_without_prompt_exits_2(self): | ||
| runner = CliRunner() | ||
| result = runner.invoke(agent, ["--no-interactive"]) | ||
| assert result.exit_code == 2 | ||
| # Plain text on stderr, not JSON, since --json wasn't requested | ||
| assert "prompt" in (result.stderr or result.output).lower() |
There was a problem hiding this comment.
CliRunner defaults to mix_stderr=True, which merges stderr into result.output and leaves result.stderr as an empty string. The guard result.stderr or result.output always evaluates to result.output, so this test never actually verifies that the error was written to stderr specifically. Use mix_stderr=False and assert on result.stderr to make the intent explicit and reliable.
| def test_no_interactive_without_prompt_exits_2(self): | |
| runner = CliRunner() | |
| result = runner.invoke(agent, ["--no-interactive"]) | |
| assert result.exit_code == 2 | |
| # Plain text on stderr, not JSON, since --json wasn't requested | |
| assert "prompt" in (result.stderr or result.output).lower() | |
| def test_no_interactive_without_prompt_exits_2(self): | |
| runner = CliRunner(mix_stderr=False) | |
| result = runner.invoke(agent, ["--no-interactive"]) | |
| assert result.exit_code == 2 | |
| # Plain text on stderr, not JSON, since --json wasn't requested | |
| assert "prompt" in result.stderr.lower() |
Prompt To Fix With AI
This is a comment left during a code review.
Path: tests/test_cli_agent_json.py
Line: 80-85
Comment:
`CliRunner` defaults to `mix_stderr=True`, which merges stderr into `result.output` and leaves `result.stderr` as an empty string. The guard `result.stderr or result.output` always evaluates to `result.output`, so this test never actually verifies that the error was written to stderr specifically. Use `mix_stderr=False` and assert on `result.stderr` to make the intent explicit and reliable.
```suggestion
def test_no_interactive_without_prompt_exits_2(self):
runner = CliRunner(mix_stderr=False)
result = runner.invoke(agent, ["--no-interactive"])
assert result.exit_code == 2
# Plain text on stderr, not JSON, since --json wasn't requested
assert "prompt" in result.stderr.lower()
```
How can I resolve this? If you propose a fix, please make it concise.![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
3 issues found across 3 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/reverse_api/cli.py">
<violation number="1" location="src/reverse_api/cli.py:109">
P2: `har_path` is resolved against the default output dir, so JSON output can be wrong when `--output-dir` (or non-default config) is used.</violation>
<violation number="2" location="src/reverse_api/cli.py:1343">
P2: `agent --json` emits an inconsistent object shape on missing `--prompt`, which breaks stable-schema parsing for automation.</violation>
<violation number="3" location="src/reverse_api/cli.py:1367">
P1: Interrupted auto-capture can be reported as successful because `KeyboardInterrupt` is converted to a result without an error flag.</violation>
</file>
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Test Results🛡️ 3.75/6 Results
Issues Foundagent --json success path not reliably honoring mocked capture result: In a manual Branch not green outside the targeted tests: The requested focused tests pass, but the overall branch still fails SummaryThe new targeted tests for the agent-friendly CLI surface pass, and the documented flags/JSON/error contracts mostly line up with the implementation. However, I found a concerning mismatch in manual View full run details · Tested by Kind I tested |
- Add 7 more tests covering: run --no-interactive (multi-script error, --file passthrough, refusal to install missing deps), --auto-install (skips questionary.confirm), agent --json KeyboardInterrupt path, and stdout-purity (Rich noise lands on stderr, not stdout) - Add --help epilogs to agent / list / show / run with examples, JSON schema, and exit-code documentation so an agent inspecting --help can self-discover the scripted contract - Expand README "Scripted / Agent Usage" section with a per-field JSON schema table and an exit-code table Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Three issues flagged by cubic-dev-ai on PR #61: P1 — Interrupted auto-capture silently reported as ok When KeyboardInterrupt was caught inside run_auto_capture, the function returned a dict with no error flag, so _build_agent_payload saw a run_id and emitted status=ok. Now the inner KeyboardInterrupt bubbles up an "error: interrupted" key in the result dict, which _build_agent_payload propagates to status=error. P2 — har_path resolved against the wrong output_dir _build_agent_payload computed the HAR path with the default config root, ignoring the user's --output-dir. Threaded output_dir through the agent click handler to _build_agent_payload so HAR resolution matches the actual capture location. P2 — misuse JSON emitted incomplete schema When --json was set and --prompt was missing, the early-exit path printed only {schema_version, status, error}. Agents parsing the output expected every documented field. Now uses _build_agent_payload so misuse responses contain the full schema (with nulls). Adds 3 regression tests covering each fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Thanks for the reviews. Status of the feedback: @cubic-dev-ai — all three issues acknowledged and fixed in c91d1fa
@kind-agent — partial Ack
Re-review welcome at c91d1fa. |
|
Re-running tests because the PR changed after the previous run. Triggered by Kind |
@kalil0321 All three fixes look correct in the latest diff — thanks for the thorough ack. A couple of optional nits from the Greptile review that are still open (both P2 and easy to knock out):
None of these block the merge. The core |
Test Results🛡️ 4/6 Results
Issues FoundDirect SummaryThe re-reviewed targeted fixes from the author are in place and the new focused regression tests pass cleanly. However, I found a remaining contract bug in the primary user-facing View full run details · Tested by Kind I tested |
# Conflicts: # src/reverse_api/cli.py
Implements the first two items of the agent-friendliness backlog (#62): #1 TTY-detect at REPL entry Without a TTY and no subcommand, the prompt_toolkit REPL would block on stdin forever. Now detects `not sys.stdin.isatty()`, prints `--help` to stderr, and exits 2 (misuse). Subcommands are unaffected. #2 engineer --json / --no-interactive Mirrors the agent-mode contract (PR #61): --json redirects Rich output to stderr and emits a stable JSON payload on stdout. New helper `_build_engineer_payload` produces the schema {schema_version, status, run_id, prompt, fresh, script_path, usage, error}. KeyboardInterrupt surfaces as `error: "interrupted"`. Exit codes: 0 ok, 1 runtime error. --no-interactive is reserved for symmetry (engineer mode has no questionary prompts internally). Also surfaces `--json` / `--no-interactive` in the root --help (item #6 partial). Adds tests/test_cli_followups.py (14 tests) covering TTY-detect end-to- end via subprocess, _build_engineer_payload edge cases, and the click wiring for engineer --json (success, not-found, KeyboardInterrupt, exception, prompt-vs-fresh threading). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
After the first successful generation, ClaudeEngineer (and ClaudeAuto-
Engineer) loops on `BaseEngineer._prompt_follow_up()` which awaits
`input(" > ")` via an executor. With `--json` or `--no-interactive`
that block on stdin defeats the scripted contract: pipelines like
`engineer <run_id> --json | jq` would hang before emitting the JSON.
- Add `interactive: bool = True` to BaseEngineer; `_prompt_follow_up`
now returns None immediately when interactive is False (still flushes
sync so partial output reaches disk).
- Thread `interactive` through `run_reverse_engineering` and all three
SDK constructors (Claude, OpenCode, Copilot — the latter two don't
use the follow-up loop today but are wired for symmetry / future).
- Thread `interactive` through `cli.run_engineer`, `cli.run_auto_capture`,
`cli.run_agent_capture`, and the auto-engineer constructors.
- The `agent` and `engineer` click commands derive `interactive = not
(as_json or no_interactive)` and pass it down.
Adds 6 regression tests in test_cli_followups.py covering: the base
short-circuit (input() raises if reached), default interactive=True
(REPL UX preserved), and that all three click-command flag combinations
(--json, --no-interactive, default) thread the right value through
run_engineer / run_agent_capture.
Reported by chatgpt-codex-connector on PR #65 (P2).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Caught by ruff F401 during end-to-end smoke testing of PR #65.
Required for scripted/CI/VPS use cases where no X server is available. Threads through the full chain to both ManualBrowser (Playwright launch) and the MCP-spawned browser (auto and chrome-mcp providers). Behavior: - `manual --headless`: Playwright launches Chromium with headless=True. Also drops `--no-sandbox` from `ignore_default_args` so Chromium can start on hosts without unprivileged user namespaces (Ubuntu 24.04+ with AppArmor restrictions). Headed mode keeps the existing stealth default (sandbox stripped for fingerprint realism). - `agent --headless` (auto provider): rae-playwright-mcp spawned with `--headless` so the MCP-controlled Chromium runs without UI. - `agent --headless` (chrome-mcp provider): drops `--autoConnect` from the MCP args because auto-connect requires a real headed Chrome with remote-debugging enabled. Adds `--headless` so the MCP spawns its own headless Chromium instead. The "auto-connect setup" preamble is replaced by a one-line note explaining the headless path. All three SDK paths (Claude/OpenCode/Copilot, both ClaudeAutoEngineer and the OpenCode/Copilot variants) accept and propagate `headless` via kwargs to their auto-engineer constructors. 8 new tests in test_cli_followups.py covering: click flag wiring on manual + agent (default false, --headless threads true), and the MCP arg construction for all four headed/headless × playwright/chrome-mcp combinations. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per design intent, `manual` is interactive-only: a human navigates a real, visible browser, clicks through flows, submits forms, etc. A headless variant defeats the purpose; agents and CI should use `agent --json --headless` instead. - Remove --headless from the manual click command (passing it now fails with `No such option`) - Remove headless parameter from ManualBrowser; restore the original Playwright launch (always headed, always strips --no-sandbox for fingerprint realism) - Restore run_manual_capture signature - Update the manual command docstring to make it explicit that the mode requires a human and to redirect agents to `agent --headless` Tests: replace the two `manual --headless` wiring tests with checks that (a) `manual --headless` exits non-zero with `No such option` and (b) `manual --help` mentions both "human" and "agent" so an agent inspecting --help self-routes correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
feat(cli): TTY-detect at REPL + engineer --json (issue #62 items 1, 2, 6)
1 participant
Summary
--jsonand--no-interactiveto theagentcommand so other agents can drive reverse engineering without TTY prompts.--jsonredirects all Rich output to stderr and emits a single stable JSON object on stdout (schema_version,status,run_id,prompt,url,mode,har_path,script_path,usage,error).run_auto_capturereturn shape so the JSON payload is stable regardless of the underlying engineer SDK (Claude/OpenCode/Copilot).list --jsonnow emits[]on empty history;show <id> --jsonemits a structured{"error": ...}and exits 1 when the run is missing.rungains--no-interactive(fail instead of opening the script-picker) and--auto-install(install missing deps without confirm).tests/test_cli_agent_json.py(10 new tests).Branched from
mainso it stacks cleanly without depending on #60.Example
```bash
reverse-api-engineer agent \
--prompt "capture the public jobs api" \
--url https://example.com/jobs \
--json | jq
{
"schema_version": 1,
"status": "ok",
"run_id": "deadbeef0001",
"har_path": "/.../recording.har",
"script_path": "/.../api_client.py",
...
}
```
Exit codes: `0` success, `1` runtime error, `2` misuse (missing required args).
Test plan
🤖 Generated with Claude Code
Summary by cubic
Makes the CLI fully scriptable and non-blocking for agents with
--json/--no-interactive, addsagent --headlessfor CI/headless runs, and standardizes JSON payloads and exit codes across commands. Also prevents the REPL from hanging when stdin isn’t a TTY.New Features
agentsupports--json(single payload on stdout; logs to stderr),--no-interactive(fail fast on missing input), and--headless(runs MCP-controlled browser headless; disables--autoConnectforchrome-mcp).engineeradds--json/--no-interactivewith a stable JSON schema; fields:schema_version,status,run_id,prompt,fresh,script_path,usage,error.agent --jsonpayload; fields:schema_version,status,run_id,prompt,url,mode,har_path,script_path,usage,error.list --jsonreturns[]on no results;show <id> --jsonreturns{"error": ...}and exits 1 when missing.runkeeps--no-interactive(no picker; requires--filewhen multiple scripts) and--auto-install(installs missing deps without confirm).--helpand command epilogs document examples, schemas, and exit codes; README adds a Scripted / Agent Usage section.0success,1runtime error,2misuse.Bug Fixes
status: "error"witherror: "interrupted"(even when caught internally).--helpto stderr, exits 2) to avoid hangs in CI/agents.har_pathresolves against the provided--output-dir.--jsonemits the full documented schema (fields present with nulls).--jsonoutputs exactly one JSON object; all Rich output is routed to stderr.Written for commit 7a8d8b6. Summary will update on new commits.
Greptile Summary
This PR adds a machine-readable CLI surface (
--json,--no-interactive,--auto-install) to theagent,list,show, andruncommands, and normalises the return shape ofrun_auto_captureso the JSON payload is stable across SDK backends. The implementation is well-structured and the test coverage is solid; the findings below are all P2 quality-of-life items.Confidence Score: 4/5
Safe to merge; all findings are P2 style/robustness concerns with no correctness impact on the happy path.
No P0 or P1 defects were found. Three P2 issues: inconsistent use of Rich's private _file attribute for console restoration, a test that doesn't truly validate stderr routing, and a loss of filtering context in the empty-JSON-array response for list --json. None of these affect the primary feature behaviour.
The _quiet_consoles_for_json() context manager in src/reverse_api/cli.py and the corresponding test in tests/test_cli_agent_json.py are the two areas that could use a quick follow-up.
Important Files Changed
Comments Outside Diff (1)
src/reverse_api/cli.py, line 2018-2025 (link)When
as_json=True, both an empty history and a filter that excludes all runs produce the same[]response. An agent consumer that passes--modeor--searchhas no way to tell whether the empty array means "no runs at all" or "your filter matched nothing", which could silently mask mistakes in filter arguments.Prompt To Fix With AI
Prompt To Fix All With AI
Reviews (1): Last reviewed commit: "feat: agent-friendly CLI surface" | Re-trigger Greptile