Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,23 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Removed
- **`browser-use` and `stagehand` agent providers**: Both have been retired due to upstream churn. Agent mode now offers only `auto` (Playwright MCP) and `chrome-mcp` (Chrome DevTools MCP). Configs with `agent_provider` set to `browser-use` or `stagehand` migrate to `auto`; the `browser_use_model` and `stagehand_model` keys are dropped silently. The `[agent]` optional-dependency extra is removed.
- **Tag system (`@record-only`, `@codegen`, `@id`, `@docs`, `@help`)**: The inline tag syntax inside REPL prompts has been removed. Surviving capabilities have first-class CLI flags / arguments instead:
- `@record-only` → `manual --no-engineer` (already existed)
- `@id <run_id>` → `engineer <run_id>` (positional, already existed)
- `@id <run_id> <prompt>` → `engineer <run_id> --prompt "..."` (new flag; layered as additional instructions on top of the captured run's original goal)
- `@id <run_id> --fresh <prompt>` → `engineer <run_id> --fresh --prompt "..."` (new flag; with `--fresh`, `--prompt` fully replaces the original goal)
- `@codegen` and Playwright-action recording: removed entirely with the `ActionRecorder` and `playwright_codegen` modules (low usage, replaced by the standard capture+engineer flow)
- `@docs` (OpenAPI generation): removed for now; will return as a dedicated `docs` subcommand in a follow-up
- `@help`: superseded by the existing `/help` slash command in the REPL
- **`--reverse-engineer/--no-engineer` flag on `agent`**: was parsed but never propagated to the auto-capture path — removed. Agent mode runs an integrated capture + engineering pipeline; use `manual --no-engineer` for HAR-only recordings.

### Added
- **`engineer --prompt`** and **`engineer --fresh`** CLI flags to replace the equivalent `@id <run_id> [--fresh] <prompt>` REPL syntax.

## [0.7.1] - 2026-04-06

### Fixed
Expand Down
86 changes: 8 additions & 78 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,6 @@ No more manual reverse engineering—just browse, capture, and get clean API cod
- [Engineer Mode](#engineer-mode)
- [Agent Mode](#agent-mode)
- [Collector Mode](#collector-mode)
- [Tags](#tags)
- [Configuration](#-configuration)
- [Model Selection](#model-selection)
- [Agent Configuration](#agent-configuration)
Expand All @@ -48,7 +47,7 @@ No more manual reverse engineering—just browse, capture, and get clean API cod
## ✨ Features

- 🌐 **Browser Automation**: Built on Playwright with stealth mode for realistic browsing
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents (auto mode with MCP, browser-use, stagehand)
- 🤖 **Autonomous Agent Mode**: Fully automated browser interaction using AI agents via MCP (Playwright MCP or Chrome DevTools MCP)
- 📊 **HAR Recording**: Captures all network traffic in HTTP Archive format
- 🧠 **AI-Powered Generation**: Uses Claude 4.6 to analyze traffic and generate clean Python code
- 🔍 **Collector Mode**: Data collection with automatic JSON/CSV export
Expand All @@ -57,7 +56,6 @@ No more manual reverse engineering—just browse, capture, and get clean API cod
- 📦 **Production Ready**: Generated scripts include error handling, type hints, and documentation
- 💾 **Session History**: All runs saved locally with full message logs
- 💰 **Cost Tracking**: Detailed token usage and cost estimation with cache support
- 🏷️ **Tag System**: Powerful tags for fine-grained control (@record-only, @codegen, @docs, @id)

### Limitations

Expand All @@ -68,21 +66,12 @@ No more manual reverse engineering—just browse, capture, and get clean API cod

### Using uv (recommended)
```bash
# Basic installation
uv tool install reverse-api-engineer

# With agent mode support (includes browser-use with HAR recording)
uv tool install 'reverse-api-engineer[agent]' --with 'browser-use @ git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455'
```

### Using pip
```bash
# Basic installation
pip install reverse-api-engineer

# With agent mode support
pip install 'reverse-api-engineer[agent]'
pip install git+https://github.com/browser-use/browser-use.git@49a345fb19e9f12befc5cc1658e0033873892455
```

### Post-installation
Expand Down Expand Up @@ -174,9 +163,8 @@ Fully automated browser interaction using AI agents:

**Agent Provider Options:**

- **auto** (default): Uses MCP-based browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow. No additional installation required beyond the base package.
- **browser-use**: Uses browser-use library for browser automation. Requires installation with `[agent]` extra and browser-use from specific git commit (includes HAR recording support).
- **stagehand**: Uses Stagehand for browser automation with Computer Use models.
- **auto** (default): Uses Playwright MCP browser automation with Claude Agent SDK & Opencode. Combines browser control and real-time reverse engineering in a single workflow.
- **chrome-mcp**: Uses [Chrome DevTools MCP](https://www.npmjs.com/package/chrome-devtools-mcp) to drive your real Chrome browser (with existing sessions, cookies, and auth). Requires Chrome 146+ and Node.js 20.19+.

Change agent provider in `/settings` → "agent provider".

Expand Down Expand Up @@ -210,58 +198,20 @@ $ reverse-api-engineer
# Data saved to: ./collected/js_frameworks/
```

## 🏷️ Tags

Tags provide additional control and functionality within each mode:

### Manual/Agent Mode Tags

- **`@record-only`** - Record HAR file only, skip reverse engineering step
- Example: `@record-only navigate checkout flow`
- Useful when you want to capture traffic for later analysis

- **`@codegen`** - Record browser actions and generate Playwright automation script
- Example: `@codegen navigate to google`
- Captures clicks, fills, and navigations to create a reusable Playwright script

### Engineer Mode Tags

- **`@id <run_id>`** - Switch context to a specific run ID
- Example: `@id abc123`
- Loads a previous capture session for re-engineering

- **`@id <run_id> <prompt>`** - Run engineer on a specific run with instructions
- Example: `@id abc123 extract user profile`
- Re-processes a capture with new instructions

- **`@id <run_id> --fresh <prompt>`** - Start fresh (ignore previous scripts)
- Example: `@id abc123 --fresh restart analysis`
- Generates new code from scratch, ignoring previous implementations

- **`@docs`** - Generate API documentation (OpenAPI spec) for the latest run
- Example: `@docs`
- Creates OpenAPI specification from captured traffic

- **`@id <run_id> @docs`** - Generate API documentation for a specific run
- Example: `@id abc123 @docs`
- Creates OpenAPI specification for a specific capture session

## 🔧 Configuration

Settings stored in `~/.reverse-api/config.json`:
```json
{
"agent_provider": "auto",
"browser_use_model": "bu-llm",
"claude_code_model": "claude-sonnet-4-6",
"collector_model": "claude-sonnet-4-6",
"opencode_model": "claude-sonnet-4-6",
"opencode_provider": "anthropic",
"output_dir": null,
"output_language": "python",
"real_time_sync": true,
"sdk": "claude",
"stagehand_model": "openai/computer-use-preview-2025-03-11"
"sdk": "claude"
}
```

Expand All @@ -284,32 +234,12 @@ If you use Opencode, look at the [models](https://models.dev).
Configure AI agents for autonomous browser automation.

**Agent Providers:**
- **auto** (default): MCP-based browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.
- **browser-use**: Supports Browser-Use LLM, OpenAI, and Google models. Requires installation with `[agent]` extra.
- **stagehand**: Supports OpenAI and Anthropic Computer Use models

**Agent Models:**
- **auto** (default): Playwright MCP browser automation with real-time reverse engineering. Uses Claude Agent SDK with browser MCP tools. Combines browser control and API reverse engineering in a single unified workflow. Works with Claude SDK (default) or OpenCode SDK.
- **chrome-mcp**: Drives your real Chrome browser via [Chrome DevTools MCP](https://www.npmjs.com/package/chrome-devtools-mcp). Useful when you need existing sessions, cookies, or auth. Requires Chrome 146+ and Node.js 20.19+; enable auto-connect at `chrome://inspect/#remote-debugging`.

**Browser-Use Provider:**
- `bu-llm` (default) - Requires `BROWSER_USE_API_KEY`
- `openai/gpt-4`, `openai/gpt-3.5-turbo` - Requires `OPENAI_API_KEY`
- `google/gemini-pro`, `google/gemini-1.5-pro` - Requires `GOOGLE_API_KEY`

**Stagehand Provider (Computer Use only):**
- `openai/computer-use-preview-2025-03-11` - Requires `OPENAI_API_KEY`
- `anthropic/claude-sonnet-4-6-20260301` - Requires `ANTHROPIC_API_KEY`
- `anthropic/claude-haiku-4-5-20251001` - Requires `ANTHROPIC_API_KEY`
- `anthropic/claude-opus-4-6-20260301` - Requires `ANTHROPIC_API_KEY`

**Setting API Keys:**
```bash
export BROWSER_USE_API_KEY="your-api-key" # For Browser-Use
export OPENAI_API_KEY="your-api-key" # For OpenAI models
export ANTHROPIC_API_KEY="your-api-key" # For Anthropic models
export GOOGLE_API_KEY="your-api-key" # For Google models
```
The agent's reasoning model is the same as the SDK model — see [Model Selection](#model-selection).

Change in `/settings` → "agent provider" and "agent model"
Change in `/settings` → "agent provider"

### SDK Selection

Expand Down
5 changes: 0 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -36,11 +36,6 @@ dependencies = [
]

[project.optional-dependencies]
agent = [
# browser-use must be installed separately from git (PyPI doesn't allow git dependencies)
# See installation instructions in README
"stagehand"
]
pricing = [
"litellm>=1.83.0",
]
Expand Down
51 changes: 0 additions & 51 deletions src/reverse_api/action_recorder.py

This file was deleted.

1 change: 0 additions & 1 deletion src/reverse_api/base_engineer.py
Original file line number Diff line number Diff line change
Expand Up @@ -504,7 +504,6 @@ def _build_prompts(self) -> tuple[str, str]:
scripts_dir=str(self.scripts_dir),
existing_client_guidance=self._get_existing_client_guidance(),
additional_instructions=additional_instructions,
tag_extra="@docs" if is_docs else "",
tag_mode_label="Documentation" if is_docs else "Re-engineer",
run_id=self.run_id,
har_parent=str(self.har_path.parent),
Expand Down
Loading
Loading