Skip to content

Add browse cdp command for streaming CDP events#1905

Merged
derekmeegan merged 4 commits intomainfrom
derek/add_cdp_tailing
Apr 7, 2026
Merged

Add browse cdp command for streaming CDP events#1905
derekmeegan merged 4 commits intomainfrom
derek/add_cdp_tailing

Conversation

@derekmeegan
Copy link
Copy Markdown
Contributor

@derekmeegan derekmeegan commented Mar 29, 2026

Summary

  • Adds browse cdp <url|port> command that connects to any CDP WebSocket target and streams DevTools protocol events as NDJSON to stdout
  • Supports --domain filtering (defaults to Network, Console, Runtime, Log, Page), --pretty for human-readable output
  • Auto-attaches to page targets and enables CDP domains per session, handles multi-tab scenarios
  • Output is pipeable to files (browse cdp ws://... > log.jsonl) or jq (browse cdp ws://... | jq '.method')
  • Accepts bare port numbers (e.g. browse cdp 9222) via existing resolveWsTarget utility

Test plan

  • Tested NDJSON output piped to file against a Browserbase session
  • Tested --pretty mode shows human-readable event summaries
  • Verified jq compatibility with NDJSON output
  • Tested graceful SIGINT shutdown
  • Test against local Chrome with --remote-debugging-port=9222
  • Test --domain filtering (e.g. --domain Network only)

🤖 Generated with Claude Code

Adds a new `browse cdp <url|port>` command that connects to any Chrome
DevTools Protocol target and streams events as NDJSON to stdout. Supports
--domain filtering, --pretty for human-readable output, and clean piping
to files or jq. Enables Network, Console, Runtime, Log, and Page domains
by default with automatic multi-tab target tracking.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Mar 29, 2026

🦋 Changeset detected

Latest commit: 253cf91

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 0 packages

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files

Confidence score: 4/5

  • This PR looks safe to merge overall: the reported issue is low-to-moderate severity (4/10) and focused on error visibility rather than core command execution flow.
  • In packages/cli/src/index.ts, CDP command failures are currently only surfaced in pretty output, so NDJSON users can miss invalid domains or failed enable calls when piping output.
  • Pay close attention to packages/cli/src/index.ts - ensure CDP errors are logged in NDJSON mode to avoid silent operational failures.
Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/cli/src/index.ts">

<violation number="1" location="packages/cli/src/index.ts:2962">
P2: Log CDP command errors even in NDJSON mode. Right now errors are only printed in pretty mode, so invalid domains or failed enable calls are silently ignored when piping output.</violation>
</file>
Architecture diagram
sequenceDiagram
    participant User as User / Shell
    participant CLI as Browse CLI
    participant Resolver as resolveWsTarget
    participant Browser as CDP Server (Chrome/Browserbase)

    Note over User,Browser: Initialization Flow
    User->>CLI: browse cdp <url|port> [--pretty] [--domain]
    CLI->>Resolver: resolveWsTarget(target)
    Resolver-->>CLI: ws://... URL
    
    CLI->>Browser: NEW: Establish WebSocket Connection
    
    opt Connection Successful
        Browser-->>CLI: onOpen
        CLI->>Browser: NEW: Target.setAutoAttach (filter: page)
        CLI->>Browser: NEW: Target.setDiscoverTargets (discover: true)
    end

    Note over User,Browser: Session & Event Handling
    
    loop Every Message
        Browser-->>CLI: Protocol Message (NDJSON)
        
        alt Message is response to CLI command
            CLI->>CLI: Match ID & discard
        else NEW: Message is "Target.attachedToTarget"
            CLI->>CLI: Track targetId -> sessionId map
            loop for each requested domain
                CLI->>Browser: NEW: [Domain].enable (with sessionId)
            end
        else NEW: Message is "Target.detachedFromTarget"
            CLI->>CLI: Remove session from tracking
        else NEW: General CDP Event (Network/Console/etc)
            alt --pretty flag set
                CLI->>CLI: Format human-readable string
            else default (NDJSON)
                CLI->>CLI: Stringify original JSON
            end
            
            CLI->>User: Write to stdout (+ \n)
            
            opt Pipe Broken (EPIPE)
                CLI->>CLI: Graceful exit(0)
            end
        end
    end

    Note over User,Browser: Shutdown Flow
    
    alt SIGINT / SIGTERM received
        CLI->>Browser: Close WebSocket
        CLI->>User: Exit process
    else Browser closes connection
        CLI-->>User: "Disconnected" (if --pretty)
    end
Loading

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread packages/cli/src/index.ts Outdated
Also adds @types/ws to fix typecheck.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
derekmeegan and others added 2 commits March 29, 2026 13:00
Previously CDP command errors (e.g. failed domain enable) were only
logged in pretty mode. Now errors always go to stderr regardless of
output mode, so piped NDJSON users see failures too.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@derekmeegan derekmeegan merged commit a4ca430 into main Apr 7, 2026
562 of 571 checks passed
@miguelg719 miguelg719 mentioned this pull request Apr 8, 2026
derekmeegan added a commit to browserbase/skills that referenced this pull request Apr 27, 2026
* Add browser-observability skill

Capture the full CDP firehose plus periodic screenshots and DOM dumps
alongside any browser automation (browse, Stagehand, Playwright, Puppeteer
— anything that speaks CDP), then bisect the stream into per-page
searchable buckets keyed by top-level navigations.

The skill is purely observational: it adds a second CDP client that
enables read-only domains (Network, Console, Runtime, Log, Page) and
never sends action commands, so it can attach to an in-flight automation
without disrupting it.

Layout produced under .o11y/<run-id>/:
  manifest.json, index.jsonl
  cdp/raw.ndjson                              full firehose, NDJSON
  cdp/summary.json                            {sessionId, duration, totalEvents, pages[]}
  cdp/<domain>/...                            session-wide buckets
  cdp/pages/<pid>/                            per-page slices, indexed by Page.frameNavigated
    summary.json, raw.jsonl, network/, console/, page/, ...
  screenshots/<iso-ts>.png, dom/<iso-ts>.html
  browserbase/                                added by bb-finalize.sh on remote runs

Scripts (pure bash + jq, no Python/Node deps beyond the existing browse CLI):
  start-capture.sh / stop-capture.sh / snapshot-loop.sh
  bisect-cdp.sh — splits raw.ndjson into session-wide and per-page buckets,
                  writes the structured summary.json
  query.sh      — drill-down helper: list, page <pid> [bucket], errors,
                  hosts, host <h>, timeline, summary
  bb-capture.sh / bb-finalize.sh — Browserbase wrappers; bb-capture handles
                  --keep-alive + connectUrl resolution + manifest stamping with
                  the session id, region, and live debugger URL; bb-finalize
                  writes browserbase/{session.json, logs.json, downloads.zip}
                  and optionally releases the session

The CDP firehose is sourced via `browse cdp <target>`, which streams
DevTools-protocol events as NDJSON. That command requires
@browserbasehq/browse-cli alpha or a release that includes
browserbase/stagehand#1905; lifecycle events specifically require
browserbase/stagehand#2056.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Convert browser-observability scripts from bash to .mjs

Mirrors the cookie-sync skill's pattern: each script becomes a Node ESM
module, with a shared scripts/lib.mjs for common helpers and a minimal
package.json (no dependencies — pure Node stdlib, so no `npm install`
step is needed).

  scripts/lib.mjs                      shared helpers, BUCKETS table, isTopNav
  scripts/start-capture.mjs            spawn detached `browse cdp` + sampler
  scripts/snapshot-loop.mjs            screenshot+DOM+url poll loop (internal)
  scripts/stop-capture.mjs             SIGTERM+grace+SIGKILL, sweep .partials
  scripts/bisect-cdp.mjs               page-aware bisect + structured summary.json
  scripts/query.mjs                    drill-down: list/page/errors/hosts/host/timeline
  scripts/bb-capture.mjs               Browserbase wrapper (--new or attach by id)
  scripts/bb-finalize.mjs              pull session.json/logs.json/downloads.zip

Output layout, per-page bisection logic, summary.json shape, and the
query.mjs subcommands are identical to the bash versions. SKILL.md,
REFERENCE.md, and EXAMPLES.md updated to invoke `node scripts/foo.mjs`
instead of `bash scripts/foo.sh`.

Dogfooded end-to-end against a Browserbase session and against the
saved /Users/d/Desktop/browserbase/o11y-demo-1 run — page bisection,
durations, error attribution, host counts all match the bash output.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Rename browser-observability → browser-trace

Shorter, more action-oriented name that reads cleanly as a slash command
and pairs naturally with the autobrowse skill (which can use the trace
output as the structured "what just happened" data for its loop).

Renames:
- skills/browser-observability/ → skills/browser-trace/
- SKILL.md frontmatter `name`, doc H1s, package.json name
- marketplace.json plugin entry (name, skills path, description, keywords)
- README.md skills table row
- "observer" → "tracer" throughout the prose for consistency with the new name

No behaviour change — all script interfaces, output layout, summary.json
shape, and query.mjs subcommands are unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Address Cursor Bugbot review on browser-trace

Fix three issues flagged on PR #83:

- query.mjs cmdTimeline: was printing every navigation followed by every
  lifecycle event, so all `[NAV …]` lines bunched before any
  `[init]`/`[load]` lines. Now reads cdp/raw.ndjson directly and emits
  top-level frameNavigated and lifecycle events in actual stream order.

- query.mjs cmdHost: was using `url.startsWith('https://' + hostname)`,
  which would match `https://api.example.com.evil.tld/...` when the user
  asked for `api.example.com`. Switched to exact `new URL(url).hostname`
  comparison via a small `hostMatches` helper.

- query.mjs: removed the unused `emitJsonl` helper (defined but never
  called — same class of dead-code issue Bugbot raised against the bash
  version's `cat_bucket_across`).

Bugbot also flagged the bash bb-capture.sh's `| grep ... || true` that
silently swallowed start-capture failures. The .mjs version already
handles this correctly via `if (start.status !== 0) process.exit(...)`,
so no change needed there.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fix Console domain bucket + align hosts/host port handling

Two more issues from Cursor Bugbot on PR #83:

1. Console errors and warnings were being silently dropped from the
   per-page summary. computePageSummary classified each event by
   `method.split('.')[0]`, which puts Runtime.consoleAPICalled events
   into a `Runtime` bucket — but errors/warnings from those events were
   being recorded under the key `Console`. The output loop iterates
   `counts.entries()` and looks up errors/warnings by that domain key,
   so the Console errors/warnings never made it into the rollup.

   Added a `domainFor(method)` helper that returns 'Console' for
   `Runtime.consoleAPICalled` and the standard `method.split('.')[0]`
   otherwise. Now matches the schema documented in SKILL.md.

2. cmdHosts grouped by `URL.host` (port included) but cmdHost compared
   against `URL.hostname` (port stripped), so `hosts` output couldn't
   be piped into `host` when a port was present. Switched
   `hostMatches` to `URL.host` for consistency. Impostor protection
   (rejecting `example.com.evil.tld` for `example.com` queries) is
   preserved by the exact-equality check.

Verified against /Users/d/Desktop/browserbase/o11y-demo-1: every page
now shows Console + Runtime as distinct domain blocks; page 4
(httpbin/418) correctly attributes the 418 status to a Log error.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants