Add right/middle click support on act and observe

[miguelg719]

why

Locator.click has support for both right and middle clicks, but these are not properly piped or prompted for act or observe

what changed

• clickElement on act handler reads the button from args (defaults to left if undefined)
• Updated act/observe prompts to request "right" or "middle"
• Moved MouseButton type to types from locator

test plan

• Both env local & browserbase work with multiple models on https://www.onlinemictest.com/mouse-test/

---

Summary by cubic

Added right and right middle click support to act/observe so flows can trigger context menus and middle-click behaviors. Handlers now pass the mouse button to Locator.click.

• New Features
  • clickElement reads the button from args and calls locator.click({ button }).
  • Updated act/observe prompts to request "right" or "middle" for non-left clicks.
  • Exported MouseButton type for use across handlers.

^{Written for commit 10a9866. Summary will update on new commits. Review in cubic}

[changeset-bot]

🦋 Changeset detected

Latest commit: 10a9866

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[greptile-apps]

Greptile Overview

Greptile Summary

Added support for right and middle click actions in the act and observe methods. The LLM prompts now instruct the model to specify "right" or "middle" as arguments when choosing non-left click actions. The clickElement handler reads the button type from args[0] and passes it to locator.click().

Changes:

• Updated buildObserveSystemPrompt and buildActPrompt to include instructions for specifying right/middle click
• Modified clickElement in actHandlerUtils.ts to extract button from args and pass to locator
• Exported MouseButton type from locator.ts for use in handlers

Minor concern:

• When args[0] is undefined (typical for left-click), it's passed as undefined as MouseButton which works due to the default in locator.click but could be more explicit

Confidence Score: 4/5

• This PR is safe to merge with low risk - it's a straightforward feature addition that enhances click functionality
• The implementation correctly adds right/middle click support by updating prompts and plumbing the button argument through to the locator. The locator already handles defaults properly. Minor style improvement could make undefined handling more explicit, but current code functions correctly.
• No files require special attention - the changes are simple and well-contained

Important Files Changed

Filename	Overview
packages/core/lib/prompt.ts	Added prompt instructions for right/middle click support to observe and act system prompts
packages/core/lib/v3/handlers/handlerUtils/actHandlerUtils.ts	Modified clickElement to read button from args and pass to locator.click; should handle undefined args[0]
packages/core/lib/v3/understudy/locator.ts	Exported MouseButton type for external use

Sequence Diagram

sequenceDiagram
    participant User
    participant Act/Observe
    participant LLM
    participant actHandlerUtils
    participant Locator
    participant Browser

    User->>Act/Observe: Request action (e.g., "right click button")
    Act/Observe->>LLM: Send instruction + DOM snapshot
    Note over LLM: Prompt includes instruction:<br/>"provide right or middle as argument"
    LLM-->>Act/Observe: Returns element + method="click" + args=["right"|"middle"]
    Act/Observe->>actHandlerUtils: performUnderstudyMethod(method, xpath, args)
    actHandlerUtils->>actHandlerUtils: clickElement(ctx)
    Note over actHandlerUtils: Reads args[0] as MouseButton
    actHandlerUtils->>Locator: locator.click({ button: args[0] })
    Note over Locator: Defaults to "left" if button undefined
    Locator->>Browser: Input.dispatchMouseEvent(button)
    Browser-->>Locator: Click executed
    Locator-->>actHandlerUtils: Success
    actHandlerUtils-->>Act/Observe: Action completed
    Act/Observe-->>User: Result

Loading

[greptile-apps]

_{3 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

[cubic-dev-ai]

1 issue found across 3 files

Confidence score: 4/5

• Only a minor concern about exposing an internal helper type as public API in packages/core/lib/v3/understudy/locator.ts, which could reduce refactoring flexibility but is unlikely to break behavior now
• Overall risk seems low; this looks more like an API hygiene consideration than a functional regression
• Pay close attention to packages/core/lib/v3/understudy/locator.ts - internal helper type exposure and intended stability

Prompt for AI agents (all issues)

Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/core/lib/v3/understudy/locator.ts">

<violation number="1" location="packages/core/lib/v3/understudy/locator.ts:20">
P2: Avoid exposing this internal helper type as a public API unless it’s intentionally stable and user‑facing; keep it module‑private to preserve refactoring flexibility.

(Based on your team's feedback about avoiding exposing internal types as public APIs.) [FEEDBACK_USED]</violation>
</file>

Architecture diagram

sequenceDiagram
    *   participant LLM as LLM (Reasoning Engine)
    *   participant Prompt as prompt.ts
    *   participant Handler as actHandlerUtils (clickElement)
    *   participant Locator as Locator (Understudy)
    *   participant Browser as Browser Instance

    Note over LLM, Browser: Execution Flow for Act/Observe

    Prompt->>LLM: CHANGED: System prompt now instructs LLM to provide "right" or "middle" for non-left clicks
    LLM-->>Handler: Returns action (click) + args (e.g., ["right"])
    
    rect rgb(23, 37, 84)
    Note right of Handler: NEW: Action Context Handling
    Handler->>Handler: Extract args[0] as MouseButton
    Handler->>Locator: CHANGED: click({ button: args[0] })
    end

    Locator->>Browser: Trigger mouse click (button: left|right|middle)
    Browser-->>Locator: Click event completed
    Locator-->>Handler: Success/Failure
    Handler-->>LLM: Result of interaction

Loading

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.}

[cubic-dev-ai]

P2: Avoid exposing this internal helper type as a public API unless it’s intentionally stable and user‑facing; keep it module‑private to preserve refactoring flexibility.

(Based on your team's feedback about avoiding exposing internal types as public APIs.)

View Feedback

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/understudy/locator.ts, line 20:

<comment>Avoid exposing this internal helper type as a public API unless it’s intentionally stable and user‑facing; keep it module‑private to preserve refactoring flexibility.

(Based on your team's feedback about avoiding exposing internal types as public APIs.) </comment>

<file context>
@@ -17,7 +17,7 @@ import { NormalizedFilePayload } from "../types/private/locator";
 const MAX_REMOTE_UPLOAD_BYTES = 50 * 1024 * 1024; // 50MB guard copied from Playwright
 
-type MouseButton = "left" | "right" | "middle";
+export type MouseButton = "left" | "right" | "middle";
 
 /**
</file context>

Suggested change

	export type MouseButton = "left" | "right" | "middle";
	type MouseButton = "left" | "right" | "middle";

[seanmcguire12]

dont forget changeset

[seanmcguire12]

exporting is fine but maybe move to types/public/locator.ts