Skip to content

LLM generates unsupported method names, eg. dblclick #1608

@neongreen

Description

@neongreen

I'm calling observe() without arguments and getting this as one of the results:

{"description":"Static text element labeled 'DOUBLE-CLICK HERE' for the Double-Click Detection test.","selector":"xpath=/html[1]/body[1]/div[1]/div[2]/div[1]/div[2]/section[2]/div[1]/div[2]/div[1]","arguments":[],"backendNodeId":0,"method":"dblclick"}

Then act fails with:

Failed to perform act after self-heal: Method dblclick not supported

I'm using Stagehand API

LLM analysis, feel free to ignore

LLM: This happens because `doubleClick` exists in `METHOD_HANDLER_MAP` but is not included in the `SupportedPlaywrightAction` enum that gets passed to the LLM prompt.

Root Cause

There's a mismatch between the methods the LLM is told about and the methods that are actually supported:

SupportedPlaywrightAction enum (9 methods) - sent to LLM via buildActPrompt():

// packages/core/lib/v3/types/private/handlers.ts
export enum SupportedPlaywrightAction {
  CLICK = "click",
  FILL = "fill",
  TYPE = "type",
  PRESS = "press",
  SCROLL = "scrollTo",
  NEXT_CHUNK = "nextChunk",
  PREV_CHUNK = "prevChunk",
  SELECT_OPTION_FROM_DROPDOWN = "selectOptionFromDropdown",
  HOVER = "hover",
}

METHOD_HANDLER_MAP (16 methods) - actually supported at runtime:

// packages/core/lib/v3/handlers/handlerUtils/actHandlerUtils.ts
const METHOD_HANDLER_MAP = {
  scrollIntoView,
  scrollByPixelOffset,
  scrollTo: scrollElementToPercentage,
  scroll: scrollElementToPercentage,
  "mouse.wheel": wheelScroll,
  fill: fillOrType,
  type: typeText,
  press: pressKey,
  click: clickElement,
  doubleClick,        // ← Not in enum
  dragAndDrop,        // ← Not in enum
  nextChunk: scrollToNextChunk,
  prevChunk: scrollToPreviousChunk,
  selectOptionFromDropdown: selectOption,
  selectOption: selectOption,
  hover: hover,
};

Missing from enum but supported at runtime:

  • doubleClick
  • dragAndDrop
  • scroll (alias for scrollTo)
  • scrollIntoView
  • scrollByPixelOffset
  • mouse.wheel
  • selectOption (alias)

Additionally, the Zod schema in inference.ts uses z.string() instead of z.enum(), so invalid method names pass schema validation and only fail at execution time.

Steps to Reproduce

  1. Call act() with instruction: "double-click the submit button"
  2. LLM generates { method: "dblclick", ... } (or similar variant)
  3. performUnderstudyMethod() throws Method dblclick not supported

Suggested Fix

  1. Add missing methods to SupportedPlaywrightAction enum:
export enum SupportedPlaywrightAction {
  CLICK = "click",
  DOUBLE_CLICK = "doubleClick",  // Add
  FILL = "fill",
  TYPE = "type",
  PRESS = "press",
  SCROLL = "scrollTo",
  NEXT_CHUNK = "nextChunk",
  PREV_CHUNK = "prevChunk",
  SELECT_OPTION_FROM_DROPDOWN = "selectOptionFromDropdown",
  HOVER = "hover",
  DRAG_AND_DROP = "dragAndDrop",  // Add
}
  1. Consider using z.enum() in the schema to provide stricter validation:
method: z.enum(Object.values(SupportedPlaywrightAction)).describe("...")

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions