Skip to content

Conversation

@tusharmctrl
Copy link

LocalBrowserProvider Configuration

This PR explains and have added the use-case on how to use the clientType setting when configuring a HyperAgent instance with the LocalBrowserProvider.

Usage

Use the clientType parameter to control whether the browser context should emulate a desktop or a mobile device.

const agent = new HyperAgent({
  llm: llm,
  browserProvider: "Local",
  clientType: "desktop", // Options: 'desktop' or 'mobile'
  debug: true,
});

Notes

  • Default value: "desktop"
  • This solution will work on both LocalProvider and HyperBrowserProvide.

: LocalBrowserProvider;
private browserProviderType: T;
private actions: Array<AgentActionDefinition> = [...DEFAULT_ACTIONS];
private clientType: "desktop" | "mobile";
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you take the types directly from the devices offered from the devices list ?
Something like private clientType?: keyof typeof devices;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the constructor too can you assign the clientType on the object instance ?
Like this.clientType = params.clientType

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes - I tried this, but it wouldn't give actual suggestions to user - reason being, playwright doesnt do export on Devices type. So, considering user perspective - asking user to select mobile / desktop - would make more sense since we're not able to provide suggestions.


if (device === "mobile") {
const iPhone = devices["iPhone 12"];
return await this.session.newContext({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pass in the entire device object, don't specify individual params here

 this.session.newContext({...devices[device]})

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried that - that wouldn't work because, there's this property where sharp is contradicting.

When we pass the entire object, it will throw an error like:

Error: Image to composite must have same dimensions or smaller
    at Sharp.toBuffer (/home/tushar-rupani/missionctrl/HyperAgent/node_modules/sharp/lib/output.js:163:17)
    at compositeScreenshot (/home/tushar-rupani/missionctrl/HyperAgent/src/agent/tools/agent.ts:38:6)
    at runAgentTask (/home/tushar-rupani/missionctrl/HyperAgent/src/agent/tools/agent.ts:156:31)
    at HyperAgent.executeTask (/home/tushar-rupani/missionctrl/HyperAgent/src/agent/index.ts:335:14)
    at <anonymous> (/home/tushar-rupani/missionctrl/HyperAgent/src/agent/test.ts:13:18)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason behind that is deviceScaleFactor

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And isMobile too.

return this.session;
}

public async getContext(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add this method on the abstract base class that they inherit from ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants