Skip to content

feat: add Runway Characters avatar plugin#1254

Merged
tinalenguyen merged 8 commits intolivekit:mainfrom
runwayml:add-runway-avatar-plugin
Apr 18, 2026
Merged

feat: add Runway Characters avatar plugin#1254
tinalenguyen merged 8 commits intolivekit:mainfrom
runwayml:add-runway-avatar-plugin

Conversation

@robinandeer
Copy link
Copy Markdown
Contributor

Description

Add @livekit/agents-plugin-runway — an avatar plugin for Runway Characters (GWM-1 avatars). Follows the same pattern as the Bey/Anam/Lemonslice avatar plugins and mirrors the Python livekit-plugins-runway implementation (livekit/agents#5355).

Changes Made

  • New plugin plugins/runway/AvatarSession class with preset or custom avatar support
  • Sends livekit payload (url, token, roomName, agentIdentity) so Runway's avatar worker joins the agent's existing room
  • Audio routed via DataStreamAudioOutput at 16kHz
  • Sends User-Agent: @livekit/agents-plugin-runway/<version> header for server-side debugging
  • Example agent examples/src/runway_avatar.ts using Gemini Realtime + Silero VAD
  • Added Runway env vars to turbo.json globalEnv

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title
  • Video demo: Tested manually against Runway staging API

Testing

  • Plugin builds successfully
  • Example agent connects to LiveKit room and starts Runway avatar session
  • Manual testing against Runway realtime sessions API

Additional Notes

Environment variables needed:

  • RUNWAYML_API_SECRET — Runway API key
  • RUNWAYML_BASE_URL — (optional) Runway API URL, defaults to https://api.dev.runwayml.com
  • RUNWAY_AVATAR_PRESET_ID — (optional) preset avatar ID, defaults to cat-character
  • Standard LiveKit env vars (LIVEKIT_URL, LIVEKIT_API_KEY, LIVEKIT_API_SECRET)
  • GOOGLE_API_KEY — for the example agent (Gemini Realtime)

Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

Add `@livekit/agents-plugin-runway` for Runway Characters avatar integration.
Follows the same pattern as the Bey/Tavus avatar plugins — mirrors the Python
`livekit-plugins-runway` implementation (livekit/agents#5355).

- Plugin: `AvatarSession` with preset or custom avatar support
- Sends `livekit` payload so Runway's avatar worker joins the agent's room
- Audio routed via `DataStreamAudioOutput` at 16kHz
- Sends `User-Agent` header for server-side debugging
- Example agent using Gemini Realtime + Silero VAD
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 14, 2026

🦋 Changeset detected

Latest commit: 1b6d004

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 25 packages
Name Type
@livekit/agents-plugin-runway Patch
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@toubatbrian
Copy link
Copy Markdown
Contributor

toubatbrian commented Apr 14, 2026

@codex review and check the plugin package format against others. Compared with other avatar plugins and check for alignment or discrepancies

chatgpt-codex-connector[bot]

This comment was marked as resolved.

livekitToken: string,
roomName: string,
agentIdentity: string,
): Promise<void> {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use ThrowsPromise here for error type safety

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ThrowsPromise doesn't exist in the codebase — searched all of agents/ and plugins/ with no matches. No other avatar plugin (bey, trugen, hedra, lemonslice, anam) annotates error types on their methods either. Happy to add it if there's a utility type you have in mind, but for now this matches the existing plugin convention.

Copy link
Copy Markdown
Contributor

@toubatbrian toubatbrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a small comment, otherwise LGTM! You may also want to fix the linting issue by running pnpm run lint:fix

Copy link
Copy Markdown
Member

@tinalenguyen tinalenguyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thank you!

@tinalenguyen tinalenguyen merged commit ad09698 into livekit:main Apr 18, 2026
6 checks passed
@github-actions github-actions Bot mentioned this pull request Apr 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants