Skip to content

feat(stt): add FallbackAdapter for automatic STT provider failover#1278

Merged
theomonnom merged 7 commits intolivekit:mainfrom
lottiehq-oss:feat/stt-fallback-adapter
Apr 21, 2026
Merged

feat(stt): add FallbackAdapter for automatic STT provider failover#1278
theomonnom merged 7 commits intolivekit:mainfrom
lottiehq-oss:feat/stt-fallback-adapter

Conversation

@drain-zine
Copy link
Copy Markdown
Contributor

@drain-zine drain-zine commented Apr 20, 2026

Description

Port the Python livekit-agents/livekit/agents/stt/fallback_adapter.py to TypeScript so STT has parity with the existing llm.FallbackAdapter and tts.FallbackAdapter. When the active STT fails, the adapter switches to the next available provider; failed providers are monitored by a parallel probe stream on live audio and flipped back to available on the first non-empty FINAL transcript. Non-streaming STTs are auto-wrapped with stt.StreamAdapter when a VAD is provided.

Supersedes the stale #1083. Closes the STT half of #868.

Project specific fork of this adapter has been running in production at Lottie since 2026-04-20 on a LiveKit voice-agent (AssemblyAI primary + Deepgram Flux fallback). Tested by running agent in playground and emitting synthetic error events from primary and fallback STTs.

Changes Made

  • agents/src/stt/fallback_adapter.ts — new FallbackAdapter + FallbackSpeechStream. Hot-standby recovery probes, mid stream failover, telephony-tuned defaults (attemptTimeoutMs: 10_000, maxRetryPerSTT: 1, retryIntervalMs: 5_000). Child error events deliberately not forwarded — would otherwise cause AgentSession.onModelError to close the session on the first transient provider error and defeat the fallback. Terminal errors still surface via the adapter's own run() throwing APIConnectionError after exhausting all providers.
  • agents/src/stt/fallback_adapter.test.ts — 14 unit tests covering constructor contract, _recognize fallthrough/exhaustion non-APIError handling, stt_availability_changed wiring, background recovery, metrics forwarding, close() detach, and the full streaming path (success / failover / all-fail).
  • agents/src/stt/index.ts — re-export FallbackAdapter, FallbackAdapterOptions, AvailabilityChangedEvent.

Pre-Review Checklist

  • Build passes: tsc + tsup + ESLint clean on changed files.
  • AI-generated code reviewed
  • Changes explained: see "Changes Made" above and inline comments.
  • Scope appropriate: single feature, no unrelated changes.
  • Video demo: n/a (library addition, not an example / playground change).

Testing

  • Automated tests added/updated
  • All tests pass
  • N/A: Make sure both restaurant_agent.ts and realtime_agent.ts work properly (for major changes)

Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 20, 2026

🦋 Changeset detected

Latest commit: 2707c1d

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 26 packages
Name Type
@livekit/agents Major
@livekit/agents-plugin-anam Major
@livekit/agents-plugin-assemblyai Major
@livekit/agents-plugin-baseten Major
@livekit/agents-plugin-bey Major
@livekit/agents-plugin-cartesia Major
@livekit/agents-plugin-cerebras Major
@livekit/agents-plugin-deepgram Major
@livekit/agents-plugin-elevenlabs Major
@livekit/agents-plugin-google Major
@livekit/agents-plugin-hedra Major
@livekit/agents-plugin-inworld Major
@livekit/agents-plugin-lemonslice Major
@livekit/agents-plugin-livekit Major
@livekit/agents-plugin-mistral Major
@livekit/agents-plugin-neuphonic Major
@livekit/agents-plugin-openai Major
@livekit/agents-plugin-phonic Major
@livekit/agents-plugin-resemble Major
@livekit/agents-plugin-rime Major
@livekit/agents-plugin-runway Major
@livekit/agents-plugin-sarvam Major
@livekit/agents-plugin-silero Major
@livekit/agents-plugins-test Major
@livekit/agents-plugin-trugen Major
@livekit/agents-plugin-xai Major

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@drain-zine drain-zine marked this pull request as ready for review April 20, 2026 11:55
@drain-zine drain-zine changed the title Feat/stt fallback adapter feat(stt): STT Fallback Adapter Apr 20, 2026
@drain-zine drain-zine changed the title feat(stt): STT Fallback Adapter feat(stt): add FallbackAdapter for automatic STT provider failover Apr 20, 2026
devin-ai-integration[bot]

This comment was marked as resolved.

@theomonnom theomonnom merged commit 90f5601 into livekit:main Apr 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants