Skip to content

feat(mistral): add full node.js support for mistralai plugin#1326

Merged
toubatbrian merged 16 commits intolivekit:mainfrom
jeanprbt:jean/feat/add-mistral-node-sdk
May 4, 2026
Merged

feat(mistral): add full node.js support for mistralai plugin#1326
toubatbrian merged 16 commits intolivekit:mainfrom
jeanprbt:jean/feat/add-mistral-node-sdk

Conversation

@jeanprbt
Copy link
Copy Markdown
Contributor

@jeanprbt jeanprbt commented Apr 27, 2026

Description

Rewrite the Mistral AI plugin (plugins/mistral -> plugins/mistralai) with full Node.js support, mirroring the Python Mistral plugin: LLM, STT (batch + realtime), and TTS.

  • The LLM now uses the Conversations API (startStream only), matching the Python plugin
  • Add Mistral-specific provider format (agents/src/llm/provider_format/mistralai.ts) for converting ChatContext to Conversations API entries

Changes Made

Core framework (agents/)

  • Add mistralai provider format with toChatCtx() mapping system/developer messages to instructions, user messages to message.input, assistant messages to message.output, function calls to function.call, and tool outputs to function.result
  • Register 'mistralai' in the ProviderFormat union and dispatch

Plugin (plugins/mistralai/)

  • LLM: Conversations API, retryable error handling matching the Python plugin pattern
  • STT: Batch transcription via audio.transcriptions.complete and realtime streaming via RealtimeTranscription WebSocket with client-side VAD for endpointing
  • TTS: Streaming PCM synthesis via audio.speech.complete with f32le-to-s16le conversion, voice presets and reference audio (voice cloning) support
  • Models: Full model type definitions for chat, STT, TTS, and voice presets

Pre-Review Checklist

  • Build passes: All builds (lint, typecheck, tests) pass locally
  • AI-generated code reviewed: Removed unnecessary comments and ensured code quality
  • Changes explained: All changes are properly documented and justified above
  • Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included

Testing

  • Automated tests added/updated (if applicable)
  • All tests pass

@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented Apr 27, 2026

🦋 Changeset detected

Latest commit: 5adcc87

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 29 packages
Name Type
@livekit/agents-plugin-mistralai Patch
@livekit/agents-plugin-mistral Patch
@livekit/agents Patch
@livekit/agents-plugin-anam Patch
@livekit/agents-plugin-assemblyai Patch
@livekit/agents-plugin-baseten Patch
@livekit/agents-plugin-bey Patch
@livekit/agents-plugin-cartesia Patch
@livekit/agents-plugin-cerebras Patch
@livekit/agents-plugin-deepgram Patch
@livekit/agents-plugin-elevenlabs Patch
@livekit/agents-plugin-google Patch
@livekit/agents-plugin-hedra Patch
@livekit/agents-plugin-inworld Patch
@livekit/agents-plugin-lemonslice Patch
@livekit/agents-plugin-liveavatar Patch
@livekit/agents-plugin-livekit Patch
@livekit/agents-plugin-minimax Patch
@livekit/agents-plugin-neuphonic Patch
@livekit/agents-plugin-openai Patch
@livekit/agents-plugin-phonic Patch
@livekit/agents-plugin-resemble Patch
@livekit/agents-plugin-rime Patch
@livekit/agents-plugin-runway Patch
@livekit/agents-plugin-sarvam Patch
@livekit/agents-plugin-silero Patch
@livekit/agents-plugin-trugen Patch
@livekit/agents-plugin-xai Patch
@livekit/agents-plugins-test Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

@davidzhao davidzhao requested a review from a team April 27, 2026 15:08
Copy link
Copy Markdown
Member

@tinalenguyen tinalenguyen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for the PR! a few notes:

  • could you add a patch changeset?
  • it seems that this would require a package rename mistral -> mistralai on npm, @toubatbrian would this be possible?
  • we have not added support for provider tools yet in JS, perhaps we can port that over before merging this PR

@jeanprbt jeanprbt force-pushed the jean/feat/add-mistral-node-sdk branch from 2d97c19 to aab2ef1 Compare April 28, 2026 07:12
@jeanprbt
Copy link
Copy Markdown
Contributor Author

could you add a patch changeset?

I added a patch changeset, including package renaming & STT/LLM/TTS support from Mistral.

we have not added support for provider tools yet in JS, perhaps we can port that over before merging this PR

That would be great! However if you prefer I can remove the provider tools support from this PR now so as to make merging easier, and I'll add it later on when you will port that over.

@tinalenguyen
Copy link
Copy Markdown
Member

@jeanprbt thanks for iterating! it seems that some files are still renamed to mistralai and in the changeset, could you resolve those and the merge conflicts?

and yes, i think we should remove the provider tools file for now. i'd be happy to add them back once we support them so you wouldn't have to do anything on your end! :)

Comment thread plugins/mistralai/src/llm.ts Outdated
@jeanprbt jeanprbt force-pushed the jean/feat/add-mistral-node-sdk branch from aab2ef1 to 2a5884a Compare April 30, 2026 06:05
@jeanprbt
Copy link
Copy Markdown
Contributor Author

it seems that some files are still renamed to mistralai and in the changeset, could you resolve those and the merge conflicts?

I did not understand you wanted to keep mistral as the package name, I switched all files to mistralai since I'd like to match python plugin naming. Is this an issue?

and yes, i think we should remove the provider tools file for now.

I removed them, you're right! Thanks 🙌

Comment thread plugins/mistral/src/models.ts
Comment thread plugins/mistral/src/llm.ts
Comment thread pnpm-lock.yaml
Comment thread plugins/mistralai/package.json
devin-ai-integration[bot]

This comment was marked as resolved.

@jeanprbt
Copy link
Copy Markdown
Contributor Author

I maintained backward compatibility by adding back former mistral package, with a single index.ts file which wraps the new package's models.ts & llm.ts files. I also added a deprecation warning & change former README to account for deprecation. 🙏

devin-ai-integration[bot]

This comment was marked as resolved.

Comment thread plugins/mistralai/src/index.ts Outdated
Copy link
Copy Markdown
Contributor

@toubatbrian toubatbrian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you also fix the linting in CI; otherwise LGTM!

devin-ai-integration[bot]

This comment was marked as resolved.

Co-authored-by: devin-ai-integration[bot] <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@jeanprbt
Copy link
Copy Markdown
Contributor Author

jeanprbt commented May 4, 2026

Linting & package names are fixed 🙌

@toubatbrian toubatbrian merged commit f610193 into livekit:main May 4, 2026
6 checks passed
@github-actions github-actions Bot mentioned this pull request May 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants