feat(inference): introduce XAIModels type and enhance LLMModels with reasoning support

[russellmartin-livekit]

Description

Add XAI (Grok) model support to the LiveKit inference layer. This introduces the XAIModels type, strips unsupported parameters for XAI reasoning models, generalizes the provider-specific extra_content handling (previously Google-only) to support multiple providers (Google, LiveKit, XAI), and propagates LLM-generated extra metadata through to chat context messages.

Changes Made

• agents/src/inference/llm.ts: Added XAIModels type with 5 Grok model variants. Added dropUnsupportedParams function that strips parameters incompatible with reasoning models (OpenAI o-series, GPT-5, XAI Grok reasoning). XAI reasoning models only need presence_penalty, frequency_penalty, and stop stripped (they still support temperature/top_p). Also strips reasoning_effort from GPT-5.2/5.4 when tools are present. Applied dropUnsupportedParams to the request options in LLMStream.run().
• agents/src/inference/index.ts: Exported the new XAIModels type.
• agents/src/llm/provider_format/openai.ts: Replaced the Google-specific getGoogleExtra helper with a generalized filterExtra function using an allowlist of provider keys (google, livekit, xai). Applied filterExtra to tool calls, function call items, and message items when serializing to OpenAI provider format.
• agents/src/voice/generation.ts: Added generatedExtra field to _LLMGenerationData and accumulate chunk.delta.extra from the LLM stream via Object.assign.
• agents/src/voice/agent_activity.ts: Pass generatedExtra as the extra field on assistant ChatMessage creation for both interrupted and non-interrupted playout paths.

Pre-Review Checklist

• Build passes: All builds (lint, typecheck, tests) pass locally
• AI-generated code reviewed: Removed unnecessary comments and ensured code quality
• Changes explained: All changes are properly documented and justified above
• Scope appropriate: All changes relate to the PR title, or explanations provided for why they're included
• Video demo: A small video demo showing changes works as expected and did not break any existing functionality using Agent Playground (if applicable)

Testing

Tested using https://agents-playground.livekit.io/ and locally running agent-gateway

• Automated tests added/updated (if applicable)
• All tests pass
• Make sure both restaurant_agent.ts and realtime_agent.ts work properly (for major changes)

Additional Notes

• The dropUnsupportedParams logic and UNSUPPORTED_PARAMS map are a direct port from the Python agents repo (livekit-agents/livekit/agents/inference/llm.py), maintaining parity across SDKs.
• The Object.assign shallow merge for generatedExtra matches the Python SDK's dict.update() semantics — this is fine given providers send extra metadata once per stream rather than progressively across chunks.
• No new tests were added for dropUnsupportedParams or filterExtra. Consider adding unit tests for prefix matching edge cases (e.g., ensuring grok-4-1-fast-non-reasoning does not match the reasoning entry).

---

Note to reviewers: Please ensure the pre-review checklist is completed before starting your review.

[changeset-bot]

🦋 Changeset detected

Latest commit: 155a164

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 24 packages

Name	Type
@livekit/agents	Patch
@livekit/agents-plugin-anam	Patch
@livekit/agents-plugin-assemblyai	Patch
@livekit/agents-plugin-baseten	Patch
@livekit/agents-plugin-bey	Patch
@livekit/agents-plugin-cartesia	Patch
@livekit/agents-plugin-deepgram	Patch
@livekit/agents-plugin-elevenlabs	Patch
@livekit/agents-plugin-google	Patch
@livekit/agents-plugin-hedra	Patch
@livekit/agents-plugin-inworld	Patch
@livekit/agents-plugin-lemonslice	Patch
@livekit/agents-plugin-livekit	Patch
@livekit/agents-plugin-mistral	Patch
@livekit/agents-plugin-neuphonic	Patch
@livekit/agents-plugin-openai	Patch
@livekit/agents-plugin-phonic	Patch
@livekit/agents-plugin-resemble	Patch
@livekit/agents-plugin-rime	Patch
@livekit/agents-plugin-sarvam	Patch
@livekit/agents-plugin-silero	Patch
@livekit/agents-plugins-test	Patch
@livekit/agents-plugin-trugen	Patch
@livekit/agents-plugin-xai	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[devin-ai-integration]

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 5 additional findings.