feat(tts): integrate AsyncAI TTS engine into livekit #3596
Conversation
|
Thanks for the PR! While testing it, I'm finding a few issues:
should the connection be kept open? if not then you shouldn't use a connection pool, but just using it for each synthesis
please use |
tinalenguyen
left a comment
tinalenguyen
left a comment
There was a problem hiding this comment.
Hey, thank you for submitting this PR! I left a few comments.
I also experienced the same timeout error as @davidzhao. I believe it's related to the context_id; if we implement the multi-context approach, the websocket connection shouldn't close
ashotbagh
force-pushed
the
feature/asyncai-tts
branch
from
f746e27 to
2cd953b
Compare
|
Hey @davidzhao , @tinalenguyen The earlier version of this PR was indeed out of date. Since then, our AsyncAI WebSocket endpoint has changed and now supports multi-context sessions, which required updating the connection handling logic. I’ve updated the implementation accordingly and rebased the branch on the latest main. Happy to make further adjustments if you still see any issues or have any other suggestions. |
|
@ashotbagh Thanks for making the changes! I still see these errors when interrupting the agent: Could it be that each context must be flushed or closed after each synthesis? I also occasionally see empty audio messages like: |
|
@tinalenguyen Thanks for the comments! Regarding the “connection closed unexpectedly” issue: I’ve tried to reproduce it extensively (including interruption scenarios), but haven’t been able to trigger it locally so far. At the moment, the context flush happens on the final chunk, where we explicitly send: I’ve also made a small update to silently ignore empty audio messages (e.g. audio == "" on non-final frames), instead of logging them as unexpected, and applied a few minor fixes to satisfy ruff checks. Could you please let me know if the issue still persists with the latest changes, and, if possible, share a reliable way to reproduce it? I’d be happy to dig deeper once I can observe it on my side. |
|
testing with |
|
Thanks for the report. The freeze after a tool call was caused by a deadlock in the Async TTS streaming path: the receive loop was waiting for input_sent_event, which was only set when at least one sentence token was emitted. In tool-call flows, it’s possible for the tokenizer to emit no tokens before the end packet is sent, which caused the recv task to wait indefinitely. I’ve fixed this by ensuring the receive loop is unblocked even when no tokens are emitted, so the stream always progresses correctly after tool calls. I also cleaned up import ordering to satisfy ruff checks. I’ve re-tested with basic_agent.py, including repeated tool calls, and the agent no longer hangs. Please let me know if you still see any issues or edge cases I should cover. |
|
Hi @davidzhao , @tinalenguyen . |
ashotbagh
force-pushed
the
feature/asyncai-tts
branch
from
fa89709 to
09e5b73
Compare
📝 WalkthroughWalkthroughAdds a new AsyncAI LiveKit plugin package with TTS streaming via WebSocket, packaging metadata, plugin registration, constants, logger, typed TTS models/languages, versioning, README, and a substantial streaming TTS implementation including session/pool management and stream lifecycle. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant TTS
participant Stream as SynthesizeStream
participant Tokenizer
participant AsyncAI_WS as AsyncAI (WebSocket)
participant Emitter as AudioEmitter
Client->>TTS: stream()
TTS->>Stream: create(tts, conn_options)
Stream->>AsyncAI_WS: _connect_ws(init payload)
AsyncAI_WS-->>Stream: connected
Stream->>Tokenizer: tokenize(input text)
Tokenizer-->>Stream: sentences
par Streaming
Stream->>AsyncAI_WS: send(sentences)
AsyncAI_WS-->>Stream: audio segments / events
Stream->>Emitter: push(audio data / final signals)
end
Emitter-->>Client: audio stream
Client->>Stream: aclose()/stop
Stream->>AsyncAI_WS: _close_ws()
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py`:
- Around line 1-14: The file
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py is not
formatted to ruff standards; run the formatter (e.g., `ruff format` or your
project's `uv run ruff format`) on that file and commit the resulting reformat
so the repo passes `ruff format --check`; no code changes required beyond
applying ruff formatting to tts.py.
- Around line 72-103: The docstring for Async TTS __init__ incorrectly states
language defaults to "en" while the method signature sets language: str | None =
None; update one of them so they match — either change the signature default to
language: str = "en" in __init__ if the implementation expects a default
language, or update the docstring to state that language defaults to None (and
describe behavior when None), ensuring references to the parameter name
"language" and the constructor "__init__" are consistent.
- Around line 210-213: The synthesize method in the AsyncAI TTS implementation
currently is a stub that returns None and therefore violates the abstract TTS
base class contract which requires returning a ChunkedStream; replace the pass
with an explicit runtime error (e.g., raise NotImplementedError or a custom
RuntimeError) that states AsyncAI TTS only supports streaming and direct
synthesize() is unsupported, and update the method signature to include the
correct return type annotation (ChunkedStream) to match the base class; locate
and modify the synthesize(self, text: str, *, conn_options: APIConnectOptions =
DEFAULT_API_CONNECT_OPTIONS) method in this class to implement those changes.
- Around line 144-163: The WebSocket URL is built by raw f-string interpolation
in _connect_ws using API_AUTH_HEADER, API_VERSION_HEADER and self._opts.api_key
which can break if the API key contains reserved URL characters; update
_connect_ws to build the query string with urllib.parse.urlencode (encode
API_AUTH_HEADER=self._opts.api_key and API_VERSION_HEADER=API_VERSION) and pass
that encoded query to self._opts.get_ws_url instead of interpolating directly so
the URL is safely encoded.
In `@livekit-plugins/livekit-plugins-asyncai/pyproject.toml`:
- Around line 14-25: The pyproject declarations are missing the aiohttp runtime
dependency used by tts.py; update the dependencies list (the dependencies =
[...] entry) to include "aiohttp" alongside "livekit-agents>=1.2.14", and
optionally extend the classifiers array (the classifiers = [...] entry) to add
"Programming Language :: Python :: 3.11" and "Programming Language :: Python ::
3.12" if you have verified compatibility and tests pass for those versions.
In `@livekit-plugins/livekit-plugins-asyncai/README.md`:
- Around line 3-5: Replace the un-hyphenated phrase "text to speech" with the
hyphenated "text-to-speech" in the line that reads "Support for text to speech
with [Async](https://async.ai/)." and convert the bare URL
"https://docs.livekit.io/agents/integrations/tts/asyncai/" into a proper
Markdown link with descriptive text (e.g., "See the LiveKit Async TTS docs" or
"See the Async TTS docs") so the README uses "text-to-speech" and avoids a bare
URL.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
livekit-plugins/livekit-plugins-asyncai/README.mdlivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/__init__.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/constants.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/log.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/models.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/version.pylivekit-plugins/livekit-plugins-asyncai/pyproject.toml
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/constants.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/log.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/models.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/version.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.pylivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/__init__.py
🧬 Code graph analysis (1)
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/__init__.py (1)
livekit-agents/livekit/agents/plugin.py (2)
Plugin(13-56)register_plugin(31-36)
🪛 GitHub Actions: CI
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
[error] 1-1: Command 'uv run ruff format --check .' failed with exit code 1. Ruff format would reformat 1 file: livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py. Run 'ruff format' to fix.
🪛 LanguageTool
livekit-plugins/livekit-plugins-asyncai/README.md
[grammar] ~3-~3: Use a hyphen to join words.
Context: ...gin for LiveKit Agents Support for text to speech with [Async](https://async.ai/...
(QB_NEW_EN_HYPHEN)
[grammar] ~3-~3: Use a hyphen to join words.
Context: ... for LiveKit Agents Support for text to speech with Async. ...
(QB_NEW_EN_HYPHEN)
🪛 markdownlint-cli2 (0.18.1)
livekit-plugins/livekit-plugins-asyncai/README.md
5-5: Bare URL used
(MD034, no-bare-urls)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: unit-tests
🔇 Additional comments (13)
livekit-plugins/livekit-plugins-asyncai/pyproject.toml (3)
1-3: LGTM!Build system configuration using hatchling is appropriate and consistent with other livekit plugin packages.
27-30: LGTM!Project URLs are correctly configured and consistent with other LiveKit plugins.
32-39: LGTM!Hatch configuration correctly follows the namespace package pattern used by other livekit plugins, and the version file is properly located at the specified path.
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/log.py (1)
1-3: LGTM — consistent module logger setup.livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/version.py (1)
1-15: Version module looks good.livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/constants.py (1)
1-3: LGTM — clear, centralized constants.livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/__init__.py (1)
30-35: Verify import-time registration occurs on the main thread.
Plugin.register_pluginraises if called off the main thread. Since registration happens at import time, ensure import/discovery always runs on the main thread; otherwise consider deferring registration to an explicit init hook.livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/models.py (1)
1-7: LGTM — clear typed config surface.livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py (5)
54-68: Centralized option/URL helpers look good.
Keeps configuration and URL construction localized and easy to extend.
178-208: Option updates and stream factory look solid.
215-221: Graceful stream/pool cleanup looks good.
223-228: Stream setup and option snapshotting look good.
229-333: Streaming loop and cleanup sequencing are solid.
The empty-audio guard and end-of-input handling should help avoid stalls.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py`:
- Around line 55-62: The _TTSOptions dataclass currently types the language
field as str but the TTS class __init__ accepts language: str | None and passes
that through, causing a mypy type error; update the _TTSOptions declaration for
the language attribute to language: str | None so it matches the constructor and
the downstream None checks in methods like the TTS implementation that inspect
language for None.
♻️ Duplicate comments (3)
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py (3)
91-94: Align language docstring defaults with the signature.The docstrings state a default of
"en", but the signature defaults toNone. Please align them to avoid confusion.Also applies to: 192-194
17-25: URL-encode WebSocket query params.Raw interpolation can break if the API key contains reserved characters. Use
urllib.parse.urlencode()for the query string.🔧 Proposed fix
+from urllib.parse import urlencode @@ - url = self._opts.get_ws_url( - f"/text_to_speech/websocket/ws?{API_AUTH_HEADER}={self._opts.api_key}&{API_VERSION_HEADER}={API_VERSION}" - ) + query = urlencode({API_AUTH_HEADER: self._opts.api_key, API_VERSION_HEADER: API_VERSION}) + url = self._opts.get_ws_url(f"/text_to_speech/websocket/ws?{query}")Also applies to: 144-148
210-213:synthesize()should fail fast instead of returningNone.This stub violates the base class contract. Raise an explicit error (and optionally annotate the return type) to make the unsupported path clear.
🔧 Proposed fix
- def synthesize( - self, text: str, *, conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS - ): - pass + def synthesize( + self, text: str, *, conn_options: APIConnectOptions = DEFAULT_API_CONNECT_OPTIONS + ) -> tts.ChunkedStream: + raise NotImplementedError( + "AsyncAI TTS currently supports streaming only; use stream()." + )
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (1)
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Applies to **/*.py : Run ruff linter and auto-fix issues
Applied to files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Applies to **/*.py : Format code with ruff
Applied to files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
🧬 Code graph analysis (1)
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py (6)
livekit-agents/livekit/agents/_exceptions.py (3)
APIConnectionError(84-88)APIStatusError(45-81)APITimeoutError(91-95)livekit-agents/livekit/agents/types.py (1)
APIConnectOptions(54-88)livekit-agents/livekit/agents/utils/misc.py (2)
is_given(25-26)shortuuid(21-22)livekit-agents/livekit/agents/utils/http_context.py (1)
http_session(40-51)livekit-agents/livekit/agents/utils/connection_pool.py (1)
connection(73-86)livekit-plugins/livekit-plugins-inworld/livekit/plugins/inworld/tts.py (1)
mime_type(81-91)
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Fix all issues with AI agents
In `@livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py`:
- Around line 292-297: The code currently passes possibly-None segment_id into
output_emitter.start_segment (via data.get("context_id")), which can break
downstream logic; modify the handling in the block around current_segment_id,
segment_id and output_emitter.start_segment so that if data.get("context_id") is
None you fall back to an existing local context (e.g., current_segment_id or a
designated default_context_id), only call output_emitter.start_segment when the
resolved segment_id is non-None, and ensure current_segment_id is set to that
resolved id; update the logic in the function/method containing
current_segment_id, segment_id and output_emitter.start_segment accordingly.
- Around line 185-195: The docstring for update_options incorrectly states fixed
default values for model and language; clarify that model and language are
optional and that omitting them will retain the current instance values (i.e.,
"keep existing if omitted") rather than defaulting to
"asyncflow_multilingual_v1.0" or "en". Update the docstring on the
update_options method to reflect that behavior, remove the misleading Defaults:
entries, and mention that voice, model, and language are optional and will not
change if not provided; reference the update_options function and the TTSModels
type in the description so readers can locate the method and expected types.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
uv.lockis excluded by!**/*.lock
📒 Files selected for processing (4)
livekit-plugins/livekit-plugins-asyncai/README.mdlivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/py.typedlivekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.pylivekit-plugins/livekit-plugins-asyncai/pyproject.toml
🚧 Files skipped from review as they are similar to previous changes (1)
- livekit-plugins/livekit-plugins-asyncai/README.md
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
🧠 Learnings (3)
📓 Common learnings
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Applies to **/*.py : Run ruff linter and auto-fix issues
Applied to files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Applies to **/*.py : Format code with ruff
Applied to files:
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: type-check (3.9)
- GitHub Check: type-check (3.13)
- GitHub Check: unit-tests
🔇 Additional comments (11)
livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py (10)
55-69: LGTM — options container and URL helpers are straightforward.
73-135: LGTM — constructor wiring and initialization look solid.
137-143: LGTM — lightweight properties are clear and consistent.
145-164: LGTM — connection setup is clean and easy to follow.
166-176: LGTM — close/ensure/prewarm helpers are minimal and clear.
203-220: LGTM — stream API, explicit unsupported synthesize, and cleanup are good.
224-228: LGTM — stream init is concise and correct.
229-238: LGTM — emitter initialization is well structured.
239-272: LGTM — input/tokenization flow and end-packet handling are solid.
314-336: LGTM — task orchestration, cleanup, and error mapping look robust.livekit-plugins/livekit-plugins-asyncai/pyproject.toml (1)
1-42: LGTM — packaging metadata and build configuration are consistent.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| """ | ||
| Update the Text-to-Speech (TTS) configuration options. | ||
|
|
||
| This method allows updating the TTS settings, including model type, language and voice. | ||
| If any parameter is not provided, the existing value will be retained. | ||
|
|
||
| Args: | ||
| model (TTSModels, optional): The Async TTS model to use. Defaults to "asyncflow_multilingual_v1.0". | ||
| language (str, optional): The language code for synthesis. Defaults to "en". | ||
| voice (str, optional): The voice ID. | ||
| """ |
There was a problem hiding this comment.
Docstring defaults in update_options() don’t match behavior.
model and language are “keep existing if omitted,” not defaulting to fixed values. Align the docstring to avoid confusion.
📘 Suggested docstring alignment
- model (TTSModels, optional): The Async TTS model to use. Defaults to "asyncflow_multilingual_v1.0".
- language (str, optional): The language code for synthesis. Defaults to "en".
+ model (TTSModels, optional): The Async TTS model to use. If omitted, the current value is kept.
+ language (str, optional): The language code for synthesis. If omitted, the current value is kept.📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| """ | |
| Update the Text-to-Speech (TTS) configuration options. | |
| This method allows updating the TTS settings, including model type, language and voice. | |
| If any parameter is not provided, the existing value will be retained. | |
| Args: | |
| model (TTSModels, optional): The Async TTS model to use. Defaults to "asyncflow_multilingual_v1.0". | |
| language (str, optional): The language code for synthesis. Defaults to "en". | |
| voice (str, optional): The voice ID. | |
| """ | |
| """ | |
| Update the Text-to-Speech (TTS) configuration options. | |
| This method allows updating the TTS settings, including model type, language and voice. | |
| If any parameter is not provided, the existing value will be retained. | |
| Args: | |
| model (TTSModels, optional): The Async TTS model to use. If omitted, the current value is kept. | |
| language (str, optional): The language code for synthesis. If omitted, the current value is kept. | |
| voice (str, optional): The voice ID. | |
| """ |
🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py`
around lines 185 - 195, The docstring for update_options incorrectly states
fixed default values for model and language; clarify that model and language are
optional and that omitting them will retain the current instance values (i.e.,
"keep existing if omitted") rather than defaulting to
"asyncflow_multilingual_v1.0" or "en". Update the docstring on the
update_options method to reflect that behavior, remove the misleading Defaults:
entries, and mention that voice, model, and language are optional and will not
change if not provided; reference the update_options function and the TTSModels
type in the description so readers can locate the method and expected types.
| data = json.loads(msg.data) | ||
| segment_id = data.get("context_id") | ||
| if current_segment_id is None: | ||
| current_segment_id = segment_id | ||
| output_emitter.start_segment(segment_id=segment_id) | ||
| final = data.get("final") |
There was a problem hiding this comment.
Guard against missing context_id in responses.
If AsyncAI ever omits context_id, output_emitter.start_segment() receives None, which can break downstream assumptions. Consider falling back to the local context id.
🛠️ Suggested fallback
- segment_id = data.get("context_id")
+ segment_id = data.get("context_id") or asyncai_context_id🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-asyncai/livekit/plugins/asyncai/tts.py`
around lines 292 - 297, The code currently passes possibly-None segment_id into
output_emitter.start_segment (via data.get("context_id")), which can break
downstream logic; modify the handling in the block around current_segment_id,
segment_id and output_emitter.start_segment so that if data.get("context_id") is
None you fall back to an existing local context (e.g., current_segment_id or a
designated default_context_id), only call output_emitter.start_segment when the
resolved segment_id is non-None, and ensure current_segment_id is set to that
resolved id; update the logic in the function/method containing
current_segment_id, segment_id and output_emitter.start_segment accordingly.
|
@davidzhao All checks should be passing now. Please let me know if there’s anything else you’d like me to adjust - happy to iterate. Thanks! |
3 participants
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.