Feature/krisp viva sdk support#4370
Conversation
|
|
Add proper integration point for Krisp noise cancellation in LiveKit Agents using AudioInput wrapper pattern for human-to-bot audio filtering.
📝 WalkthroughWalkthroughThis pull request introduces the Krisp VIVA plugin for LiveKit Agents, adding noise reduction and audio-based turn detection capabilities. The submission includes comprehensive documentation, core implementation files, example usage scripts, and integration tests for the new plugin. Changes
Sequence Diagram(s)sequenceDiagram
participant Agent as LiveKit Agent
participant FP as KrispVivaFilterFrameProcessor
participant SDK as KrispSDKManager
participant KrispSDK as Krisp VIVA SDK
participant RTC as LiveKit RTC
Agent->>FP: __init__(model_path, noise_level)
FP->>SDK: acquire()
SDK->>SDK: increment reference_count
SDK->>KrispSDK: Initialize SDK
FP->>KrispSDK: Load model & create session
loop For each audio frame
RTC->>FP: process(AudioFrame)
FP->>FP: validate session alignment
FP->>KrispSDK: process_frame(audio_data)
KrispSDK-->>FP: filtered_audio
FP->>RTC: return AudioFrame(filtered_audio)
end
Agent->>FP: close()
FP->>SDK: release()
SDK->>SDK: decrement reference_count
SDK->>KrispSDK: Destroy SDK (if count == 0)
sequenceDiagram
participant Agent as LiveKit Agent
participant TD as KrispVivaTurn
participant SDK as KrispSDKManager
participant KrispSDK as Krisp VIVA SDK
participant RTC as LiveKit RTC
Agent->>TD: __init__(model_path, threshold)
TD->>SDK: acquire()
SDK->>KrispSDK: Initialize SDK
TD->>KrispSDK: Load model & create turn session
loop For each audio frame with VAD
RTC->>TD: process_audio(AudioFrame, is_speech)
TD->>TD: buffer frame data
TD->>TD: convert to float32
TD->>KrispSDK: predict(audio_buffer)
KrispSDK-->>TD: probability
TD->>TD: update state & frame_probabilities
TD-->>RTC: return probability
end
alt Probability > threshold
TD->>Agent: signal turn completion
end
Agent->>TD: close()
TD->>SDK: release()
SDK->>KrispSDK: Destroy SDK (if count == 0)
Estimated code review effort🎯 4 (Complex) | ⏱️ ~70 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 3✅ Passed checks (3 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🤖 Fix all issues with AI agents
In
`@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py`:
- Around line 101-152: Guard the acquire/release methods against krisp_audio not
being installed by checking the KRISP_AUDIO_AVAILABLE flag: in acquire(cls) at
the start of the locked section, if not KRISP_AUDIO_AVAILABLE raise a
RuntimeError with a helpful message referencing krisp_audio is unavailable (do
not let NameError propagate); in release(cls) make it a safe no-op when
KRISP_AUDIO_AVAILABLE is False (still honor the lock but skip reference count
manipulation and cleanup) so cleanup sequences don't break; reference the
existing class symbols acquire, release, KRISP_AUDIO_AVAILABLE, krisp_audio,
cls._reference_count, cls._initialized, and cls._lock when making the changes.
- Around line 61-79: The int_to_krisp_frame_duration and
int_to_krisp_sample_rate helpers should first check krisp availability and raise
a clear error if missing (e.g., guard on KRISP_AVAILABLE or the krisp_audio
import) before computing supported lists, then include Google-style docstrings
describing parameters, return type, and raised exceptions; update
int_to_krisp_frame_duration and int_to_krisp_sample_rate to fail fast with a
descriptive ImportError (or RuntimeError) when KRISP_AVAILABLE is false and add
concise docstrings for each function.
In
`@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_turn_audiofile.py`:
- Around line 91-92: Replace the Python 3.10 union type syntax in the function
signature for the parameter named output_file (currently declared as
"output_file: str | None = None") with typing.Optional to maintain Python 3.9
compatibility: import Optional from the typing module at the top of the file and
change the annotation to "Optional[str]". Ensure the rest of the signature
(return type -> None) remains unchanged and that the new import is added
alongside other imports.
In `@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py`:
- Around line 15-24: Ruff formatting failed for viva_filter.py; run the
formatter and fix style issues by running `ruff format` (or `python -m ruff
format`) on livekit/plugins/krisp/viva_filter.py, then address any remaining
ruff/flake8 complaints (import ordering around "from __future__ import
annotations", unused imports like "os" or "Any", whitespace, and trailing
newlines) so the file passes `ruff format --check` and CI.
- Around line 223-225: The _process method currently raises ValueError on a
sample-rate mismatch; instead detect the mismatch (self._session is None or
self._sample_rate != frame.sample_rate), call the session-creation routine to
recreate the session for the new sample rate (e.g., invoke the existing session
init helper such as _create_session or the same logic used in __init__), update
self._sample_rate to frame.sample_rate, and then continue processing; reference
symbols: method _process, attributes _session and _sample_rate, and the
frame.sample_rate property.
In `@livekit-plugins/livekit-plugins-krisp/pyproject.toml`:
- Around line 36-40: Update the livekit-agents dependency in pyproject.toml from
"livekit-agents>=1.3.6" to "livekit-agents>=1.3.10" to match the minimum used by
other plugins; leave the "livekit>=1.0.23,<2" and "numpy>=1.20.0" entries
unchanged. Use the dependency string "livekit-agents>=1.3.10" so tools will pick
up the consistent minimum across plugins.
In `@livekit-plugins/livekit-plugins-krisp/README.md`:
- Around line 143-154: Update the README usage examples to call the actual test
script name test_viva_filter_audiofile.py instead of test_audio_filtering.py:
replace all occurrences in the three examples (basic test, with visualization,
custom parameters) to use python test_viva_filter_audiofile.py input.wav
output.wav, and either remove the --visualize option from the examples or
implement the --visualize flag in test_viva_filter_audiofile.py to match the
documentation.
🧹 Nitpick comments (14)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_with_logging.py (1)
6-6: Potential import path issue with relative import.The import
from test_audio_filtering import mainassumes the module is in the same directory or onsys.path. This may fail when running via pytest from the repository root or when the test directory is not the current working directory.Consider using explicit relative imports if this is intended as a package:
from .test_audio_filtering import mainOr, if this script is only meant to be run directly from its directory, consider adding a note in the docstring clarifying this limitation.
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/version.py (1)
1-1: Consider updating copyright year.The copyright year is
2023, but this is new code being added in late 2025. Consider updating to2025or2023-2025for accuracy.livekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.py (1)
28-29: Consider removing unnecessaryasyncdecorator.The
test_krisp_filterfunction is declared asasyncbut contains noawaitstatements. Sincekrisp_processor.process()is a synchronous method, theasynckeyword is unnecessary here.This is minor for an example script, but making it a regular function would more accurately reflect the actual API usage.
♻️ Suggested simplification
-async def test_krisp_filter(): +def test_krisp_filter(): """Test Krisp filter with synthetic audio."""And update the
main()function:-async def main(): +def main(): """Run all tests.""" # Test frame processor - await test_krisp_filter() + test_krisp_filter() if __name__ == "__main__": - asyncio.run(main()) + main()livekit-plugins/livekit-plugins-krisp/pyproject.toml (1)
24-35: Consider adding Python 3.13 classifier if supported.The classifiers list Python 3.9 through 3.12 but omit 3.13. If Python 3.13 is supported (as indicated by the
requires-python = ">=3.9.0"without upper bound), consider adding it for completeness:"Programming Language :: Python :: 3.13",This is optional and depends on whether the
krisp-audioSDK supports Python 3.13.livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/__init__.py (1)
1-1: Minor: Copyright year may need updating.The copyright year is 2023, but this is new code being added in late 2025. Consider updating to 2025 or the appropriate year range.
livekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.py (1)
14-14: Consider removing PR reference from documentation.The prerequisite mentions "livekit-agents (with PR
#4145support for FrameProcessor)" which will become outdated once the PR is merged. Consider referencing the version number or capability instead.📝 Suggested change
- - livekit-agents (with PR `#4145` support for FrameProcessor) + - livekit-agents>=1.0.0 # or appropriate version with FrameProcessor supportlivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_turn_audiofile.py (1)
273-275: Consider specifying encoding when writing files.For cross-platform consistency, explicitly specify
encoding="utf-8"when opening the file for writing.📝 Suggested change
- with open(output_file, "w") as f: + with open(output_file, "w", encoding="utf-8") as f: for prob in all_probabilities: f.write(f"{prob}\n")livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/audio_file_utils.py (2)
49-55: Float audio values outside [-1.0, 1.0] will clip.The conversion
audio_data * 32767assumes float values are normalized to [-1.0, 1.0]. Some audio files may have values slightly outside this range (common with certain normalizations), which would overflow int16 bounds. Consider addingnp.clipfor robustness.🛡️ Proposed fix to handle edge cases
elif info.subtype in ["FLOAT", "DOUBLE"]: # File is float format, read as float32 and scale to int16 audio_data, sample_rate = sf.read(input_path, dtype="float32") # Convert float32 (-1.0 to 1.0) to int16 (-32768 to 32767) - audio_data = (audio_data * 32767).astype(np.int16) + audio_data = np.clip(audio_data * 32767, -32768, 32767).astype(np.int16) if verbose: print("Read as float32 and scaled to int16")
107-115: Consider usingos.path.splitextfor extension extraction.The current string manipulation for extracting the extension is unconventional. Using
os.path.splitextis more idiomatic and handles edge cases better.📝 Suggested change
+import os.path + def write_audio_file( output_path: str, audio_data: np.ndarray, sample_rate: int, verbose: bool = False ) -> None: ... # Validate output file extension valid_extensions = [".wav", ".flac", ".ogg"] - output_ext = output_path[output_path.rfind(".") :].lower() if "." in output_path else "" + _, output_ext = os.path.splitext(output_path) + output_ext = output_ext.lower() if output_ext not in valid_extensions:livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_filter_audiofile.py (3)
94-101: Async function without anyawaitcalls.The
process_audio_filefunction is defined asasyncbut doesn't contain anyawaitexpressions. Sinceaudio_filter.process()is synchronous (as noted in the comment on line 190), this function could be a regular synchronous function, simplifying the call site.📝 Suggested change
-async def process_audio_file( +def process_audio_file( input_path: str, output_path: str, noise_level: int = 100, frame_duration_ms: int = 20, chunk_duration_ms: int = 20, verbose: bool = False, ) -> None:And update the call site:
- asyncio.run( - process_audio_file( - args.input, - ... - ) - ) + process_audio_file( + args.input, + ... + )
202-209: Adding unfiltered samples may cause audio artifacts.Appending incomplete frames unfiltered could create audible artifacts at the end of the audio. Consider either:
- Padding the incomplete frame to process it through the filter
- Discarding the incomplete samples (with a warning)
- Documenting this behavior clearly
The current approach silently mixes filtered and unfiltered audio.
240-240: Potential division by zero in real-time factor calculation.If
process_durationis extremely small (e.g., near zero for very short audio), this could cause a division by zero or very large values. Consider adding a guard.🛡️ Suggested change
- print(f" - Real-time factor: {(len(audio_data) / sample_rate) / process_duration:.2f}x") + if process_duration > 0: + print(f" - Real-time factor: {(len(audio_data) / sample_rate) / process_duration:.2f}x") + else: + print(" - Real-time factor: N/A (processing too fast to measure)")livekit-plugins/livekit-plugins-krisp/README.md (1)
118-133: Minor: Table formatting inconsistency.Static analysis flagged table column style issues (MD060). The table pipes have inconsistent spacing. This is a minor style issue that most markdown renderers handle fine.
📝 Consistent table formatting
-| Parameter | Type | Default | Description | -|-----------|------|---------|-------------| -| `model_path` | str | env var | Path to noise reduction `.kef` model | +| Parameter | Type | Default | Description | +|---------------------------|------|---------|-----------------------------------------| +| `model_path` | str | env var | Path to noise reduction `.kef` model |Or simply ignore this as most renderers handle it correctly.
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py (1)
145-147: Use ValueError for invalid model extension.
Raising a bare Exception makes it harder to catch input errors cleanly.🛠️ Suggested change
- if not self._model_path.endswith(".kef"): - raise Exception("Model is expected with .kef extension") + if not self._model_path.endswith(".kef"): + raise ValueError("Model is expected with .kef extension")
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (15)
livekit-plugins/livekit-plugins-krisp/README.mdlivekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.pylivekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/__init__.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/log.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/audio_file_utils.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_filter_audiofile.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_turn_audiofile.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_with_logging.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/version.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.pylivekit-plugins/livekit-plugins-krisp/pyproject.tomlpyproject.toml
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
📄 CodeRabbit inference engine (AGENTS.md)
**/*.py: Format code with ruff
Run ruff linter and auto-fix issues
Run mypy type checker in strict mode
Maintain line length of 100 characters maximum
Ensure Python 3.9+ compatibility
Use Google-style docstrings
Files:
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/version.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/log.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/audio_file_utils.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/__init__.pylivekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.pylivekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_turn_audiofile.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_with_logging.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_filter_audiofile.pylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py
🧠 Learnings (1)
📚 Learning: 2026-01-16T07:44:56.353Z
Learnt from: CR
Repo: livekit/agents PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-01-16T07:44:56.353Z
Learning: Follow the Plugin System pattern where plugins in livekit-plugins/ are separate packages registered via the Plugin base class
Applied to files:
livekit-plugins/livekit-plugins-krisp/pyproject.tomlpyproject.toml
🧬 Code graph analysis (6)
livekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.py (3)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py (1)
model(286-288)livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py (1)
KrispVivaFilterFrameProcessor(58-333)livekit-agents/livekit/agents/voice/room_io/types.py (2)
RoomOptions(111-236)AudioInputOptions(65-79)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py (2)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py (4)
int_to_krisp_frame_duration(61-70)int_to_krisp_sample_rate(73-79)acquire(102-130)release(133-152)livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py (1)
_create_session(174-206)
livekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.py (2)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py (3)
KrispVivaFilterFrameProcessor(58-333)process(266-268)close(300-302)livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py (1)
close(361-372)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_with_logging.py (1)
livekit-agents/livekit/agents/cli/log.py (1)
format(114-146)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py (1)
livekit-agents/livekit/agents/cli/cli.py (1)
LogLevel(1381-1387)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py (1)
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py (5)
KrispSDKManager(82-172)int_to_krisp_frame_duration(61-70)int_to_krisp_sample_rate(73-79)acquire(102-130)release(133-152)
🪛 GitHub Actions: CI
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py
[error] 1-1: ruff format check failed: 1 file would be reformatted. Run 'ruff format' to fix code style issues. Command: uv run ruff format --check .
🪛 LanguageTool
livekit-plugins/livekit-plugins-krisp/README.md
[style] ~242-~242: To form a complete sentence, be sure to include a subject.
Context: ...plementation for Krisp noise reduction. Can be used directly with the `noise_cancel...
(MISSING_IT_THERE)
🪛 markdownlint-cli2 (0.20.0)
livekit-plugins/livekit-plugins-krisp/README.md
119-119: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
119-119: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the right for style "compact"
(MD060, table-column-style)
128-128: Table column style
Table pipe is missing space to the left for style "compact"
(MD060, table-column-style)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: unit-tests
🔇 Additional comments (24)
pyproject.toml (1)
27-27: LGTM!The new workspace source entry for
livekit-plugins-krispfollows the established alphabetical ordering and workspace pattern used by other plugins. Based on learnings, this correctly follows the Plugin System pattern where plugins inlivekit-plugins/are separate packages.livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/version.py (1)
15-15: LGTM!Version declaration follows the standard pattern for hatch-based versioning and integrates correctly with the pyproject.toml configuration.
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/log.py (1)
15-17: LGTM!The logger is correctly namespaced as
"livekit.plugins.krisp", following hierarchical logging conventions. This allows proper log filtering and configuration at the plugin level.livekit-plugins/livekit-plugins-krisp/examples/krisp_minimal_example.py (1)
35-96: LGTM!The example demonstrates good practices:
- Clear step-by-step logging for debugging
- Proper synthetic audio generation with realistic PCM conversion
- Validation of output frame dimensions
- Multi-frame processing test for stability
- Explicit cleanup with
close()This provides a useful minimal test path for verifying Krisp integration.
livekit-plugins/livekit-plugins-krisp/pyproject.toml (1)
42-45: Good documentation of proprietary dependency.The comment clearly explains that
krisp-audiois not on PyPI and must be installed separately. This is helpful for users encountering import errors.livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/__init__.py (1)
25-56: LGTM!The plugin initialization follows the standard LiveKit plugin pattern correctly. The
__all__exports are appropriate, and the plugin registration at import time is consistent with other LiveKit plugins.livekit-plugins/livekit-plugins-krisp/examples/krisp_agent_example.py (2)
81-100: LGTM!The configuration correctly ensures
frame_size_msmatches Krisp'sframe_duration_ms(both 10ms), and sample rates are consistent at 16000Hz. The inline comments clearly document these requirements.
53-57: Not an issue. Thegenerate_replymethod is synchronous (not async) and returnsSpeechHandle, a regular object. No await is needed or should be used here.Likely an incorrect or invalid review comment.
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/audio_file_utils.py (1)
61-69: LGTM!Good handling of stereo-to-mono conversion with overflow protection by using int32 intermediate for int16 data. This prevents potential overflow when averaging channel values.
livekit-plugins/livekit-plugins-krisp/README.md (1)
1-113: LGTM!The documentation is comprehensive and well-structured. It clearly explains:
- Installation requirements including the proprietary SDK note
- Prerequisites for both noise reduction and turn detection
- Quick start examples with proper code samples
- The audio pipeline flow
livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/krisp_instance.py (3)
29-58: Solid krisp_audio fallback and capability mapping.
The guarded import plus explicit warning and empty maps make the absence clear and safe to reason about.
96-99: Logging callback is simple and thread-safe.
155-172: Accessor methods are properly lock-protected.livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py (4)
38-56: Good upfront dependency/feature gate.
The FrameProcessor guard and krisp_audio availability warning provide clear feedback early.
173-202: Session creation and config wiring look solid.
266-333: Lifecycle helpers are thorough and consistent.
91-97: UseOptional[...]instead of PEP 604 union syntax for Python 3.9 compatibility.While the file contains
from __future__ import annotations, the PEP 604 syntax (str | None,int | None) should still be replaced withOptional[...]to maintain explicit Python 3.9 compatibility as required by the coding guidelines. This change applies to function parameters at lines 91–97 and attribute annotations at lines 123–125.🛠️ Proposed type-hint adjustments
-from typing import Any +from typing import Any, Optional @@ - model_path: str | None = None, + model_path: Optional[str] = None, @@ - sample_rate: int | None = None, + sample_rate: Optional[int] = None, @@ - self._session: Any | None = None + self._session: Optional[Any] = None @@ - self._sample_rate: int | None = None + self._sample_rate: Optional[int] = None⛔ Skipped due to learnings
Learnt from: CR Repo: livekit/agents PR: 0 File: AGENTS.md:0-0 Timestamp: 2026-01-16T07:44:56.353Z Learning: Applies to **/*.py : Ensure Python 3.9+ compatibilitylivekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_turn.py (7)
38-47: Clear dependency warning for krisp_audio.
119-173: Initialization flow and SDK acquisition are clean.
174-207: Session creation logic is well-structured.
208-275: Audio processing loop looks robust and buffered correctly.
276-359: State reset and compatibility helpers are consistent.
361-405: Cleanup and shutdown handling are thorough.
50-57: No changes needed. The file already usesfrom __future__ import annotations(line 24), which makes PEP 604 union syntax valid on Python 3.9+. With deferred annotations (PEP 563), type hints are treated as strings and evaluated lazily, allowing modern syntax across Python 3.9+. Converting toOptional/Unionis unnecessary and goes against current Python best practices.Likely an incorrect or invalid review comment.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
| output_file: str | None = None, | ||
| ) -> None: |
There was a problem hiding this comment.
Python 3.9 compatibility issue with union type syntax.
The str | None syntax requires Python 3.10+. For Python 3.9 compatibility (as per coding guidelines), use Optional[str] from the typing module.
🐍 Proposed fix for Python 3.9 compatibility
Add import at the top:
from typing import OptionalThen update the signature:
def analyze_audio_file(
input_path: str,
threshold: float = 0.5,
frame_duration_ms: int = 20,
chunk_duration_ms: int = 20,
verbose: bool = False,
- output_file: str | None = None,
+ output_file: Optional[str] = None,
) -> None:📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| output_file: str | None = None, | |
| ) -> None: | |
| output_file: Optional[str] = None, | |
| ) -> None: |
🤖 Prompt for AI Agents
In
`@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/tests/test_viva_turn_audiofile.py`
around lines 91 - 92, Replace the Python 3.10 union type syntax in the function
signature for the parameter named output_file (currently declared as
"output_file: str | None = None") with typing.Optional to maintain Python 3.9
compatibility: import Optional from the typing module at the top of the file and
change the annotation to "Optional[str]". Ensure the rest of the signature
(return type -> None) remains unchanged and that the new import is added
alongside other imports.
| """Krisp VIVA noise reduction audio filter for LiveKit Agents. | ||
|
|
||
| This module provides an audio filter implementation using Krisp VIVA SDK | ||
| for real-time noise suppression in LiveKit voice agents. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import os | ||
| from typing import Any |
There was a problem hiding this comment.
Ruff format check failed for this file.
CI reports “ruff format --check” would reformat one file. Please run ruff format (and fix any follow‑up ruff issues) before merge.
As per coding guidelines: Format code with ruff and auto-fix issues.
🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py`
around lines 15 - 24, Ruff formatting failed for viva_filter.py; run the
formatter and fix style issues by running `ruff format` (or `python -m ruff
format`) on livekit/plugins/krisp/viva_filter.py, then address any remaining
ruff/flake8 complaints (import ordering around "from __future__ import
annotations", unused imports like "os" or "Any", whitespace, and trailing
newlines) so the file passes `ruff format --check` and CI.
| if self._session is None or self._sample_rate != frame.sample_rate: | ||
| raise ValueError(f"Session not created or sample rate mismatch: {frame.sample_rate}Hz") | ||
|
|
There was a problem hiding this comment.
Recreate the session when input sample rate changes.
_process currently raises on a sample‑rate mismatch even though init logs say the session will be recreated; this will break if the incoming sample rate changes. Consider recreating the session instead of hard‑failing.
🛠️ Suggested change
- if self._session is None or self._sample_rate != frame.sample_rate:
- raise ValueError(f"Session not created or sample rate mismatch: {frame.sample_rate}Hz")
+ if self._session is None or self._sample_rate != frame.sample_rate:
+ self._create_session(frame.sample_rate)🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-krisp/livekit/plugins/krisp/viva_filter.py`
around lines 223 - 225, The _process method currently raises ValueError on a
sample-rate mismatch; instead detect the mismatch (self._session is None or
self._sample_rate != frame.sample_rate), call the session-creation routine to
recreate the session for the new sample rate (e.g., invoke the existing session
init helper such as _create_session or the same logic used in __init__), update
self._sample_rate to frame.sample_rate, and then continue processing; reference
symbols: method _process, attributes _session and _sample_rate, and the
frame.sample_rate property.
| dependencies = [ | ||
| "livekit-agents>=1.3.6", | ||
| "livekit>=1.0.23,<2", # FrameProcessor support requires livekit>=1.0.23 (PR #4145) | ||
| "numpy>=1.20.0", | ||
| ] |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
echo "=== Checking for version.py files in livekit-agents ==="
fd -t f "version.py" --base-directory . | grep -E "(livekit-agents|agents)" | head -5
echo ""
echo "=== Livekit-agents version constraints in pyproject.toml files ==="
rg "livekit-agents>=" --type toml | head -15
echo ""
echo "=== Livekit version constraints in pyproject.toml files ==="
rg "livekit>=" --type toml | grep -v "livekit-agents" | head -15
echo ""
echo "=== Checking krisp pyproject.toml ==="
cat livekit-plugins/livekit-plugins-krisp/pyproject.toml | grep -A 10 "dependencies"
echo ""
echo "=== Checking other plugins for comparison ==="
find livekit-plugins -name "pyproject.toml" -type f | head -5 | xargs -I {} sh -c 'echo "File: {}"; cat {} | grep -A 5 "dependencies" | head -8'Repository: livekit/agents
Length of output: 3162
Update livekit-agents minimum version to 1.3.10 to match other plugins.
The krisp plugin specifies livekit-agents>=1.3.6, but all other plugins in the repository require livekit-agents>=1.3.10 (resemble, langchain, tavus, turn-detector, speechmatics, soniox, simli, rtzr, silero, bey, nltk, nvidia, mistralai, neuphonic, lmnt, etc.). Update the constraint to >=1.3.10 for consistency.
The livekit>=1.0.23,<2 constraint is appropriate based on the FrameProcessor support requirement noted in the comment.
🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-krisp/pyproject.toml` around lines 36 - 40,
Update the livekit-agents dependency in pyproject.toml from
"livekit-agents>=1.3.6" to "livekit-agents>=1.3.10" to match the minimum used by
other plugins; leave the "livekit>=1.0.23,<2" and "numpy>=1.20.0" entries
unchanged. Use the dependency string "livekit-agents>=1.3.10" so tools will pick
up the consistent minimum across plugins.
| # Basic test | ||
| python test_audio_filtering.py input.wav output.wav | ||
|
|
||
| # With visualization (spectrograms) | ||
| python test_audio_filtering.py input.wav output.wav --visualize | ||
|
|
||
| # Custom parameters | ||
| python test_audio_filtering.py input.wav output.wav \ | ||
| --level 80 \ | ||
| --frame-duration 20 \ | ||
| --visualize | ||
| ``` |
There was a problem hiding this comment.
Test script filename mismatch.
The documentation references test_audio_filtering.py, but the actual test file in the PR is named test_viva_filter_audiofile.py. Update the documentation to match the actual filename.
📝 Suggested fix
### Test with Audio Files
```bash
# Basic test
-python test_audio_filtering.py input.wav output.wav
+python test_viva_filter_audiofile.py input.wav output.wav
# With visualization (spectrograms)
-python test_audio_filtering.py input.wav output.wav --visualize
+python test_viva_filter_audiofile.py input.wav output.wav --visualize
# Custom parameters
-python test_audio_filtering.py input.wav output.wav \
+python test_viva_filter_audiofile.py input.wav output.wav \
--level 80 \
--frame-duration 20 \
- --visualizeNote: The --visualize flag doesn't appear to be implemented in the actual test script. Consider removing it or implementing the feature.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| # Basic test | |
| python test_audio_filtering.py input.wav output.wav | |
| # With visualization (spectrograms) | |
| python test_audio_filtering.py input.wav output.wav --visualize | |
| # Custom parameters | |
| python test_audio_filtering.py input.wav output.wav \ | |
| --level 80 \ | |
| --frame-duration 20 \ | |
| --visualize | |
| ``` | |
| # Basic test | |
| python test_viva_filter_audiofile.py input.wav output.wav | |
| # With visualization (spectrograms) | |
| python test_viva_filter_audiofile.py input.wav output.wav --visualize | |
| # Custom parameters | |
| python test_viva_filter_audiofile.py input.wav output.wav \ | |
| --level 80 \ | |
| --frame-duration 20 |
🤖 Prompt for AI Agents
In `@livekit-plugins/livekit-plugins-krisp/README.md` around lines 143 - 154,
Update the README usage examples to call the actual test script name
test_viva_filter_audiofile.py instead of test_audio_filtering.py: replace all
occurrences in the three examples (basic test, with visualization, custom
parameters) to use python test_viva_filter_audiofile.py input.wav output.wav,
and either remove the --visualize option from the examples or implement the
--visualize flag in test_viva_filter_audiofile.py to match the documentation.
| import sys | ||
|
|
||
| import numpy as np | ||
| import soundfile as sf |
There was a problem hiding this comment.
sf isn't declared as a dependency (which is correct, since this isn't needed by the plugin). what would users get when they run the test files?
There was a problem hiding this comment.
removed internal tests similar to other plugins.
…nals to mypy that the livekit.plugins.krisp package is typed and can be type-checked.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| if self._session is None or self._sample_rate != frame.sample_rate: | ||
| raise ValueError(f"Session not created or sample rate mismatch: {frame.sample_rate}Hz") |
There was a problem hiding this comment.
🔴 _process raises ValueError after _close during track transitions instead of recreating session
After a track transition (e.g., participant reconnects or track switches), the framework calls _close_stream() (livekit-agents/livekit/agents/voice/room_io/_input.py:168-169) which invokes self._processor._close(). This sets self._session = None in the Krisp filter. When a new track arrives, _on_track_available (_input.py:184-199) creates a new stream, passing the same FrameProcessor to rtc.AudioStream.from_track. The AudioStream then calls _process() for each frame, but _process() at line 223 raises a ValueError instead of recreating the session.
Root Cause and Impact
The _close() method at line 288-299 intentionally only destroys the session (not the SDK reference), with a comment explaining it's called during track transitions. However, _process() at line 223 does not handle the self._session is None case by recreating the session — it simply raises:
if self._session is None or self._sample_rate != frame.sample_rate:
raise ValueError(f"Session not created or sample rate mismatch: {frame.sample_rate}Hz")This means after any track transition, every subsequent call to _process() will raise ValueError. While _apply_audio_processor in _input.py:362-364 catches exceptions and falls back to the original frame, the main audio processing path through the rtc.AudioStream FrameProcessor integration may not have the same safety net, causing noise cancellation to silently stop working or errors to propagate.
The fix should recreate the session in _process() when self._session is None, similar to how KrispVivaTurn.process_audio() handles it at viva_turn.py:219-220.
| if self._session is None or self._sample_rate != frame.sample_rate: | |
| raise ValueError(f"Session not created or sample rate mismatch: {frame.sample_rate}Hz") | |
| if self._session is None or self._sample_rate != frame.sample_rate: | |
| self._create_session(frame.sample_rate) | |
Was this helpful? React with 👍 or 👎 to provide feedback.
| def close(self) -> None: | ||
| """Clean up processor session resources (public method for backward compatibility).""" | ||
| self._close() |
There was a problem hiding this comment.
🔴 KrispVivaFilterFrameProcessor.close() never releases SDK reference, causing resource leak
The public close() method (line 301-303) and context manager __exit__ (line 331-333) only call _close(), which nullifies the session but never releases the SDK reference via KrispSDKManager.release(). The SDK reference is only released in __del__, which may never be called promptly or at all.
Comparison with KrispVivaTurn and Impact
Contrast with KrispVivaTurn.close() at viva_turn.py:361-370, which correctly releases the SDK reference:
def close(self) -> None:
self._tt_session = None
self._pre_load_turn_session = None
self._state.audio_buffer.clear()
if getattr(self, "_sdk_acquired", False):
KrispSDKManager.release()
self._sdk_acquired = FalseThe documentation in the README (lines 199-203) explicitly shows close() releasing the SDK reference:
noise_processor.close() # SDK still active (turn_detector holds reference)
turn_detector.close() # SDK now destroyed (last reference released)But KrispVivaFilterFrameProcessor.close() at line 301-303 just delegates to _close() which only sets self._session = None. This means the SDK reference count is never decremented, the SDK is never cleaned up, and KrispSDKManager._reference_count grows monotonically.
The fix needs to be careful: _close() is called during track transitions by the framework and should NOT release the SDK, but close() is the user-facing cleanup and SHOULD release it.
| def close(self) -> None: | |
| """Clean up processor session resources (public method for backward compatibility).""" | |
| self._close() | |
| def close(self) -> None: | |
| """Clean up processor resources (public method for backward compatibility).""" | |
| self._close() | |
| if getattr(self, "_sdk_acquired", False): | |
| KrispSDKManager.release() | |
| self._sdk_acquired = False | |
Was this helpful? React with 👍 or 👎 to provide feedback.
* upstream/main: (31 commits) chore: reduce renovate noise (livekit#5421) Rename e2ee to encryption in JobContext.connect (livekit#5454) feat: add Runway Characters avatar plugin (livekit#5355) Fix FrameProcessor lifecycle for selector based noise cancellation (livekit#5433) Feature - Configurable session close transcript timeout (livekit#5328) add ToolSearchToolset and ToolProxyToolset for dynamic tool discovery (livekit#5140) feat(beta/workflows): add InstructionParts for modular instruction customization (livekit#5077) fix: allow multiple AsyncToolsets by deduplicating management tools (livekit#5369) fix: empty transcript blocks commit_user_turn until timeout (livekit#5429) Feature/krisp viva sdk support (livekit#4370) feat: add service_tier parameter to Responses API LLM (livekit#5346) fix(inworld): do not leak connections when when cancelled (livekit#5427) update: Sarvam STT - add verbose error loggin and remove retry connection (livekit#5373) chore(deps): update github workflows (major) (livekit#5424) (azure openai): ensure gpt-realtime-1.5 compatibility (livekit#5407) chore(deps): update dependency nltk to v3.9.4 [security] (livekit#5418) chore(deps): update dependency aiohttp to v3.13.4 [security] (livekit#5416) chore(deps): update dependency langchain-core to v1.2.28 [security] (livekit#5417) chore: pin GHA by commit (livekit#5415) fix(aws): unwrap doubly-encoded JSON tool arguments from Nova Sonic (livekit#5411) ...
3 participants
Add Krisp VIVA plugin for noise reduction and turn detection
Add KrispAudioInput wrapper for Krisp NC integration
Add AudioInput wrapper class to integrate Krisp noise cancellation
into LiveKit Agents audio pipeline for human-to-bot conversations.
Summary by CodeRabbit
Release Notes
✏️ Tip: You can customize this high-level summary in your review settings.