Cwilson/audio processor by yepher · Pull Request #124 · livekit/track-processors-js

yepher · 2026-03-04T16:39:06Z

Build an example audio processor and accompanying documentation to support the example.

…e up comming audio processor work

changeset-bot · 2026-03-04T16:39:15Z

🦋 Changeset detected

Latest commit: e4df303

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@livekit/track-processors	Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copilot

Pull request overview

Adds an example Web Audio–based GainAudioProcessor (plus docs and sample app UI) to demonstrate how to build and use audio TrackProcessors alongside the existing video background processors.

Changes:

Export new GainAudioProcessor + options from the package entrypoint.
Add GainAudioProcessor implementation using a GainNode Web Audio graph.
Add new audio/video processor documentation and update the sample app + README to reference them.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/index.ts	Re-exports the new `GainAudioProcessor` from the package entrypoint.
src/audio/GainAudioProcessor.ts	Introduces a Web Audio gain-based audio `TrackProcessor`.
processor-docs/video-processors.md	Adds standalone documentation for video processors.
processor-docs/audio-processors.md	Adds standalone documentation for audio processors and how to build them.
example/sample.ts	Adds sample app controls to insert/remove the audio processor and adjust gain.
example/index.html	Adds UI for the audio processor toggle + gain slider.
README.md	Refactors README to link out to the new processor docs and mention audio processors.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

1egoman

Just left a bunch of minor mostly documentation nitpicks, great work! I gave it a shot locally for me and it worked.

One other minor thing: before merging, if you could add a changesets entry per the comment on the pull request that would be appreciated!

1egoman · 2026-03-04T20:16:06Z

@@ -1,91 +1,51 @@
 # LiveKit track processors

+Prebuilt audio and video track processors for [LiveKit](https://livekit.io), implementing the [`TrackProcessor`](https://docs.livekit.io/home/client/tracks/manipulate/#track-processors) interface from `livekit-client`.


issue: FYI, this https://docs.livekit.io/home/client/tracks/manipulate/#track-processors link looks to be dead.

Oh, I thought I tried all the links. oops. Will update to this one.
https://docs.livekit.io/reference/client-sdk-js/interfaces/TrackProcessor.html

1egoman · 2026-03-04T20:17:47Z

- `BackgroundProcessor({ mode: 'disabled' })`
-
-### Usage example
+Background blur and virtual background for video tracks:


thought: It might be worthwhile adding a small section here on video track processors here conceptually rather than immediately going right to the "prebuilt" reference processors.

Good idea. I will throw together some mermaid diagrams too

1egoman · 2026-03-04T20:18:38Z

+## Audio processors

-A track processor is instantiated with a Transformer.
+Gain control for audio tracks, and a reference implementation for building custom audio processors:


thought: Same thing as the video track section - It might be worthwhile adding a small section here on audio track processors here conceptually rather than immediately going right to the "prebuilt" reference example processors.

1egoman · 2026-03-04T20:20:30Z

+```ts
+import { BackgroundProcessor } from '@livekit/track-processors';
+
+const videoTrack = await createLocalVideoTrack();
+const processor = BackgroundProcessor({ mode: 'background-blur' });
+await videoTrack.setProcessor(processor);
+room.localParticipant.publishTrack(videoTrack);
+
+async function disableBackgroundBlur() {
+  await videoTrack.stopProcessor();
+}
+
+async function updateBlurRadius(radius) {
+  return processor.switchTo({ mode: 'background-blur', blurRadius: radius });
+}
+```
+
+### Avoiding visual artifacts when switching
+
+In a real application, you'll likely want to toggle background effects on and off. You could call `videoTrack.setProcessor()` / `videoTrack.stopProcessor()` on demand, but these functions can sometimes produce visual artifacts during the switching process, resulting in a poor user experience.
+
+A better approach is to initialize the `BackgroundProcessor` in `disabled` mode and then switch to the desired mode later. This avoids artifacts entirely:
+
+```ts
+const videoTrack = await createLocalVideoTrack();
+const processor = BackgroundProcessor({ mode: 'disabled' });
+await videoTrack.setProcessor(processor);
+room.localParticipant.publishTrack(videoTrack);
+
+async function enableBlur(radius) {
+  await processor.switchTo({ mode: 'background-blur', blurRadius: radius });
+}
+
+async function disableBlur() {
+  await processor.switchTo({ mode: 'disabled' });
+}
+```


nitpick: This example is basically wholesale repeated twice.

1egoman · 2026-03-04T20:29:11Z

+Video processors in this package are built on two layers:
+


nitpick: It might also be worth adding a note here that you don't necessarily have to follow this Transformer pattern, but if you do and pass it to ProcessorWrapper, then ProcessorWrapper abstracts away the actual MediaStreamTrack => VideoFrame => run through transformer => VideoFrame => MediaStreamTrack conversion which is probably not something the majority of track processor use cases would care about.

1egoman · 2026-03-04T20:31:17Z

+  // Optional lifecycle hooks — included for completeness as a reference implementation
+  async onPublish(room: Room): Promise<void> {
+    // No-op: override in subclasses if you need room context
+  }
+
+  async onUnpublish(): Promise<void> {
+    // No-op: override in subclasses for room lifecycle cleanup
+  }


nitpick: IMO, either add a debug log in these so that they do something and you can actually see when they fire in the example app, or get rid of them outright given that they are probably fairly easily discoverable by reading the TrackProcessor type definitions.

I like that they are there a super simple obvious example so will add a log

yepher added 2 commits March 4, 2026 10:24

Restructure readme and adds a couple of docs to split up video and th…

a311b38

…e up comming audio processor work

Adds a sample audio processor example

a525ed0

Adds some changes based on PR review

f5b48d6

yepher requested review from 1egoman, Copilot and lukasIO March 4, 2026 18:30

Copilot started reviewing on behalf of yepher March 4, 2026 18:31 View session

Copilot AI reviewed Mar 4, 2026

View reviewed changes

Fixes items suggested by copilot.

8f4b7d4

1egoman approved these changes Mar 4, 2026

View reviewed changes

Adds changes based on feedback from 1egoman PR review.

e4df303

yepher merged commit 9ef5191 into main Mar 5, 2026
4 checks passed

yepher deleted the cwilson/audio_processor branch March 5, 2026 13:52

github-actions Bot mentioned this pull request Mar 5, 2026

Version Packages #125

Open

		@@ -1,91 +1,51 @@
		# LiveKit track processors

		Prebuilt audio and video track processors for [LiveKit](https://livekit.io), implementing the [`TrackProcessor`](https://docs.livekit.io/home/client/tracks/manipulate/#track-processors) interface from `livekit-client`.

Conversation

yepher commented Mar 4, 2026

Uh oh!

changeset-bot Bot commented Mar 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

1egoman left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

changeset-bot Bot commented Mar 4, 2026 •

edited

Loading