Skip to content

feat: add OpenNOW Native Streamer C++ Based#208

Draft
zortos293 wants to merge 48 commits intodevfrom
capy/native-streamer-backend
Draft

feat: add OpenNOW Native Streamer C++ Based#208
zortos293 wants to merge 48 commits intodevfrom
capy/native-streamer-backend

Conversation

@zortos293
Copy link
Copy Markdown
Collaborator

@zortos293 zortos293 commented Apr 7, 2026

This PR introduces a first-class native streaming backend (opennow-native-streamer/) alongside the existing Chromium renderer, enabled via a beta settings toggle. The native backend handles WebRTC negotiation (via libdatachannel), media playback (via FFmpeg and SDL3), and input (via native SDL capture and GFN protocol encoding), while preserving full Chromium/WebRTC fallback when the setting is disabled.

  • Electron integration: Added persisted enableNativeStreamer setting, NativeStreamerManager service, IPC lifecycle management, and renderer branching in App.tsx
  • Native media/session: Implemented native WebRtcSession (offer/answer/ICE/data channels) and MediaPipeline (FFmpeg decode + SDL3 render/audio)
  • Native input: Implemented InputEncoder and InputBridge to send GFN-compatible packets via libdatachannel data channels
  • Protocol/compatibility: Ported SDP helpers (fixServerIp, preferCodec, mungeAnswerSdp, buildNvstSdp) and manual ICE injection for GFN ICE-lite

Microphone, recording, and screenshot features remain on the Chromium path. The native backend requires SDL3, FFmpeg, and libdatachain runtime dependencies.

Open in Capy OPE-31 · 5.4

@zortos293 zortos293 added the capy Generated by capy.ai label Apr 7, 2026 — with Capy AI
Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 2 comments

zortos293 and others added 14 commits April 8, 2026 08:46
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 2 comments

Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 1 comment

Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 3 comments

Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 1 comment

Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 1 comment

Copy link
Copy Markdown
Contributor

@capy-ai capy-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added 2 comments

setEscHoldReleaseIndicator({ visible: false, progress: 0 });
setStreamBackend(null);
setNativeStreamerState(null);
nativeStopExpectedRef.current = false;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟡 Medium]

resetLaunchRuntime clears nativeStopExpectedRef immediately, but stopNativeStreamer() tears down IPC asynchronously and can still emit a late failed/exited state while streamBackendRef/streamStatusRef are still stale from the previous render tick. That late event then enters the failure branch and surfaces a spurious launch error after a user-initiated stop. Keep the stop-expected guard set until native teardown has fully settled (or clear it only after refs/state have converged to idle/non-native). ```ts
// opennow-stable/src/renderer/src/App.tsx
setStreamBackend(null);
setNativeStreamerState(null);
nativeStopExpectedRef.current = false;
...
if (nativeStopExpectedRef.current || streamStatusRef.current === "idle") {

}
while (avcodec_receive_frame(audio_decoder_ctx_, audio_frame_) == 0) {
if (!swr_context_) {
swr_alloc_set_opts2(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[🟡 Medium]

The resampler setup passes nullptr for out_ch_layout and does not check swr_alloc_set_opts2/swr_init results before calling swr_convert. This leaves audio conversion initialization undefined/failing on first use and can drop native audio output even when decode succeeds. Provide an explicit output channel layout (matching the SDL output format) and gate conversion on successful init. ```cpp
// opennow-native-streamer/src/media/media_pipeline.cpp
swr_alloc_set_opts2(
&swr_context_,
nullptr,
AV_SAMPLE_FMT_S16,
48000,

@zortos293 zortos293 changed the title feat: add OpenNOW Native Streamer backend and fallback feat: add OpenNOW Native Streamer C++ Based Apr 11, 2026
@zortos293 zortos293 added the Testing this issue/pr is being tested not ready for official build yet label Apr 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

capy Generated by capy.ai Testing this issue/pr is being tested not ready for official build yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant