feat(hdr): layered HDR compositing, shader transitions, and HDR image support#268
Merged
vanceingalls merged 8 commits intomainfrom Apr 20, 2026
Merged
feat(hdr): layered HDR compositing, shader transitions, and HDR image support#268vanceingalls merged 8 commits intomainfrom
vanceingalls merged 8 commits intomainfrom
Conversation
85de722 to
0b70603
Compare
eabca43 to
a7db7f6
Compare
Collaborator
Author
This was referenced Apr 16, 2026
32cf08c to
6976409
Compare
8b9096f to
3e571ca
Compare
6976409 to
c62a8f6
Compare
3e571ca to
c8cb0d0
Compare
da57514 to
a593e3f
Compare
c8cb0d0 to
dc89f54
Compare
a0a2680 to
8830de9
Compare
d8cb575 to
8284e5a
Compare
f3bbf79 to
8396136
Compare
8284e5a to
26fadc0
Compare
8396136 to
05c2b21
Compare
26fadc0 to
453f8e4
Compare
05c2b21 to
e523278
Compare
5845b44 to
3284788
Compare
Collaborator
Author
CI fix: typecheck regression in
|
c7f106c to
840df0c
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 19, 2026
- Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268.
6f97988 to
e2d1dfd
Compare
eab565e to
559f8d5
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 19, 2026
- Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268.
e2d1dfd to
9f2df0f
Compare
- 15 GLSL→TypeScript shader transitions on rgb48le buffers - Dual-scene compositing with scene detection via window.__hf.transitions - --hdr flag gates ffprobe probing (zero overhead on SDR compositions) - Cross-transfer conversion (PQ↔HLG) via OOTF-corrected composite LUT - Buffer.from() copy in writeFrame() fixes streaming encoder race condition - SDR rendering fixes (three stacked bugs) - Object.assign fix for window.__hf preservation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Tighten the all-transitions smoke test thresholds: at progress=0 we now require the center pixel R-channel > 35000 (was > 25000) and at progress=1 < 15000 (was < 25000). The old midpoint of 25000 sat exactly halfway between the test from-pixel (40000) and to-pixel (10000), so a half-blended transition would silently pass. - Add a runtime assertion in HyperShader.init() that every scene id resolves to a DOM element with the .scene class. Without this, missing ids silently no-op when textures + querySelectorAll(.scene) run later. Addresses deferred review feedback from PR #268.
…rOrchestrator Commit c6b4619 ("feat(hdr): shader transitions, --hdr flag, and SDR rendering fixes") accidentally removed two pieces of the deterministic rendering pipeline: 1. The `VIRTUAL_TIME_SHIM` injected via `createFileServer.preHeadScripts`, which freezes `Date.now()` and `requestAnimationFrame` so RAF-driven animations advance only when `window.__hf.seek(t)` is called. 2. The `applyRenderModeHints` function and its post-`compileForRender` call site, which auto-forces screenshot capture mode for compositions the compiler flagged as needing it (RAF, iframes, etc.). Without (1), RAF animations advanced by wall-clock between the main-loop seek and the per-DOM-layer seek inside `compositeToBuffer`, producing the sawtooth PSNR pattern on `raf-ball-render-compat` (high PSNR at integer seconds, ~24 dB everywhere else). Without (2), `iframe-render-compat` lost its automatic fallback to screenshot mode and the child-document motion stopped being captured. Both helpers are still produced by `htmlCompiler` and exercised by `renderOrchestrator.test.ts` — the orchestrator just stopped calling them. Restored: - Re-import `VIRTUAL_TIME_SHIM` from `./fileServer.js` - Pass `preHeadScripts: [VIRTUAL_TIME_SHIM]` to both `createFileServer` call sites (probe + main render) - Re-add `applyRenderModeHints` (matching the test expectations) and call it immediately after `compileForRender` - Persist `renderModeHints` in `summary.json` and the "Compiled composition metadata" log line Fixes the `iframe-render-compat` and `raf-ball-render-compat` regression failures on `feat/hdr-layered-compositing`. Made-with: Cursor
…ment Adds: - 8 new sampleRgb48le bilinear-interpolation tests covering boundary pixels, sub-pixel weights, edge clamping, and odd-byte-offset Buffers. - uint16-alignment-audit.test.ts documenting the alignment requirement for Uint16Array views over Buffer slices vs. readUInt16LE/writeUInt16LE. Background: ~105 hot-loop sites in shader transitions still use readUInt16LE/writeUInt16LE. Switching to Uint16Array views would cut overhead but requires guaranteed even byteOffsets — these tests document the contract before any future refactor lands.
The HDR layered compositor blits z-ordered layers over a shared canvas. DOM
layers used a full-page screenshot from `captureAlphaPng`, which captures
*every* painted pixel on the page — root background, sibling-scene content,
overlay UI elements that aren't part of the current layer. Those opaque
pixels were then blitted over the canvas, overwriting any HDR content
composited beneath in earlier layers.
The previous workaround toggled `display:none` on hide ids via
`hideVideoElements`/`showVideoElements`. That correctly hid native videos
but did nothing about the root composition's background or about overlay
elements that the layer grouping considered part of a different layer.
This commit replaces the workaround with a precise CSS mask installed
before each DOM screenshot:
1. `applyDomLayerMask` injects a stylesheet that hides every `body *` and
re-shows the layer's elements (and their descendants and their injected
`__render_frame_*` siblings) with `visibility: visible !important`. CSS
visibility is *not* multiplicative through descendants — a child with
`visibility: visible` overrides an ancestor's `visibility: hidden`, so
deeply nested layer content still paints even though every intermediate
ancestor is hidden by the mass-hide rule.
2. Non-layer data-start ids are inline-hidden with
`visibility: hidden !important`. Inline `!important` beats stylesheet
`!important`, so this overrides the show rule for elements that fall
under a show selector but should NOT paint — most importantly HDR
videos and other-layer SDR videos that live as descendants of `#root`.
3. `removeDomLayerMask` tears the stylesheet down and clears the inline
`visibility`/`opacity` properties so subsequent video frame injection
gets a clean slate.
Crucially the mask only sets `visibility`, never `opacity`. CSS opacity
*is* multiplicative — `opacity: 0` on `#root` would zero out every
descendant including layer videos, even with `visibility: visible`. We
also extend `initTransparentBackground` to force the composition root
(`[data-composition-id]`) transparent in addition to `html`/`body`,
because compositions almost always set `#root { background: ... }` and
that background paints across the whole viewport otherwise.
Both compositing paths use the new helpers:
- The per-layer DOM branch (`compositeToBuffer`) for normal frames.
- The transition path (single DOM screenshot per scene) so transition
frames also get a clean per-scene capture.
Adds extensive `KEEP_TEMP=1`-gated diagnostics to `compositeToBuffer`:
per-layer pixel-add accounting, dumps of every captured DOM PNG, and a
periodic raw `rgb48le` snapshot of the composite buffer. These were
essential to diagnosing the root-overwrite bug and stay zero-cost in
normal renders. Also stops the workDir / per-video frame-dir cleanup
when `KEEP_TEMP=1` so the dumps survive past frame N.
Made-with: Cursor
SDR clips inside an HDR composition were rendering at full opacity even
when the user had animated their wrapper opacity (e.g. fade-in or
yoyo). Two bugs in the per-layer screenshot path conspired to drop the
GSAP-applied opacity on the floor:
1. removeDomLayerMask was unconditionally calling
`el.style.removeProperty("opacity")` on every wrapper after each
layer capture. applyDomLayerMask only ever sets `visibility`, so the
only inline opacity present is the value GSAP wrote. Stripping it
between layer captures means that on the next capture (at the same
timestamp), GSAP's `totalTime(t, false)` no-ops because the timeline
is already at that time — the opacity is never restored, and the
wrapper renders fully opaque.
2. injectVideoFramesBatch was reading the source <video>'s computed
opacity via `parseFloat(computedStyle.opacity) || 1` and copying it
onto the injected <img>. Because syncVideoFrameVisibility forces the
<video> to `opacity: 0 !important` to hide it during capture, the
computed value is always 0, which `|| 1` then silently flips to
full opacity. The <img> is a sibling of the <video> inside the same
wrapper, so it should inherit opacity from the wrapper directly
instead of having a value hard-set on it.
Fix both: drop the opacity removal in removeDomLayerMask, skip opacity
when copying visual properties from <video> to <img>, and explicitly
clear any stale inline opacity on the <img> so it inherits from the
wrapper that GSAP is animating.
Made-with: Cursor
9f2df0f to
a7e1d86
Compare
559f8d5 to
8548a17
Compare
The diagnostic logging block in executeRenderJob's HDR layer composite path referenced an undeclared `hdrLayerStartTimes` map. The correct variable, declared and populated earlier in the same function, is `hdrVideoStartTimes`. The typo was introduced alongside the DOM-layer masking work and broke the producer build/typecheck on CI. Made-with: Cursor
a7e1d86 to
6da105c
Compare
Commit 188ebcc removed the opacity copy from `injectVideoFramesBatch` on the assumption that the <img> sibling would inherit GSAP's opacity from a shared wrapper. That breaks any composition where GSAP animates opacity directly on the <video> element itself: the <img> has no animated ancestor and renders at full opacity throughout any fade, even when the user's intent is partial or zero opacity. The CI `style-7-prod` and `style-8-prod` regressions caught this: the <video id="aroll"> fade-in from 3.0-3.5s rendered as a hard cut because the <img> inherited opacity 1 regardless of GSAP's tween. Restore the old explicit copy from `computedStyle.opacity` to the <img>'s inline opacity, with the `|| 1` fallback intentionally preserved. The fallback is load-bearing: GSAP's seek does not re-apply tweens that have already completed, so post-fade frames read opacity 0 from the stale `opacity: 0 !important` we apply to hide the native <video>. The `|| 1` recovers the tween's end-state opacity 1 for those frames, matching the final on-screen intent and the existing baseline renders. Handles both DOM shapes: - GSAP on wrapper: video's own computed opacity is 1, img set to 1, wrapper's opacity applies via stacking as before. - GSAP on <video>: video's computed opacity is the tween value, copied to img directly since they are siblings. Fixes: - style-7-prod: 0 failed frames (was 2 @ t=3.17, 3.33) - style-8-prod: 0 failed frames (was 2 @ t=3.05, 3.24) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
vanceingalls
added a commit
that referenced
this pull request
Apr 20, 2026
The --hdr flag was silently dropped when combined with --docker because renderDocker did not include it in the args passed to the in-container hyperframes render invocation. Forwarding it produces the same HDR10 MP4 (HEVC, yuv420p10le, BT.2020 PQ, mastering display + max-cll) as a local render — verified end-to-end with the new hdr-pq regression fixture. This PR also fills in the documentation gaps from the HDR feature stack (#288, #289, #290, #265, #268), which each added a slice of HDR support without pulling it together into a single guide. Code change: - packages/cli/src/commands/render.ts: refactor renderDocker to use a new pure helper buildDockerRunArgs (packages/cli/src/utils/dockerRunArgs.ts) that explicitly threads every render flag — including --hdr, --gpu, --quiet — into the docker run argv. Snapshot-tested in dockerRunArgs.test.ts so any future flag drop fails loudly. Docs: - New docs/guides/hdr.mdx — top-level HDR guide: quickstart, how detection works, source media requirements (HDR video + 16-bit HDR PNG), output format constraints, ffprobe verification, Docker rendering, limitations, common pitfalls - docs/guides/common-mistakes.mdx — "Expected HDR but got SDR" entry; --hdr is detection, not force; notes Docker now supports it - docs/guides/rendering.mdx — --hdr row in flags table; HDR card under Next Steps - docs/packages/cli.mdx — expanded --hdr description with link to the guide - docs/packages/producer.mdx — HDR Output section under RenderConfig - docs/packages/engine.mdx — HDR APIs section (color-space utilities + WebGPU readback runtime); now compiles because DEFAULT_HDR10_MASTERING and HdrMasteringMetadata are re-exported from packages/engine/src/index.ts - docs/docs.json — register the new HDR guide in nav Review feedback from PR #346: - packages/engine/src/index.ts — export DEFAULT_HDR10_MASTERING and HdrMasteringMetadata so the engine docs example compiles - docs/guides/hdr.mdx — drop fabricated "animated HDR images log a warning and use first frame" claim (not implemented); clarify that HDR <img> is 16-bit PNG only and other formats fall back to SDR; broaden the yt-dlp example to bestvideo[dynamic_range=HDR]+bestaudio so AV1-HDR streams are also picked up; qualify the Docker render benchmark claim - packages/producer/src/services/renderOrchestrator.ts — bump the HDR + non-mp4 fallback from log.info to log.warn and make the message actionable Regression test (hdr-pq): - packages/producer/tests/hdr-regression — delete the broken sub-tree. The old hdr-pq composition relied on a CDN shader-transitions library and window.__hf injection with scenes defaulting to opacity:0, so when the library failed to load most of the render was invisible. The fixture was also nested two directories deep, so discoverTestSuites (which only scans direct children of tests/) never picked it up. - packages/producer/tests/hdr-pq — new top-level fixture: 5s of an HDR10 HEVC Main10 BT.2020 PQ source clip with a static SDR DOM overlay. Source is a 5s excerpt (1:18–1:23) from https://youtu.be/56hEFqjKG0s, downloaded via yt-dlp and re-encoded to HEVC Main10 / 1920x1080 / 30fps with HDR10 mastering metadata. Self-contained: no external CDNs, no custom fonts, no animation libraries. NOTICE.md captures attribution. - packages/producer/src/regression-harness.ts — add optional hdr boolean to TestMetadata.renderConfig, validate it, and forward it to createRenderJob so HDR fixtures actually run through the HDR pipeline. - .github/workflows/regression.yml — exclude hdr from the fast shard and add a dedicated hdr shard so the new fixture runs on CI. - .gitattributes — LFS-track packages/producer/tests/*/src/*.mp4 so source clips don't bloat the repo. - packages/engine/src/services/frameCapture.ts — inject a no-op __name shim into every page via evaluateOnNewDocument. tsx transforms the engine source on the fly with esbuild's keepNames, which wraps every named function in __name(fn, "name"). When page.evaluate() serializes a callback those calls would crash with "__name is not defined" because the helper only exists in Node. The shim makes the engine work whether it's imported from compiled dist or from source via tsx.
3 tasks
vanceingalls
added a commit
that referenced
this pull request
Apr 22, 2026
…ders The CLI flags `--crf` and `--video-bitrate` were defined and parsed in `packages/cli/src/commands/render.ts`, validated for mutual exclusivity, and threaded into `RenderConfig.crf`/`RenderConfig.videoBitrate`, but the values were silently dropped at the encoder spawn sites in `renderOrchestrator.ts`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only. This change re-introduces the override at the three encoder spawn sites: 1. HDR streaming encoder (rgb48le path) 2. SDR streaming encoder (jpeg/png path) 3. Disk-based encode (encodeFramesFromDir / encodeFramesChunkedConcat) At each site, `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we do not need to re-check it here. Also fixes the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`.
vanceingalls
added a commit
that referenced
this pull request
Apr 22, 2026
…ders The CLI flags `--crf` and `--video-bitrate` were defined and parsed in `packages/cli/src/commands/render.ts`, validated for mutual exclusivity, and threaded into `RenderConfig.crf`/`RenderConfig.videoBitrate`, but the values were silently dropped at the encoder spawn sites in `renderOrchestrator.ts`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only. This change re-introduces the override at the three encoder spawn sites: 1. HDR streaming encoder (rgb48le path) 2. SDR streaming encoder (jpeg/png path) 3. Disk-based encode (encodeFramesFromDir / encodeFramesChunkedConcat) At each site, `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we do not need to re-check it here. Also fixes the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
…ders The CLI flags `--crf` and `--video-bitrate` were defined and parsed in `packages/cli/src/commands/render.ts`, validated for mutual exclusivity, and threaded into `RenderConfig.crf`/`RenderConfig.videoBitrate`, but the values were silently dropped at the encoder spawn sites in `renderOrchestrator.ts`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only. This change re-introduces the override at the three encoder spawn sites: 1. HDR streaming encoder (rgb48le path) 2. SDR streaming encoder (jpeg/png path) 3. Disk-based encode (encodeFramesFromDir / encodeFramesChunkedConcat) At each site, `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we do not need to re-check it here. Also fixes the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
…ders The CLI flags `--crf` and `--video-bitrate` were defined and parsed in `packages/cli/src/commands/render.ts`, validated for mutual exclusivity, and threaded into `RenderConfig.crf`/`RenderConfig.videoBitrate`, but the values were silently dropped at the encoder spawn sites in `renderOrchestrator.ts`. PR #292 originally wired these through with a `baseEncoderOpts` object using `effectiveQuality`/`effectiveBitrate`; PR #268 rewrote the encode paths and reverted to `preset.quality` only. This change re-introduces the override at the three encoder spawn sites: 1. HDR streaming encoder (rgb48le path) 2. SDR streaming encoder (jpeg/png path) 3. Disk-based encode (encodeFramesFromDir / encodeFramesChunkedConcat) At each site, `quality` defaults to `preset.quality` but is overridden by `job.config.crf` when set, and `bitrate` is set from `job.config.videoBitrate`. Mutual exclusivity is enforced upstream in the CLI, so we do not need to re-check it here. Also fixes the contradictory note in `docs/packages/cli.mdx` that claimed CRF/bitrate were now driven only by `--quality`. The flags table now lists `--crf` and `--video-bitrate` consistent with `docs/guides/rendering.mdx`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
Complete HDR video and image pipeline for HyperFrames. Compositions with HDR video (PQ/HLG), SDR video, HDR images, and DOM content (text, graphics, GSAP animations) render into a single H.265 10-bit HDR output. Includes 15 shader transitions, cross-transfer conversion (PQ↔HLG), and zero regression on SDR compositions via the `--hdr` flag.
What it does
HDR Image Support (new)
Shader Transitions (Phase 5)
Critical Bug Fixes
`--hdr` Flag
Performance
Files changed
How to test
Stack position
6 of 6 — Top of the HDR stack. Stacked on #290 (GSAP transforms).
Full stack: #258 → #265 → #288 → #289 → #290 → #268
🤖 Generated with Claude Code