fix: render parity for transparent looped videos

[miguel-heygen]

Summary

• preserve alpha for render-injected video frames by detecting alpha streams with ffprobe and extracting alpha video frames as PNG
• keep <video loop> semantics through static parsing, compiler duration resolution, browser media discovery, and render frame lookup
• fail embedded preview startup before opening a broken browser page when the Studio bundle is missing
• align snapshot frame injection with looped media timing and VP9 alpha extraction

Why

The Studio preview and rendered MP4 could disagree for timed transparent looped videos. The Comfy funding composition exposed two separate parity bugs: render-injected frames needed alpha-preserving PNG extraction, and the compiler was clamping a looped data-duration="4" video down to the 3.125s source duration. After the first source cycle, render lookup treated the video as inactive, hid the native video, and produced the blank polygon/glow the user saw around the rounded 0:03 mark.

hyperframes lint and hyperframes validate did not catch this because they check syntax/load/console/accessibility, not preview-vs-render visual parity. This PR adds regression coverage for the compiler loop-duration path and frame lookup path.

Verification

• bun run --filter @hyperframes/core test -- src/compiler/timingCompiler.test.ts src/compiler/htmlCompiler.test.ts
• bun test packages/producer/src/services/htmlCompiler.test.ts
• bun run --filter @hyperframes/engine test -- videoFrameExtractor ffprobe
• bun run --filter @hyperframes/core typecheck
• bun run --filter @hyperframes/engine typecheck
• bun run --filter @hyperframes/producer typecheck
• bun run --filter @hyperframes/cli typecheck
• bun run lint
• bun run format:check ... on touched files
• Comfy project: node packages/cli/dist/cli.js validate -> no console errors, 44 text elements pass WCAG AA
• Comfy project patched render from source: /tmp/comfy-render-compare/fixed6-comfy.mp4, 1920x1080, 30fps, 21.8s, 654 frames
• 3.00s-3.97s render contact sheet: /tmp/comfy-render-compare/fixed6-window-contact.png
• targeted fixed render capture at 3.733s: /tmp/comfy-render-compare/probe-capture-fixed/captured/frame_000112.jpg
• agent-browser Studio proof screenshot at 3.7s: /tmp/comfy-render-compare/agent-browser-studio-3_7-fixed.png
• agent-browser-driven recording of 3s seek pass: /tmp/comfy-render-compare/agent-browser-wysiwyg-3s-fixed.webm

Note: bun run --filter @hyperframes/cli dev -- validate is blocked in source mode by the existing contrast-audit.browser.js default-export loader issue; packaged node packages/cli/dist/cli.js validate passes for this project.

[jrusso1020]

Verdict: approve — two independent WYSIWYG gaps, both reproduced, both cleanly fixed

Local verification

Looped-duration clamp bug reproduced on main (e8c43f08). Cherry-picked packages/core/src/compiler/htmlCompiler.test.ts onto main and ran it via vitest:

FAIL  src/compiler/htmlCompiler.test.ts > compileHtml > preserves explicit looped media durations that exceed source duration
AssertionError: expected '<video ...>' to contain 'data-duration="4"'
Expected: "data-duration=\"4\""
Received: "<video id=\"hero\" src=\"hero.webm\" data-start=\"0\" data-duration=\"3.125\"
         data-end=\"3.125\" loop data-has-audio=\"true\">"

Compiler clamped data-duration="4" → "3.125" because the source file is 3.125s and the non-looped clamp doesn't know about loop semantics. That's the exact failure shape driving the blank-polygon/glow at ~0:03 in the Comfy repro: after t=3.125s in the render, frame lookup treats the video as inactive, the injected-frame path hides the native <video>, and the rendered MP4 diverges from Studio preview (which loops natively).

Fix verified on 31184087 via vitest:

• bun run test -- src/compiler/timingCompiler.test.ts src/compiler/htmlCompiler.test.ts → 16/16 pass
• bun run test -- videoFrameExtractor ffprobe → 28/28 pass (including the 5 new parseVideoElements/FrameLookupTable/extractMediaMetadata cases)
• bun run --filter @hyperframes/core typecheck → clean

CI on 31184087: 27/28 check runs green; 1 regression-shard (styles-e) still in-flight at review time. Includes Tests on windows-latest + Render on windows-latest green, full perf suite green, all other regression-shards green.

Staff review — two distinct gaps, correctly decomposed

Gap 1: alpha stripping on frame extraction. VP9-alpha videos (pix_fmt=yuva420p or tags.alpha_mode=1) were being decoded with ffmpeg's default VP9 decoder, which drops alpha. Injected PNG/JPG frames then had the wrong colors, producing the polygon/glow artifact. Fix in three places:

• extractMediaMetadata (in ffprobe.ts) now reports hasAlpha based on pixel format (yuva* / rgba / argb / bgra / gbrap / gray*a) OR the alpha_mode stream tag. The regex (^|[^a-z])yuva|rgba|argb|bgra|gbrap|gray[a-z0-9]*a correctly anchors yuva against word boundaries so yuv420p doesn't false-positive.
• extractVideoFramesRange auto-selects PNG output for alpha streams and prepends -c:v libvpx-vp9 to ffmpeg args when the codec is VP9+alpha. The resolveFrameFormat(metadata, options.format) helper defers to an explicit caller-specified format, preserving backward compat — only auto-switches when the caller omits format.
• Snapshot path (packages/cli/src/commands/snapshot.ts) has a matching path via shouldUseVp9AlphaDecoder + alphaDecoderCache. Cache is per-render which is the right scope (file path stable, metadata stable for render lifetime).

One subtle thing worth flagging: the hasAlpha regex uses gbrap (Green-Blue-Red-Alpha Planar). That covers gbrap10le, gbrap16le, etc. — ffmpeg's naming is consistent enough that the regex holds. ✓

Gap 2: looped-duration clamp. The non-looped clamp (data-duration clamped to source duration) is correct for one-shot clips but wrong for looped ones. data-duration="4" on a 3.125s loop means "play one full cycle + 0.875s of the next." Fix propagates a single loop: boolean flag through the pipeline:

• timingCompiler.ts#extractResolvedMedia reads loop from the tag → ResolvedMediaElement.loop
• htmlCompiler.ts#compileHtml skips the duration clamp when el.loop === true
• htmlCompiler.ts (producer) filters the clamp pre-resolution similarly (!!el.src && !el.loop)
• videoFrameExtractor.ts parses loop from <video> tags → VideoElement.loop
• discoverMediaFromBrowser reads loop via htmlEl.hasAttribute("loop") → BrowserMediaElement.loop
• renderOrchestrator merges loop: true when browser discovers it
• FrameLookupTable.addVideo carries loop, and getFrameAtTime + getActiveFramePayloads wrap localTime %= loopDuration when loop && localTime >= loopDuration

That's full pipeline coverage — static parse → compiler → browser discovery → render frame lookup. Preview side doesn't need changes because the browser's native <video loop> handles it; the bug was strictly that render was diverging from preview.

hasAttr regex change is subtle but correct. The old ${attr}=["'] required =" or =', so boolean attrs like loop (which have no = in HTML) were always "not present." New \\s${attr}(?:\\s|=|>|/) correctly matches <video ... loop>, <video loop/>, <video loop>, <video loop="...">, etc.

I traced false-positive scenarios:

• class="loop": char before loop is ", not \s → no match ✓
• data-hf-loop-thing: char before loop is -, not \s → no match ✓
• looper: char after loop is r, not \s|=|>|/ → no match ✓
• <videoloop>: no space before loop → no match ✓

Note on generality: this regex change affects ALL hasAttr callers, not just the new loop one. If there were existing call sites using hasAttr with boolean-style attr names that previously returned false (e.g., checking hasAttr(tag, "muted") on <video muted>), they now return true. I didn't find any in the visible changes, but it's a behavior delta worth knowing if you ever hit a weird edge case post-merge.

Loop math correctness. loopDuration = max(0, metadata.durationSeconds - mediaStart) accounts for data-media-start trim. localTime %= loopDuration wraps. The defensive if (loop && frameIndex >= totalFrames) → last-frame fallback handles the floating-point edge where (localTime %= loopDuration) lands just above the last frame index due to rounding. Correct on both paths. Tests cover wrap (0.5/1.5/4.5 → same frame 15) and non-wrap (past-end on non-looped → undefined).

CLI fail-fast on missing Studio bundle. Orthogonal DX fix bundled in — resolveStudioBundle() now returns {available, checkedPaths} and runEmbeddedMode exits early with a helpful error listing the paths it checked. Matches the "fail before opening a broken browser page" goal in the PR summary. Clean.

Pattern observation across the 4 recent Miguel PRs (hf#474, #476, #477, #478)

Home asked me to flag any common root. There is one, and it's worth naming:

All four PRs close semantic-awareness gaps between the authoring contract and the subsystems that implement it:

• #474 (lint) — lint engine didn't know about GSAP scene-boundary-exit determinism.
• #476 (compiler) — compiler's media scraper didn't know that sub-compositions use <template> wrappers (LinkeDOM DocumentFragment semantics).
• #477 (runtime) — runtime media sync didn't know about nested composition-inherited timing (no direct data-start).
• #478 (compiler + render) — compiler didn't know loop semantics preclude source-duration clamping; render didn't know alpha codecs need -c:v libvpx-vp9 + PNG output.

Each fix adds one new bit of semantic awareness to a subsystem that was already close to correct. The failure mode is always "subsystem silently drops information" — zero-length media arrays (#476), orphan media elements (#477), clamped durations (#478), no-op lint on a real bug (#474).

The meta-question worth raising for the next review pass: is there a way to systematically ensure that subsystems maintain full awareness of the authoring contract? Options:

1. A shared authoring-contract "schema" (the set of supported attributes and their semantics) that each subsystem declares conformance against. Each feature launch requires updating the schema and getting conformance from compiler/runtime/render/lint.
2. A "WYSIWYG contract test" fixture suite that renders a representative set of compositions through both preview and render, diffs the output visually/programmatically, and catches divergence. Per the PR body: hyperframes lint / validate don't catch these because they're syntactic, not semantic-parity. A hyperframes wysiwyg-check (or similar) that exercises template-wrapped comps, nested comps, looped videos, alpha videos, GSAP exit patterns in one sweep would have caught #476/#477/#478 in one go.

Not a thing to block this PR on. Just flagging because the pattern is consistent enough across 4 PRs in 24 hours that it's a signal worth surfacing.

Non-blocking observations on #478 specifically

1. Format argument honored even when wrong for alpha. resolveFrameFormat(metadata, options.format) prefers the caller-specified format. So if any caller explicitly passes options.format = "jpg" for an alpha video, alpha silently gets stripped. I grepped the visible diffs and didn't find such a caller, but this is a foot-gun worth a comment at the helper definition ("pass undefined to auto-detect; explicit format disables alpha auto-promotion").

2. Cache key churn. The cache key includes format. Existing alpha-video cache entries keyed as JPG will become orphaned when the new logic promotes them to PNG on the same (path, mtime, size). No correctness issue (new entries are fresh), just cache-bloat. A hfcache-v3- bump would force a fresh start; not necessary.

3. extractVideoFrameToBuffer signature. Added a positional boolean useVp9AlphaDecoder = false as the third argument. Positional booleans are a minor smell when they compose — if a future fourth argument shows up, foo(a, b, true, maybe_something) gets hard to read. Options: wrap in an options object, or name it explicitly. Low priority.

4. Loop-wrap test coverage — missing source-longer-than-display case. The existing FrameLookupTable test has totalFrames=30, fps=30, end=5 → loopDuration=1s, display window of 5s — so we loop 5× in the window. A mirror test for "source longer than display window" would verify that the wrap math stays within [0, loopDuration) even when localTime / loopDuration < 1 (no wrap needed). Not blocking.

5. Snapshot loop-wrap duplication. snapshot.ts implements its own relTime wrap formula separate from getFrameAtTime's. Both should produce the same answer, but having two copies of the same math means bugs can diverge. Minor maintenance concern.

Ship once the last regression-shard settles.

Review by Rames Jusso