test(hdr-regression): tighten Window F maxFrameFailures budget after Chunk 4 fix#375
Merged
vanceingalls merged 5 commits intomainfrom Apr 23, 2026
Merged
test(hdr-regression): tighten Window F maxFrameFailures budget after Chunk 4 fix#375vanceingalls merged 5 commits intomainfrom
vanceingalls merged 5 commits intomainfrom
Conversation
This was referenced Apr 21, 2026
Collaborator
Author
This was referenced Apr 21, 2026
Open
perf(producer): hdr benchmark harness — --tags filter, peak heap/RSS tracking, bench:hdr script
#382
Open
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
72b0694 to
f328731
Compare
vanceingalls
force-pushed
the
vance/regen-window-f
branch
2 times, most recently
from
becf52d to
f4e6cc1
Compare
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
79b029f to
e78f981
Compare
vanceingalls
force-pushed
the
vance/regen-window-f
branch
2 times, most recently
from
5bfb45e to
f6b35f6
Compare
vanceingalls
force-pushed
the
vance/transform-clipping
branch
2 times, most recently
from
45009c4 to
cb840e7
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
d233789 to
404ec3f
Compare
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
05b3b6c to
81bb819
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
81bb819 to
df93842
Compare
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
404ec3f to
cf5c309
Compare
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
df93842 to
5471b3f
Compare
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
cf5c309 to
13d8aa7
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
13d8aa7 to
ff96886
Compare
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
5471b3f to
b38bd95
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
ff96886 to
b841366
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
b38bd95 to
53a345a
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/transform-clipping
branch
from
b841366 to
a56ed28
Compare
vanceingalls
added a commit
that referenced
this pull request
Apr 23, 2026
Address jrusso1020's nit on PR #365 (non-blocking review): both READMEs now explain where the tolerance values come from. - hdr-regression/README.md: add a budget-breakdown table that derives the 30 frames from the deltas in PRs #369 (window C fix → 5) and #375 (window F fix → 0). The table doubles as a contract: if a future change forces the budget back up, exactly one bucket has regressed and the table tells you which one to investigate first. - hdr-hlg-regression/README.md: add a 'Tolerance' section explaining why 0 is the right floor (HLG is a pure pass-through path, HEVC over rgb48le is byte-deterministic on the same fixture, so any drift is a real regression). The regeneration command for generate-hdr-photo-pq.py was already documented at README lines 67-71, so no changes needed there.
vanceingalls
force-pushed
the
vance/regen-window-f
branch
from
53a345a to
7724916
Compare
Replace the old hdr-pq + hdr-image-only tests with two consolidated regression suites that exercise the full HDR pipeline. hdr-regression (PQ, BT.2020, ~20s): - 8 windows (A-H) covering clip-only video, image+video composition, wrapper opacity, direct-on-video opacity, scene transitions, transform + border-radius, mid-clip cuts, and shader transitions. - Reuses the existing hdr-clip.mp4 fixture (NOTICE.md preserved). - New hdr-photo-pq.png generated via scripts/generate-hdr-photo-pq.py (writes a cICP chunk for BT.2020/PQ/full). hdr-hlg-regression (HLG, ARIB STD-B67, ~5s): - 2 windows (A-B) covering clip-only HLG playback and HLG + opacity tween. - New hdr-hlg-clip.mp4 fixture (last 5s of a user-recorded HLG iPhone clip). Both compositions follow the documented timed-element pattern: data-start, data-duration, and class="clip" applied directly to each timed leaf element (no wrapper inheritance). CI: regression workflow's hdr shard now runs the new pair sequentially. LFS: new MP4 fixtures and golden outputs are tracked via existing rules. Goldens generated with bun run test:update --sequential. ffprobe verifies HEVC/yuv420p10le/bt2020nc/smpte2084 (PQ) and arib-std-b67 (HLG). Made-with: Cursor
Four related bugs in the opacity pipeline that interact with HDR video
compositing and GSAP-controlled fades:
1A. screenshotService injectVideoFramesBatch and syncVideoFrameVisibility
were applying `opacity: 0 !important` to native <video> elements to
hide them under the injected <img>. That stomp clobbered any
GSAP-controlled inline opacity, so the next seek read 0 from
computed style and the comp went black. We now use
`visibility: hidden !important` only — visibility hides the element
from rendering without changing its opacity, so subsequent reads
(and queryElementStacking) see the real GSAP value on every frame.
The `parseFloat(...) || 1` recovery hack at injectVideoFramesBatch
was specifically there to compensate for this stomp; it’s now
replaced with a `Number.isNaN` guard that defaults to 1 only when
parsing actually fails.
1B. queryVideoElementBounds parsed `style.opacity` and `style.zIndex`
with `parseFloat(...) || N`, which silently coerces a real opacity
of 0 into 1 (and zIndex 0 into 0 only by coincidence). Switched to
explicit `Number.isNaN` checks so opacity 0 stays 0.
1C. resolveRadius cast `el as HTMLElement` to read offsetWidth/Height.
SVG and other non-HTML elements would have crashed at runtime.
Replaced the cast with an `instanceof HTMLElement` guard, and made
the numeric fallback `Number.isNaN`-safe.
1D. The opacity walk in queryVideoElementBounds started from
`el.parentElement` for HDR videos to skip past the engine’s forced
`opacity: 0` on the element itself. Now that the engine never sets
opacity, the special case is unnecessary — always walk from `el`.
Kept the `isHdrEl` lookup because transform/border-radius logic
further down still branches on it.
Verified:
- bun run --filter @hyperframes/engine typecheck (clean)
- bun run --filter @hyperframes/engine test (308/308 passing)
- bun run --filter @hyperframes/producer typecheck (clean)
- oxlint + oxfmt --check on both touched files
Next: regenerate hdr-regression window C (the opacity-fade window) and
tighten its maxFrameFailures budget — done in a follow-up commit so the
golden churn is reviewable separately from the code fix.
…Chunk 1 fix Window C (direct <video> opacity tween) was previously listed as a known failure with a maxFrameFailures budget of 30 to absorb expected drift until Chunk 1 (opacity pipeline bugs) landed. After the Chunk 1 fix, the regression test passes against the existing golden with 0 failed frames, confirming Window C now renders correctly. Regenerating the golden produces byte-identical output (HEVC encoding is deterministic and the opacity fix doesn't perturb pixels at the PSNR ≥ 28 checkpoint threshold). Tighten maxFrameFailures from 30 → 5 to leave only a small budget for encoder noise. Window F (transform + border-radius) remains pending Chunk 4; its broken state is currently baked into the golden, so the suite is green and Chunk 4's regen will catch any drift. Update README.md to reflect Window C is fixed and the tightened budget.
Collaborator
Author
Merge activity
|
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Tighten
hdr-regressionWindow FmaxFrameFailuresfrom 5 → 0 now that Chunk 4 (matrix3d support + scene initial-state) has landed.Why
Window F (transform + scale + border-radius on the video itself) was the remaining known-fail in the
hdr-regressionsuite, baked into the golden so the suite stayed green while Chunk 4 was outstanding.After Chunk 4 fixed
parseTransformMatrix(matrix3d support) and the shader-transitions initial-state, re-running the suite shows 0 failed frames against the existing golden — the encoder is byte-deterministic, and Window F's GSAP rotation/scale happens to emit 2Dmatrix()rather thanmatrix3d(), so the same golden is still correct after the fix. Tightening the budget catches any drift in the layered HDR compositor immediately.What changed
tests/hdr-regression/meta.json:maxFrameFailures5 → 0 (matcheshdr-hlg-regression).tests/hdr-regression/README.md: Window F row + Fix history section updated to reflect the new state.Test plan
bun run test --filter hdr-regression— passes with 0 failed frames at the new budget.Stack
Follow-up to Chunk 4 (transform & clipping). Reviewable separately so the budget tightening is decoupled from the code fix.