fix(engine,producer): parse sub-composition media inside <template> wrappers#475
Closed
MuTsunTsai wants to merge 2 commits intoheygen-com:mainfrom
Closed
fix(engine,producer): parse sub-composition media inside <template> wrappers#475MuTsunTsai wants to merge 2 commits intoheygen-com:mainfrom
MuTsunTsai wants to merge 2 commits intoheygen-com:mainfrom
Conversation
…rappers Sub-compositions wrapped in <template> had their media silently skipped because linkedom's querySelectorAll (like the browser) doesn't descend into template content. Unwrap one level of <template> in parseAudio / Video / ImageElements via a shared helper, and guard recompileWithResolutions from clobbering first-pass offsets when the inlined HTML yields no sub-composition media.
The browser reconciliation step copies `data-end` from the browser DOM onto already-compiled media records when they disagree by more than a tick. For sub-composition clips that is wrong: the compiled end was produced by `parseSubCompositions` after adding the host's `data-start` offset, while the browser reads the inlined DOM where the clip's `data-end` is still scene-local. The reconciler sees the gap and replaces the offset-adjusted end with the scene-local value, clipping the tail of every sub-composition video/audio at render time. Only fill `end` (and, for video, mediaStart / hasAudio) when the static pipeline had no value — keep the compiled `end` authoritative otherwise. Preview was already correct because it never runs this reconciliation.
Contributor
Author
|
Follow-up: also stop executeRenderJob's browser-metadata reconciliation from overwriting end / mediaStart on media that already has a compiled value. Without this, sub-composition clips render with their pre-offset scene-local end and the tail gets clipped even though parseSubCompositions computed the correct end. |
Contributor
Author
|
Close as #476 fixed this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Sub-compositions wrapped in
<template>had their<video>/<audio>/<img>elements silently skipped during render, so the parent host'sdata-startoffset never got applied — scene-local media played at raw scene-local time, overlapping the intro instead of following it.Two linked fixes:
parseVideoElements,parseAudioElements,parseImageElementsnow unwrap one level of<template>before querying the DOM.recompileWithResolutionspreserves the first-pass media arrays when the inlined HTML yields no sub-composition data, instead of clobbering the already-offset values with scene-local ones.Why
parseSubCompositionsreads the raw sub-composition HTML, probes its media, and adds the parent'sdata-startto each clip'sstart/end. But thoseparse*Elementshelpers use linkedom'squerySelectorAll, which follows the browser contract of not descending into template content (it lives in aDocumentFragment). The GSAP composition guide recommends wrapping sub-composition bodies in<template>, so every composition following that convention silently parsed as zero media.recompileWithResolutionsruns a second pass after durations are resolved from the browser. By that point the HTML has already been inlined, so[data-composition-src]hosts are gone andparseSubCompositionslegitimately returns empty. The old code unconditionally redid the dedupe, letting scene-local media parsed from the inlined HTML overwrite the correctly-offset media from the first pass.What changed
packages/engine/src/utils/htmlTemplate.ts— newunwrapTemplate()helper.packages/engine/src/services/audioMixer.ts—parseAudioElementscallsunwrapTemplatebeforeparseHTML.packages/engine/src/services/videoFrameExtractor.ts—parseVideoElementsandparseImageElementsdo the same.packages/producer/src/services/htmlCompiler.ts—recompileWithResolutionsgates the dedupe overwrite behind ahasSubMediacheck.Test plan
bun run --filter @hyperframes/engine testbun run --filter '*' typecheckbunx oxlint+bunx oxfmt --checkon the edited files<template>-wrapped sub-composition atdata-start="4"— confirm the scene media play starting at t=4s, not t=0.