Skip to content

feat(studio): add manual DOM editing inspector#466

Merged
miguel-heygen merged 26 commits intonextfrom
feat/studio-manual-dom-editing-v1
Apr 25, 2026
Merged

feat(studio): add manual DOM editing inspector#466
miguel-heygen merged 26 commits intonextfrom
feat/studio-manual-dom-editing-v1

Conversation

@miguel-heygen
Copy link
Copy Markdown
Collaborator

@miguel-heygen miguel-heygen commented Apr 24, 2026

Problem

Studio had the pieces for previewing compositions and editing timeline clips, but the authoring loop still forced users back into source code for common visual changes:

  • preview selection did not expose a focused, durable design surface for safe geometry, style, fill, text, or typography edits
  • unsupported move / resize states were hard to understand and could leave users guessing why a layer could not be dragged
  • image assets could be imported or dropped, but not reliably resized as timeline clips or reused as background-image fills from the design panel
  • text editing was too raw for the first design-panel version: long content, font family strings, delayed slider values, empty text states, and oversized controls made the panel feel disconnected from the rest of Studio
  • color picking used native browser UI that could appear far away from the clicked row and get clipped near the bottom of the viewport
  • composition thumbnails relied on generated JPEGs that could drift from the live iframe preview and cache stale geometry

What this fixes

Manual design inspector

  • adds direct preview selection for DOM-backed layers with durable source patching
  • keeps move / resize edits gated to safe targets: absolute or fixed positioning, pixel geometry, and no transform-driven layout
  • allows safe composition hosts such as #detail-host to move as a whole from master view while keeping inner contents drill-down only
  • adds an explicit Make movable path for block-ish layout-controlled layers instead of silently detaching them on drag
  • shows a toast when drag is blocked and points users to the panel action when detaching is available
  • clears stale overlays when Inspector closes or the user clicks empty preview space
  • keeps the header Inspector button selected while the right inspector panel is open, including the Renders tab

Design-panel controls

  • adds compact Paper-style sections for layout, flex metadata, radius, blending, fill, text, typography, and selection colors
  • replaces raw gradient strings with a visual gradient editor that supports linear, radial, conic, repeating gradients, stop editing, reversal, and click-to-add stops
  • adds image-fill selection from project assets plus inline upload from the fill panel
  • replaces the native color input with a viewport-clamped floating picker with crosshair guides, visible slider handles, saturation/value, hue, alpha, and hex controls
  • makes color rows open the color picker from the whole row without closing unexpectedly during picker interaction
  • adds wheel / arrow-key scrubbing for numeric layout fields and live value feedback for sliders
  • removes no-op card-like groups and disabled outline / border controls from the default design flow

Text and font editing

  • exposes editable text fields for selected text nodes instead of collapsing everything into one hard-to-edit blob
  • supports live text, font-size, font-weight, and font-family updates in the preview and source
  • keeps emptied text layers editable so users can delete all text and still type new content back into the selected layer
  • adds a searchable font picker backed by current document fonts, local fonts, imported font assets, and Google Fonts metadata
  • injects selected Google/imported font styles into the preview document so typography edits render immediately

Media, thumbnails, and source fidelity

  • lets imported image clips keep editable width / height instead of locking them as unsupported geometry
  • rewrites sub-composition asset URLs so asset-backed image fills resolve correctly after bundling and inline preview rendering
  • replaces generated JPEG composition-list thumbnails with the same iframe preview used on hover
  • seeks idle iframe thumbnails to 3s by default, or the composition midpoint when the duration is shorter than 3s
  • improves thumbnail cache keys for remaining generated thumbnail routes by including URL version, dimensions, source mtime, seek time, and selector
  • improves timeline source patching, target disambiguation, playhead sync, fit-mode scroll behavior, and percentage-based zoom

Root cause

The Studio UI previously mixed three different editing models:

  • timeline state could identify clips, but not always the exact DOM/source target to patch
  • preview overlays could show the selected element, but selection state was not consistently cleared or constrained by edit capability
  • the design panel was reading computed styles, but several controls still wrote raw strings or preview-only values instead of durable source edits

This PR tightens those paths so the inspector only offers edits it can persist safely, and makes the panel controls map more directly to the source mutations Studio can actually perform today.

Verification

Local checks

  • bun test packages/core/src/studio-api/routes/thumbnail.test.ts packages/studio/src/components/editor/domEditing.test.ts packages/studio/src/utils/sourcePatcher.test.ts packages/studio/src/components/sidebar/CompositionsTab.test.ts
  • bun test packages/studio/src/components/editor/colorValue.test.ts packages/studio/src/components/editor/floatingPanel.test.ts
  • bun run --filter @hyperframes/core typecheck
  • bun run --filter @hyperframes/studio typecheck
  • bunx oxlint packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/studio/src/App.tsx packages/studio/src/components/editor/DomEditOverlay.tsx packages/studio/src/components/editor/PropertyPanel.tsx packages/studio/src/components/editor/domEditing.ts packages/studio/src/components/editor/domEditing.test.ts packages/studio/src/components/editor/fontAssets.ts packages/studio/src/components/editor/colorValue.ts packages/studio/src/components/editor/colorValue.test.ts packages/studio/src/components/editor/floatingPanel.ts packages/studio/src/components/editor/floatingPanel.test.ts packages/studio/src/components/sidebar/CompositionsTab.tsx packages/studio/src/components/sidebar/CompositionsTab.test.ts packages/studio/src/utils/mediaTypes.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts
  • bunx oxfmt --check packages/core/src/studio-api/routes/thumbnail.ts packages/core/src/studio-api/routes/thumbnail.test.ts packages/studio/src/App.tsx packages/studio/src/components/editor/DomEditOverlay.tsx packages/studio/src/components/editor/PropertyPanel.tsx packages/studio/src/components/editor/domEditing.ts packages/studio/src/components/editor/domEditing.test.ts packages/studio/src/components/editor/fontAssets.ts packages/studio/src/components/editor/colorValue.ts packages/studio/src/components/editor/colorValue.test.ts packages/studio/src/components/editor/floatingPanel.ts packages/studio/src/components/editor/floatingPanel.test.ts packages/studio/src/components/sidebar/CompositionsTab.tsx packages/studio/src/components/sidebar/CompositionsTab.test.ts packages/studio/src/utils/mediaTypes.ts packages/studio/src/utils/sourcePatcher.ts packages/studio/src/utils/sourcePatcher.test.ts
  • bun run --filter @hyperframes/cli dev -- lint /Users/miguel07code/.codex/worktrees/279b/hyperframes-oss/packages/studio/data/projects/dom-edit-playground
  • pre-commit hook: lint, format, typecheck

Manual QA

  • iterated locally against http://localhost:5193/#project/dom-edit-playground
  • verified the inspector/design-panel flow through local screenshots and user testing

Notes

  • local QA artifacts, screenshots, and the ignored packages/studio/data/projects/dom-edit-playground demo fixture are intentionally not part of this PR
  • hyperframes validate could not be run through the CLI dev entrypoint in this worktree because validate.ts expects the built esbuild text-loader import for contrast-audit.browser.js
  • the worktree still has local untracked .superpowers/ and docs/qa/ folders; they were intentionally excluded from the commit

@mintlify
Copy link
Copy Markdown

mintlify Bot commented Apr 24, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
hyperframes 🟢 Ready View Preview Apr 24, 2026, 12:13 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Comment thread packages/studio/src/components/editor/PropertyPanel.tsx Fixed
Comment thread packages/studio/src/components/editor/gradientValue.ts Fixed
Comment thread packages/studio/src/components/editor/gradientValue.ts Fixed
Comment thread packages/studio/src/components/editor/PropertyPanel.tsx Fixed
Copy link
Copy Markdown
Collaborator Author

@miguel-heygen miguel-heygen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Staff-level performance review — PR #466

Reviewed the full 5,946-line diff as well as the actual hot paths (DomEditOverlay RAF loop, domEditing.ts selector resolver, PropertyPanel render tree, font routes, gradient parser). Static analysis only — I did not run a live React Profiler or Chrome perf trace in this environment, so the re-render counts below are reasoned from code, not measured. I'd strongly recommend following up with a Profiler recording on a project with ~200 timeline elements and a long composition + 1 active selection, since that's where the hotspots below compound.

This is a significant and largely well-built feature. The imperative DOM updates during drag (DomEditOverlay.onPointerMove writes directly to box.style and sel.element.style and only commits to React on pointerup) are exactly the right shape for 60fps gesture handling — that part is great.

The concerns are on the idle and incremental-edit paths, where I count at least 3 bugs that re-render the whole App shell every frame or every keystroke, and a few backend/network issues worth fixing before merge.


🔴 Critical

1. DomEditOverlay RAF loop re-queries the DOM and re-renders every 16ms, forever

DomEditOverlay.tsx:112–141 schedules an unconditional rAF loop that runs for the lifetime of the overlay. Every frame it:

  • calls findElementForSelection → which in the non-id path (most cases) runs doc.querySelectorAll(selector) + Array.from + .filter (domEditing.ts:515–533),
  • calls toOverlayRect, which does 3 × getBoundingClientRect() (iframe / overlay / target),
  • calls setOverlayRect(next) with no equality check — so React reconciles the overlay subtree on every frame even when nothing moved.

With a selection active and the user doing nothing, this is a steady 60 setStates/sec + 60 querySelectorAll/sec + ~180 layout reads/sec. During any unrelated React render (toast, resize, panel collapse) the overlay also re-renders.

Fixes, in order of ROI:

  1. Short-circuit when next is within ~0.5px of overlayRectRef.current — skip setOverlayRect entirely.
  2. Replace the rAF pump with a combination of ResizeObserver (on the target + iframe), scroll listener on the iframe's document (passive), and a MutationObserver on the iframe body for attribute/structure changes. Only rAF-coalesce when one of these actually fires.
  3. Cache iframe.getBoundingClientRect() across frames and only re-read on iframe resize.
  4. Fast-path findElementForSelection by id path first (already done) but also cache the resolved HTMLElement in a ref and verify with isConnected before re-walking — only re-query when the element is detached.

2. timelineElements baked into core callback deps causes cascade invalidation

App.tsx:1290–1308 and 1314–1351:

const applyDomSelection = useCallback((selection, ...) => { ... findMatchingTimelineElementId(selection, timelineElements) ... },
  [setSelectedTimelineElementId, timelineElements]);
const buildDomSelectionFromTarget = useCallback((target, ...) => { ... },
  [activeCompPath, compIdToSrc, fileTree, isMasterView, timelineElements]);

timelineElements comes from a Zustand selector (s.elements) — identity changes on every store mutation. That means every edit (clip drag, trim, add, delete) invalidates both callbacks, which invalidates every downstream useCallback/useEffect that depends on them, which re-attaches global listeners and re-runs selection bookkeeping. Read timelineElements through a ref inside the callback, or narrow the dep to timelineElements.length + a timelineElementsMapRef.

3. compIdToSrc Map identity churn re-triggers caption detection on every composition reload

App.tsx:481, 608compIdToSrc is a Map stored in top-level state; NLELayout calls setCompIdToSrc(new Map(...)) on every composition load, and the caption effect at line 608 depends on it. A new Map identity → the caption detection re-scans the iframe DOM on every reload even if contents are unchanged. Either move to a ref + notify only on content delta, or compute a stable hash and early-exit the effect when unchanged.

4. /api/fonts/google has no request timeout and no abort

routes/fonts.ts:203await fetch(GOOGLE_FONTS_METADATA_URL) without AbortController or a Promise.race timeout. On a stalled DNS / slow Google response, the Studio API handler hangs indefinitely and blocks the initial font picker open. Wrap in AbortController with a 3s deadline and fall back to the bundled popular list on timeout.


🟠 High

5. Font picker: linear filter on every keystroke, no virtualization, no deferral

PropertyPanel.tsx:718–722 filters the combined (document + local + Google + system) list per keystroke and .slice(0, 90). options can easily be 1500+. Two fixes:

  • Wrap query with useDeferredValue so React can yield during typing, and
  • Render the result through a virtualized list (or at minimum drop to .slice(0, 30) plus a "see more" affordance) — the current 90-item render is still a wide React subtree per keystroke.
    Longer term: build a first-letter index once and only scan matching buckets.

6. parseGradient called unmemoized on every render of GradientField

PropertyPanel.tsx:1012const parsed = parseGradient(value) ?? buildDefaultGradientModel(fallbackColor);. Every panel re-render re-parses the whole gradient string (regex, stop walk, color parse per stop). Wrap in useMemo([value, fallbackColor]).

7. Google Fonts stylesheet injected on every font-family style commit

App.tsx:1454 and 1592 call injectPreviewGoogleFont(doc, value) on every style commit for font-family. injectPreviewGoogleFont has an idempotency guard by id, but it runs DOM lookup + parse of the family string on every keystroke of the font input. Cache the last injected family in a Set ref per iframe, and only inject on first encounter. Follow up with a cleanup pass (see #14).

8. innerHTML = for text edits reparses and re-attaches children on every keystroke

App.tsx:1509, 1554el.innerHTML = nextContent; inside the text-commit handler. For plain text nodes, use el.textContent = nextContent (~10× faster and avoids HTML parsing, script execution, and image restart). Reserve innerHTML for the structured case and flag it.

9. listInstalledFontFamilies is module-cached forever + blocks the event loop on first call

routes/fonts.ts:168–187 — macOS path uses execFileSync("system_profiler", ...) with a 5s timeout; Linux/Windows path does recursive readdirSync/statSync. The result is cached in cachedFonts for the process lifetime, which is fine for a dev server but (a) the first request blocks the event loop up to several hundred ms, and (b) installing a new font during a session never shows up until restart. Consider:

  • a 5–10 min TTL instead of forever, and
  • wrapping the macOS execFile in its async form and the Linux walk in fs.promises.readdir so the first request doesn't stall.

10. parseRadialArgs allocates 4 RegExp objects per call

gradientValue.ts:156RADIAL_SIZE_KEYWORDS.find((keyword) => new RegExp(\\b${keyword}\b`, "i").test(config)). Hoist to a module-level Map<string, RegExp>` (or a single combined regex). Called on every gradient parse.


🟡 Medium

11. useEffect over-broad deps re-sync preferredFillMode on every color editPropertyPanel.tsx:1479–1481 includes backgroundImage in deps; that string changes on every solid-color edit. Drop it.

12. Object/array literals + inline arrow callbacks defeat the React.memo on child componentsPropertyPanel.tsx passes fresh options={[...]} arrays and onClick={() => ...} closures to SegmentedControl, SelectField, SliderControl, etc. Given PropertyPanel is itself wrapped in memo, those children can't skip renders either. Hoist the static options arrays to module scope; wrap event callbacks in useCallback where the memoized children are expensive (the gradient editor and font picker especially).

13. Interpolation allocates fresh sorted-stops array per callgradientValue.ts:266–272 clones + sorts every stop on every interpolateGradientStopColor invocation. Sort once in parseGradient (or mark the GradientModel as sorted-by-construction) and drop the clone/sort from interpolation.

14. Injected <link> tags are never garbage-collectedloadGoogleFontStylesheet / injectPreviewGoogleFont accumulate <link> elements in the preview head for every font the user briefly selects. Small, but across a long session can reach 10–40 nodes and accompanying network fetches with no corresponding user value. Track an activeFontsRef and remove entries not referenced by any selection on a debounced GC pass.

15. handleTimelineToggleHotkey identity changes per render re-attach the preview keydown listenerApp.tsx:680–691. Memoize with empty deps and read the latest toggleTimelineVisibility via a ref.


🟢 Low / quality

16. DomEditOverlay selection-change via key={selectionKey} (line 290) forces a full subtree unmount/remount whenever the selected element changes. Fine functionally, but means React state in the toolbar (color picker open, etc.) can't survive reselection. Reset explicitly instead of through key if you ever want to preserve UI state across reselects.

17. gestureRef.current is never explicitly cleared on overlay unmount — harmless today (ref is scoped), but add gestureRef.current = null in the rAF cleanup to make unmount-mid-drag deterministic.

18. Numerical precision drift in the gradient round-trip (gradientValue.ts:36–37, 234–252): round(x*100)/100 + parse → serialize → parse means dragging a stop by 0.1% repeatedly will eventually drift. Either use 1-decimal precision end-to-end, or preserve position: null for auto-positioned stops and avoid re-serializing them.

19. docs/contributing/studio-manual-dom-editing.mdx is a great addition — honest about constraints. One nit: mention the RAF-based overlay tracking so contributors know why selection survives iframe scrolls.


Suggested before-merge changes

I'd mark #1, #2, #3, #4, #8 as blockers for a feature that ships to designers on Retina laptops with a long timeline. The rest can land as a follow-up perf pass, but #5, #6, #7 are cheap and high-impact.

What I'd want to see in a Profiler trace

  1. Idle selection @ 60fps — with one element selected, sit idle for 5 seconds. Expect DomEditOverlay to commit ~0 renders. Today it will commit ~300.
  2. Font search typing — type a 10-char query and watch commits on FontFamilyField + PropertyPanel.
  3. Drag a gradient stop — render-by-render, only GradientField should commit; today the full PropertyPanel will commit per pointermove.
  4. Timeline clip trim — should not re-attach global keydown / pointerdown listeners. Today it does (see #2).

Happy to pair on any of these or put up a diff for #1 / #4 / #8 if helpful.

Comment thread packages/studio/src/components/editor/DomEditOverlay.tsx Outdated
Comment thread packages/studio/src/components/editor/domEditing.ts
Comment thread packages/studio/src/components/editor/gradientValue.ts Outdated
Comment thread packages/core/src/studio-api/routes/fonts.ts Outdated
Copy link
Copy Markdown
Collaborator

@jrusso1020 jrusso1020 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: approve — well-architected, tests pass, agent prompt is actionable. Findings are polish, not blockers.

Pulled the branch, installed, ran bun run --filter @hyperframes/studio typecheck (clean), bun run --filter @hyperframes/core typecheck (clean), and the editor test sweep:

src/components/editor/domEditing.test.ts     (17 tests)
src/utils/sourcePatcher.test.ts              (13 tests)
src/components/editor/colorValue.test.ts     (10 tests)
src/components/editor/gradientValue.test.ts  (7 tests)
src/components/sidebar/CompositionsTab.test.ts (6 tests)
src/components/editor/floatingPanel.test.ts  (3 tests)
→ Test Files  6 passed (6) / Tests  56 passed (56)

CI green on 006fd593 incl. regression-shards (all 10), Windows tests + render, CodeQL, perf sweep, Validate docs. Head commit subject is "fix(studio): address manual editing review feedback" so the PR has already absorbed one round.

Agent-perspective UX — buildElementAgentPrompt is the primary agent surface

Read the full prompt builder (domEditing.ts:660-703). The emitted shape, given a #safe-card selection with user instruction "Move this card to the top-left corner":

Move this card to the top-left corner

Composition: dom-edit-playground
Playback time: 00:00:00.000
Source file: index.html
DOM id: safe-card
Selector: #safe-card
Selector index: 0
Tag: <section>
Bounds: x=88, y=82, width=360, height=176
Text: Absolute layer Move and resize me Safe geometry: …

Computed styles:
position: absolute
display: block
top: 82px
left: 88px
width: 360px
height: 176px
font-size: 14px
font-weight: 800
font-family: Inter, …
color: rgb(159, 176, 194)
background-color: rgba(47, 198, 127, 0.24)
border-radius: 28px
transform: none
…

Target HTML:
<section id="safe-card" class="clip" data-start="0" data-duration="6" data-track-index="0">

Make a targeted change to this element only. Preserve the rest of the composition and its timing.

What's right. This is directly actionable for a coding agent:

  • File path + composition path + selector + index. Enough to grep and disambiguate if multiple elements share an id across files.
  • Curated computed styles. CURATED_STYLE_PROPERTIES is a 29-item allowlist — position/size/font/color/bg/radius/transform/z-index. Not a 300-property CSS dump. Signal-dense.
  • Tag snippet is the authored HTML, not the rendered DOM, which means the agent sees class="clip" data-start="0" data-duration="6" — critical for not accidentally mutating timing attrs.
  • Explicit guardrail. "Make a targeted change to this element only" is the anti-scope-creep line. An agent reading this won't refactor adjacent layers.
  • Clipboard + document.execCommand("copy") fallback in App.tsx:1898-1910 handles environments without clipboard-API access. Good.
  • Bounds included lets an agent compute deltas (e.g., "top-left" → left=20, top=20 given current bounds). Concrete math.

Agent-UX improvements worth making before this gets heavy usage.

  1. No schema version stamp. Future-you parsing a cached prompt doesn't know whether fields moved or semantics changed. A leading ## HyperFrames element edit request v1 (and a version: 1 field somewhere) makes the format a stable contract. Same concern I flagged on #480's JSON output — consistent to fix them together.

  2. Computed styles labeled without noting they're the final browser-resolved values. When an agent sees background-color: rgba(47, 198, 127, 0.24), it doesn't know whether that's inline, from a CSS rule, or inherited. If the agent then grep-searches for rgba(47, 198, 127, 0.24) and doesn't find it in the source (because the rule uses rgba(47, 198, 127, .24) — different literal), it wastes a round trip. Separating Inline styles: and Computed styles: into two blocks — or marking each style line with its origin ([inline] / [rule] / [inherited]) — would save the agent from hunting where the override lives. You already collect inlineStyles separately in DomEditSelection (domEditing.ts:66); it's not being passed into the prompt.

  3. Matching CSS selector-rules are missing from the prompt. An agent asked to "make this bubble green" needs to know the selector #safe-card { background: rgba(47, 198, 127, 0.24) } exists in the <style> block. Right now the Target HTML field shows only the open tag; the agent has to re-scan index.html for the matching CSS rule. If you walked the stylesheet and included Matching CSS rules: #safe-card { ... } in the prompt, that's a meaningful round-trip saved.

  4. The guardrail could be more specific. "Make a targeted change to this element only" is good. Adding two more bullets would make it airtight: (a) "Do not modify any other element's data-* attributes or positioning"; (b) "Prefer modifying existing CSS rules or inline styles already present on this element over introducing new rules." The current generic line can be ambiguous when the agent sees a gradient panel and wonders whether to touch the stop-color rule (for #gradient-panel) or a shared .label rule.

  5. Text field serialization handles multi-child text blocks (see serializeDomEditTextFields + test at domEditing.test.ts:390). The agent prompt just concatenates via Text: without the data-hf-text-key attribution that's used for source-patching. For multi-text elements, an agent-friendly format would be Texts: [{key: "child:0:strong", text: "..."}, {key: "child:1:span", text: "..."}] so an agent asked to "change the subheading" knows which key to target. Currently only the concatenated form reaches the prompt.

Architecture — capability model is the right abstraction

The center of gravity is resolveDomEditCapabilities (domEditing.ts:404-473). It gates the inspector operations against what's safely patchable back to source, not what's technically possible on a DOM node. Specifically:

  • canMove requires position: absolute|fixed + px left/top + transform: none. So a transform: rotate(-5deg) on #transform-card correctly opts out of drag-move (preserves authored rotation; avoids the "silently detach" failure mode the PR description calls out).
  • canResize = canMove && pxWidth|pxHeight. Conservative; good.
  • canDetachFromLayout = blockish + no transform + not inline text — that's the path behind "Ask agent to make movable" (the agent prompt for block-ish layout-controlled elements).
  • Composition hosts in master view are canEditStyles: false. Intentional — you drill down to edit inner contents. The PR handles the ambiguous #detail-host-from-master case correctly.

This is a patch-safety-first model, not a DOM-capability-first model. That's the right call for a "direct manipulation that must survive source-round-trip" feature. It's also what makes the "make movable" path explicit instead of silent — users see a button instead of wondering why drag doesn't work.

The source-patching side (sourcePatcher.ts:14-73) rolls a careful inline-style tokenizer that handles quoted strings, HTML entities, and parenthesized values (url(...), gradient(...), calc(...)). Not naive semicolon-split. Tests cover paren-depth tracking. Good.

The fonts.ts route (+247 lines) adds local font discovery + Google Fonts metadata. Not directly visible in the tests I ran — worth a local-OS note: font discovery is OS-dependent and could behave differently on headless Linux vs macOS. If validate / layout ever get run in CI Docker, the available-fonts set changes — callers should tolerate empty lists gracefully.

The htmlBundler.ts + rewriteSubCompPaths.ts changes rewrite asset URLs so background-image: url(...) fills from project assets survive sub-composition inlining. Unit-tested in rewriteSubCompPaths.test.ts (+20 lines of new tests).

Runtime cost outside Studio. The inspector is Studio-package-local. None of domEditing.ts/PropertyPanel.tsx/DomEditOverlay.tsx imports back into @hyperframes/core, @hyperframes/engine, or @hyperframes/producer. Prod renders don't pull any of this. ✓ Tree-shakeable by construction.

State & persistence. Edits flow DOM → PatchOperation[]sourcePatcher.applyPatch → fetch PUT to /api/projects/:id/files/... → preview iframe refresh. So the source file is the source of truth, not the DOM. A refresh-without-save doesn't lose edits because the PUT happens on every change (via debounce presumably; would be worth confirming there's no intermediate "dirty" state). Undo/redo isn't explicitly present in this PR — if a user overshoots a slider, their remedy is the in-app "reset" per-field or git diff. Not blocking for v1; flag for follow-up.

Test coverage assessment

Strong:

  • domEditing.test.ts — 17 tests including buildElementAgentPrompt (asserts "Source file:", "Selector:", "Computed styles:", "Target HTML:" all appear). Capability detection, selection building, text-field serialization, round-trip HTML escaping.
  • sourcePatcher.test.ts — 13 tests including the tricky cases (quoted-style preservation, entity handling, multi-attribute tags).
  • colorValue.test.ts — 10 tests for the color parser/serializer (hex/rgb/hsla round-trip).
  • gradientValue.test.ts — 7 tests for linear/radial/conic/repeating + stops parsing. This is the riskiest module (445 lines of parser); 7 tests is thinner coverage than I'd want for a gradient parser that has to round-trip through source. Flag: add tests for weird whitespace, trailing commas, inline comments, and hsl(...) stops.
  • floatingPanel.test.ts — 3 tests for viewport clamping math.

Missing coverage (non-blocking):

  • fontCatalog.ts + fontAssets.ts (126 + 32 lines total) — the Google Fonts + local-fonts lookup. Platform-dependent so harder to unit-test, but at minimum the JSON-parsing + merge-ordering logic could be covered.
  • PropertyPanel.tsx (2469 lines, no component tests) — the UI integration. A jsdom-level smoke test that renders with a sample selection and asserts the sections present would catch a big class of regressions.
  • DomEditOverlay.tsx (410 lines, no tests) — selection overlay + drag/resize handle math.

The pure logic is well-covered; the React shell isn't. Typical tradeoff for a rapid-feature-development phase.

Non-blocking observations

  1. fonts.ts uses Google Fonts metadata. Worth ensuring the fetch is cached or bundled — auditing performance on large font catalogs would help avoid slow first-paint. Also: network failure modes (offline dev, proxied CI) should degrade gracefully (empty list, not error).

  2. Font family strings in Computed styles: tend to be long (e.g. Inter, "SF Pro Display", ui-sans-serif, system-ui, -apple-system, …). Consider shortening to the first-choice family in the prompt — it preserves intent without eating agent context.

  3. PropertyPanel.tsx at 2469 lines is worth monitoring for extraction. Each feature section (layout / flex / radius / blending / fill / text / typography / selection colors) is a candidate for a subfile. Not urgent — React encourages grouping related state — but if the next round of additions pushes past ~3000 lines, splitting by section will help both reviewability and hot-reload latency.

  4. sourcePatcher.resolveSourceFile's id-first strategy is string-match id="foo" / id='foo'. A file with id="foo-bar" also matches id="foo" via substring if we're not careful — the current code uses content.includes(\id="${elementId}"`)which DOES require the full quoted attr. So false-positives aren't possible from substring collisions likefoovsfoo-bar. However, data-real-id="foo"` WOULD match. Minor; mitigated by file-order iteration preferring earlier matches.

  5. data-hf-text-key is used for child-text round-tripping. Good convention. Worth ensuring this attr is stripped or recognized by lint / validate / layout (the audit tools). If a user ships a composition with data-hf-text-key still present in source, does it cause any downstream issue? Probably not, but worth a one-line lint rule to catch that as a warning in polished output.

  6. CSP / image upload pasting. The PR says users can paste arbitrary image URLs as background-image fills. The bundler rewrites asset URLs (good) but url("https://evil.com/tracker.png") would persist in source as an absolute URL and load at render time. Not a new attack surface per se (users could always do this by hand), but worth noting that the design panel makes it one click. If validate has a "no external-origin media" rule, consider wiring it to fire when the design panel introduces an external URL.

Pattern across today's 5 Miguel PRs (#474 GSAP boundary, #477 nested-comp video seek, #478 WYSIWYG parity, #480 layout audit, #466 this)

Home asked me to call out the throughline. The arc is:

  • #474 / #477 / #478 — three reactive bug-close PRs fixing semantic-awareness gaps where subsystems silently drop information.
  • #480 — offensive agent tooling: surface layout state to agents via structured JSON so we can close gaps proactively.
  • #466 — the human-in-the-loop complement to #480. Where #480 audits the rendered frame, #466 lets a human (or agent via copy-prompt) directly edit the DOM and persist to source.

Reading these as a system, there's a coherent thesis forming: the WYSIWYG gap between preview and render was the root cause, and the strategy is (a) close it where it's silently breaking (#474/#477/#478), (b) expose it to agents for automated catch (#480), (c) expose it to humans for direct-manipulation edit (#466). The lint / validate / layout / inspector progression is becoming a first-class hyperframes-authoring toolchain.

Worth naming the ambition in a follow-up: if #480's JSON and #466's agent-prompt share a schemaVersion and a consistent element-reference format (id + selector + selectorIndex), an agent could chain them — run layout, pick an issue, paste the corresponding inspector prompt as the fix request. That's the integrated-authoring-agent play. Right now the two formats are independent; alignment is low-cost and high-leverage.

CI state

All green on 006fd593: Format / Typecheck / Lint / Test / Build / Test: runtime contract / Smoke: global install / Tests on windows-latest / Render on windows-latest / CodeQL / Perf (fps/scrub/drift/parity/load) / player-perf / regression / Validate docs / regression-shards (all 10 incl. render-compat, hdr, styles-a-g, fast). No blockers from CI.

Ship it.

Review by Rames Jusso

return fields
.map(
(field) =>
`- key=${field.key}; tag=<${field.tagName}>; source=${field.source}; text="${field.value.replace(/"/g, '\\"')}"`,
@miguel-heygen miguel-heygen changed the base branch from main to next April 25, 2026 04:08
@miguel-heygen miguel-heygen force-pushed the feat/studio-manual-dom-editing-v1 branch from 12fa3a4 to a2be8c5 Compare April 25, 2026 04:15
@miguel-heygen miguel-heygen merged commit 38666b4 into next Apr 25, 2026
20 checks passed
@miguel-heygen miguel-heygen deleted the feat/studio-manual-dom-editing-v1 branch April 25, 2026 04:17
miguel-heygen added a commit that referenced this pull request Apr 25, 2026
* fix: stabilize studio preview and runtime sync

* fix: pass selector through timeline thumbnails

* feat: add studio timeline editing

* fix: disambiguate timeline edit targets

* fix: stop timeline auto-scroll in fit mode

* feat: use percentage-based timeline zoom

* fix: sync timeline playhead on zoom changes

* fix: reset timeline scroll when returning to fit

* feat(studio): add manual DOM editing inspector

* docs: update studio manual dom editing guide

* feat(studio): add image asset picker for fills

* feat(studio): add inline image uploads for fills

* fix(studio): use real file input for image fill uploads

* fix(studio): restore toast plumbing after rebase

* fix(studio): explain in-app upload limitation

* fix(studio): reuse asset-tab upload pattern in fills

* feat(studio): refine manual design inspector

* fix(studio): polish manual design inspector

* fix(studio): keep color picker in viewport

* fix(studio): clarify color picker selection

* docs: update manual DOM editing guide

* fix(studio): keep gradient color picker open

* fix(studio): scope text color to text layers

* fix(studio): add agent fallback for immovable layers

* fix(studio): address manual editing review feedback

* fix(studio): make local font selection reliable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants