Skip to content

Add preview_record: spec, plan, and Slice 1 core primitives#128

Draft
obj-p wants to merge 1 commit intomainfrom
worktree-preview-record-spec
Draft

Add preview_record: spec, plan, and Slice 1 core primitives#128
obj-p wants to merge 1 commit intomainfrom
worktree-preview-record-spec

Conversation

@obj-p
Copy link
Copy Markdown
Owner

@obj-p obj-p commented Apr 20, 2026

Summary

  • Spec (SPEC.md): Full design for 3 new MCP tools (preview_record, preview_record_start, preview_record_stop) targeting animation debugging and stakeholder demo recording. Went through two rounds of adversarial subagent review with prior-art citations (Playwright, Cypress, Maestro, ffmpeg scene-detect, etc.).
  • Plan (tasks/plan.md, tasks/todo.md): 6 vertical slices with 3 human checkpoints. Updated for daemon architecture post-rebase.
  • Slice 1 — Core primitives: Platform-agnostic algorithms in PreviewsCore/Recording/:
    • FrameDiff — SSIM on 128×128 grayscale CGImage (Wang et al. formulation, 8×8 non-overlapping windows)
    • KeyframeSelector — pairwise threshold scene-detect with 80ms min-gap, forced endpoints, greedy highest-diff selection
    • ActionLog — Sendable actor for recording session action timelines, Codable for JSON serialization

Key design decisions in the spec

  • No ScreenCaptureKit — reuses existing Snapshot.capture (bitmapImageRepForCachingDisplay) on a 30fps timer. No TCC permissions, no CI headaches, works headless.
  • Two tools, not onepreview_record (atomic keyframes for agents) vs preview_record_start/stop (session video + action timeline for stakeholders). Different shapes, different outputs.
  • Action timeline is load-bearing — Playwright's trace-action correlation is why it beat Cypress. Session recordings return {path, durationMs, actions: [{tMs, tool, params, causedRecompile}]}.
  • No gating during recording — all tools allowed mid-recording. Recompile cuts are explained by the action timeline. Trust the timeline.
  • Touch overlay indicators — fading circles at tap coordinates, composited via CoreGraphics (keyframes) or AVVideoCompositionCoreAnimationTool (session .mov).

Test plan

  • 21 tests across 3 suites pass (swift test --filter "FrameDiff|KeyframeSelector|ActionLog")
  • swift build clean — no cross-module import issues
  • swift-format lint --strict clean
  • Review spec design decisions
  • Review algorithm correctness (SSIM, scene-detect, min-gap)
  • Approve before proceeding to Slice 2 (macOS preview_record end-to-end)

🤖 Generated with Claude Code

Spec and implementation plan for 3 new MCP tools (preview_record,
preview_record_start, preview_record_stop) targeting animation debugging
and stakeholder demo recording.

Slice 1 delivers the platform-agnostic core algorithms:
- FrameDiff: SSIM on 128×128 grayscale CGImage (Wang et al.)
- KeyframeSelector: pairwise threshold scene-detect with 80ms min-gap
- ActionLog: Sendable actor for recording session action timelines

21 tests across 3 suites, all passing.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@obj-p obj-p marked this pull request as draft April 30, 2026 01:02
@obj-p obj-p force-pushed the worktree-preview-record-spec branch from da14f4d to f521f36 Compare April 30, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant