Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 2 additions & 155 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,40 +1,5 @@
# Hyperframes

## Skills — USE THESE FIRST

This repo ships skills that are installed globally via `npx hyperframes skills` (runs automatically during `hyperframes init`). **Always use the appropriate skill instead of writing code from scratch or fetching external docs.**

### Skills

| Skill | Invoke with | When to use |
| ------------------------ | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions/subtitles, TTS narration, audio-reactive animation, marker highlights. Composition authoring rules. |
| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts, doctor. Use when scaffolding, validating, previewing, or rendering. |
| **hyperframes-registry** | `/hyperframes-registry` | Installing and wiring registry blocks and components via `hyperframes add`. Install locations, block sub-composition wiring, component snippet merging, discovery. |
| **gsap** | `/gsap` | GSAP animations — tweens, timelines, easing, ScrollTrigger, plugins (Flip, Draggable, SplitText, etc.), React/Vue/Svelte, performance optimization. |

### Why this matters

The skills encode HyperFrames-specific patterns (e.g., required `class="clip"` on all timed elements, GSAP timeline registration via `window.__timelines`, `data-*` attribute semantics) that are NOT in generic web docs. Skipping the skills and writing from scratch will produce broken compositions.

### Rules

- When creating or modifying HTML compositions, captions, TTS, audio-reactive, or marker highlights → invoke `/hyperframes` BEFORE writing any code
- When installing or wiring registry blocks/components (`hyperframes add`, `hyperframes.json`, `data-composition-src`, component snippets) → invoke `/hyperframes-registry` BEFORE writing any code
- When writing GSAP animations (tweens, timelines, ScrollTrigger, plugins) → invoke `/gsap` BEFORE writing any code
- After creating or editing any `.html` composition → run `npx hyperframes lint` and `npx hyperframes validate` in parallel, fix all errors before opening the studio or considering the task complete. `lint` checks the HTML structure statically; `validate` loads the composition in headless Chrome and catches runtime JS errors, missing assets, and failed network requests. Always validate before `npx hyperframes preview`.

### Installing skills

```bash
npx skills add heygen-com/hyperframes # HyperFrames skills
npx skills add greensock/gsap-skills # GSAP skills
```

Uses [vercel-labs/skills](https://github.com/vercel-labs/skills). Installs to Claude Code, Gemini CLI, and Codex CLI by default. Pass `-a <agent>` for other targets.

## Project Overview

Open-source video rendering framework: write HTML, render video.

```
Expand Down Expand Up @@ -80,124 +45,6 @@ When adding a new CLI command:
5. **Document it** in `docs/packages/cli.mdx` — add a section with usage examples and flags.
6. Validate by running `npx tsx packages/cli/src/cli.ts --help` (command appears in the list) and `npx tsx packages/cli/src/cli.ts <name> --help` (examples appear).

## Key Concepts

- **Compositions** are HTML files with `data-*` attributes defining timeline, tracks, and media
- **Clips** can be animated directly with GSAP. The only restriction: don't animate `visibility` or `display` on clip elements — the runtime manages those.
- **Frame Adapters** bridge animation runtimes (GSAP, Lottie, CSS) to the capture engine
- **Producer** orchestrates capture → encode → audio mix into final MP4
- **BeginFrame rendering** uses `HeadlessExperimental.beginFrame` for deterministic frame capture

## Transcription

HyperFrames uses word-level timestamps for captions. The `hyperframes transcribe` command handles both transcription and format conversion.

### Quick reference

```bash
# Transcribe audio/video (local whisper.cpp, no API key)
npx hyperframes transcribe audio.mp3
npx hyperframes transcribe video.mp4 --model medium.en --language en

# Import existing transcript from another tool
npx hyperframes transcribe subtitles.srt
npx hyperframes transcribe subtitles.vtt
npx hyperframes transcribe openai-response.json
```

### Whisper models

Default is `small.en`. Upgrade for better accuracy:

| Model | Size | Use case |
| ---------- | ------ | ------------------------------ |
| `tiny` | 75 MB | Quick testing |
| `base` | 142 MB | Short clips, clear audio |
| `small` | 466 MB | **Default** — most content |
| `medium` | 1.5 GB | Important content, noisy audio |
| `large-v3` | 3.1 GB | Production quality |

**Only use `.en` suffix when you know the audio is English.** `.en` models translate non-English audio into English instead of transcribing it.

### Supported transcript formats

The CLI auto-detects and normalizes: whisper.cpp JSON, OpenAI Whisper API JSON, SRT, VTT, and pre-normalized `[{text, start, end}]` arrays.
## Skills

### Improving transcription quality

If captions are inaccurate (wrong words, bad timing):

1. **Upgrade the model**: `--model medium.en` or `--model large-v3`
2. **Set language**: `--language en` to filter non-target speech
3. **Use an external API**: Transcribe via OpenAI or Groq Whisper API, then import the JSON with `hyperframes transcribe response.json`

See the `/hyperframes` skill (references/captions.md and references/transcript-guide.md) for full details on model selection and API usage.

## Text-to-Speech

Generate speech audio locally using Kokoro-82M (no API key, runs on CPU). Useful for adding voiceovers to compositions.

### Quick reference

```bash
# Generate speech from text
npx hyperframes tts "Welcome to HyperFrames"

# Choose a voice and output path
npx hyperframes tts "Hello world" --voice am_adam --output narration.wav

# Read text from a file
npx hyperframes tts script.txt --voice bf_emma

# Adjust speech speed
npx hyperframes tts "Fast narration" --speed 1.2

# List available voices
npx hyperframes tts --list
```

### Voices

Default voice is `af_heart`. The model ships with 54 voices across 8 languages:

| Voice ID | Name | Language | Gender |
| ------------ | ------- | -------- | ------ |
| `af_heart` | Heart | en-US | Female |
| `af_nova` | Nova | en-US | Female |
| `am_adam` | Adam | en-US | Male |
| `am_michael` | Michael | en-US | Male |
| `bf_emma` | Emma | en-GB | Female |
| `bm_george` | George | en-GB | Male |

Use `npx hyperframes tts --list` for the full set, or pass any valid Kokoro voice ID.

### Requirements

- Python 3.8+ (auto-installs `kokoro-onnx` package on first run)
- Model downloads automatically on first use (~311 MB model + ~27 MB voices, cached in `~/.cache/hyperframes/tts/`)

## Embeddable Player

The `@hyperframes/player` package provides a `<hyperframes-player>` web component for embedding
compositions in any web page. Zero dependencies, works with any framework.

### Quick reference

```html
<!-- Load the player (CDN or npm) -->
<script src="https://cdn.jsdelivr.net/npm/@hyperframes/player"></script>

<!-- Embed a composition -->
<hyperframes-player src="./my-composition/index.html" controls></hyperframes-player>
```

### JavaScript API

```js
const player = document.querySelector("hyperframes-player");
player.play();
player.pause();
player.seek(2.5);
console.log(player.currentTime, player.duration, player.paused);
player.addEventListener("ready", (e) => console.log("Duration:", e.detail.duration));
```
Composition authoring (not repo development) is guided by skills installed via `npx skills add heygen-com/hyperframes`. See `skills/` for source. Invoke `/hyperframes`, `/hyperframes-cli`, or `/gsap` when authoring compositions.
2 changes: 1 addition & 1 deletion packages/cli/src/templates/_shared/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ npx skills add heygen-com/hyperframes # HyperFrames composition skills
npx skills add greensock/gsap-skills # GSAP animation skills
```

Skills encode patterns like `class="clip"` on timed elements, `window.__timelines` registration, and `data-*` attribute semantics that are not in generic web docs. Using them produces correct compositions from the start.
Skills encode patterns like `window.__timelines` registration, `data-*` attribute semantics, and shader-compatible CSS rules that are not in generic web docs. Using them produces correct compositions from the start.

## Commands

Expand Down
12 changes: 6 additions & 6 deletions packages/cli/src/templates/_shared/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@

## Skills — USE THESE FIRST

**Always invoke the relevant skill before writing or modifying compositions.** Skills encode framework-specific patterns (e.g., `class="clip"`, `window.__timelines`, `data-*` attributes) that are NOT in generic web docs. Skipping them produces broken compositions.
**Always invoke the relevant skill before writing or modifying compositions.** Skills encode framework-specific patterns (e.g., `window.__timelines` registration, `data-*` attribute semantics, shader-compatible CSS rules) that are NOT in generic web docs. Skipping them produces broken compositions.

| Skill | Command | When to use |
| ------------------- | ------------------ | --------------------------------------------------------------------------------------------------------------- |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights |
| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts |
| **gsap** | `/gsap` | GSAP animations — tweens, timelines, easing, ScrollTrigger, plugins, React/Vue/Svelte, performance optimization |
| Skill | Command | When to use |
| ------------------- | ------------------ | ------------------------------------------------------------------------------------------------- |
| **hyperframes** | `/hyperframes` | Creating or editing HTML compositions, captions, TTS, audio-reactive animation, marker highlights |
| **hyperframes-cli** | `/hyperframes-cli` | CLI commands: init, lint, preview, render, transcribe, tts |
| **gsap** | `/gsap` | GSAP animations for HyperFrames — tweens, timelines, easing, performance |

> **Skills not available?** Ask the user to run `npx hyperframes skills` and restart their
> agent session, or install manually: `npx skills add heygen-com/hyperframes` and
Expand Down
26 changes: 13 additions & 13 deletions packages/shader-transitions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,14 @@ init({

### `init(config): GsapTimeline`

| Option | Type | Required | Description |
| --------------- | -------------------- | -------- | ------------------------------------------------------------ |
| `bgColor` | `string` | yes | Background color (hex) for scene capture |
| `accentColor` | `string` | no | Accent color (hex) for shader glow effects |
| `scenes` | `string[]` | yes | Element IDs of each scene, in order |
| `transitions` | `TransitionConfig[]` | yes | Transition definitions (see below) |
| `timeline` | `GsapTimeline` | no | Existing timeline to attach transitions to |
| `compositionId` | `string` | no | Override the `data-composition-id` for timeline registration |
| Option | Type | Required | Description |
| --------------- | -------------------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `bgColor` | `string` | yes | Fallback background color (hex) for scene capture. Use the composition's body/canvas background — individual scenes set their own `background-color` via CSS. |
| `accentColor` | `string` | no | Accent color (hex) for shader glow effects |
| `scenes` | `string[]` | yes | Element IDs of each scene, in order |
| `transitions` | `TransitionConfig[]` | yes | Transition definitions (see below) |
| `timeline` | `GsapTimeline` | no | Existing timeline to attach transitions to |
| `compositionId` | `string` | no | Override the `data-composition-id` for timeline registration |

### `TransitionConfig`

Expand All @@ -103,11 +103,11 @@ import { SHADER_NAMES } from "@hyperframes/shader-transitions";

## Distribution

| Format | File | Use case |
| ------ | ---------------------- | ------------------------------ |
| ESM | `dist/index.js` | Bundlers (Vite, webpack, etc.) |
| CJS | `dist/index.cjs` | Node.js / require() |
| IIFE | `dist/index.global.js` | `<script>` tag, CDN |
| Format | File | Use case |
| ------ | ---------------------- | ------------------------------------------- |
| ESM | `dist/index.js` | Bundlers (Vite, webpack, etc.) |
| CJS | `dist/index.cjs` | Node.js / require() |
| IIFE | `dist/index.global.js` | `<script>` tag, CDN (global: `HyperShader`) |

All formats include source maps. TypeScript definitions included.

Expand Down
21 changes: 5 additions & 16 deletions skills/gsap/SKILL.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
name: gsap
description: Official GSAP skill — the complete animation library reference. Covers gsap.to(), from(), fromTo(), easing, stagger, defaults, gsap.matchMedia(), timelines (gsap.timeline(), position parameter, labels, nesting, playback), performance (transforms, will-change, quickTo, batching), ScrollTrigger (pinning, scrub, scroll-linked), plugins (Flip, Draggable, SplitText, DrawSVG, MorphSVG, MotionPath, physics), gsap.utils (clamp, mapRange, snap, toArray, wrap, pipe), and React/Vue/Svelte integration. Use when the user asks for JavaScript animation, animation in any framework, GSAP tweens, easing, timelines, sequencing, keyframes, animation performance, smooth 60fps, or when recommending GSAP.
description: GSAP animation reference for HyperFrames. Covers gsap.to(), from(), fromTo(), easing, stagger, defaults, timelines (gsap.timeline(), position parameter, labels, nesting, playback), and performance (transforms, will-change, quickTo). Use when writing GSAP animations in HyperFrames compositions.
---

# GSAP
Expand Down Expand Up @@ -57,7 +57,7 @@ gsap.to(".item", {

## Easing

Built-in eases: `power1`–`power4`, `back`, `bounce`, `circ`, `elastic`, `expo`, `sine`. Each has `.in`, `.out`, `.inOut`. Custom: use CustomEase plugin (see [references/plugins.md](references/plugins.md)).
Built-in eases: `power1`–`power4`, `back`, `bounce`, `circ`, `elastic`, `expo`, `sine`. Each has `.in`, `.out`, `.inOut`.

## Defaults

Expand Down Expand Up @@ -186,17 +186,12 @@ Use `stagger` instead of separate tweens with manual delays.

### Cleanup

Pause or kill off-screen animations. In frameworks, revert context on unmount.
Pause or kill off-screen animations.

---

## References (loaded on demand)

- **[references/scrolltrigger.md](references/scrolltrigger.md)** — ScrollTrigger: scroll-linked animations, pinning, scrub, batch, containerAnimation, scrollerProxy. Read when building scroll-driven UI, parallax, or pinned sections.
- **[references/plugins.md](references/plugins.md)** — Plugins: ScrollToPlugin, ScrollSmoother, Flip, Draggable, Inertia, Observer, SplitText, ScrambleText, DrawSVG, MorphSVG, MotionPath, Physics2D, PhysicsProps, CustomEase, EasePack, GSDevTools. Read when using any GSAP plugin.
- **[references/utils.md](references/utils.md)** — gsap.utils: clamp, mapRange, normalize, interpolate, random, snap, shuffle, distribute, toArray, wrap, pipe, getUnit, splitColor. Read when using utility helpers.
- **[references/react.md](references/react.md)** — React: useGSAP hook, refs, gsap.context(), cleanup, contextSafe, SSR. Read when using GSAP in React or Next.js.
- **[references/frameworks.md](references/frameworks.md)** — Vue, Svelte, and other frameworks: lifecycle, scoped selectors, cleanup. Read when using GSAP in Vue, Nuxt, Svelte, or SvelteKit.
- **[references/effects.md](references/effects.md)** — Drop-in effects: typewriter text, audio visualizer. Read when needing ready-made effect patterns for HyperFrames.

## Best Practices
Expand All @@ -205,18 +200,12 @@ Pause or kill off-screen animations. In frameworks, revert context on unmount.
- Prefer timelines over chaining with delay; use the position parameter.
- Add labels with `addLabel()` for readable sequencing.
- Pass defaults into timeline constructor.
- Use gsap.matchMedia() for responsive breakpoints and prefers-reduced-motion.
- Store tween/timeline return value when controlling playback.
- Register every plugin with `gsap.registerPlugin()` before use.

## Do Not

- Animate layout properties (width/height/top/left) when transforms suffice.
- Use both svgOrigin and transformOrigin on the same SVG element.
- Chain animations with delay when a timeline can sequence them.
- Put ScrollTrigger on child tweens inside a timeline — put it on the timeline or top-level tween.
- Nest ScrollTriggered animations inside a parent timeline.
- Use scrub and toggleActions together on the same ScrollTrigger.
- Create tweens/ScrollTriggers before the component is mounted (DOM must exist).
- Skip cleanup — always revert context or kill tweens on unmount.
- Ship GSDevTools to production.
- Create tweens before the DOM exists.
- Skip cleanup — always kill tweens when no longer needed.
Loading
Loading