-
Notifications
You must be signed in to change notification settings - Fork 1
Open
Labels
ai-integrationAI/ML API integrationsAI/ML API integrationsai-video-editorAI Video Editor Flutter appAI Video Editor Flutter appfeatureNew feature implementationNew feature implementationflutterFlutter/Dart implementationFlutter/Dart implementationhigh-priorityHigh priority tasksHigh priority tasksphase-3Phase 3: AI IntelligencePhase 3: AI Intelligence
Description
Summary
Implement an AI agent using the Anthropic Claude API that transforms a high-level story idea into a fully structured scene-by-scene breakdown. This is the intelligent core of the editor, allowing non-technical users to go from concept to production-ready scene plan in minutes.
Requirements
- "Start with Story" mode: user enters a high-level idea (1-5 sentences)
- Storytelling agent flow:
- Claude analyzes the idea and asks clarifying questions (genre, style, length, audience, tone)
- User answers (or skips)
- Claude generates a full scene breakdown
- Scene breakdown output includes for each scene:
- Scene title and number
- Scene description (what happens)
- Visual style notes (lighting, camera angle, color palette)
- Shot type (wide, medium, close-up, extreme close-up)
- Camera movement (pan, tilt, dolly, static, handheld)
- Transition from previous scene (cut, dissolve, fade, wipe)
- Mood and atmosphere
- Suggested video model and parameters
- Suggested duration
- First frame description (for image generation)
- Last frame description
- Voice narration script
- Suggested background music mood
- User can review and edit the breakdown before creating scenes
- "Regenerate scene X" to get alternative for specific scene
- "Expand scene" to split one scene into multiple
- "Merge scenes" to combine adjacent scenes
- Streaming response display (show Claude's thinking in real-time)
- Save story brief as part of project
- Story consistency checker: Claude reviews all scenes for narrative coherence
- Character consistency guidance: suggest reference images for recurring characters
Acceptance Criteria
- User can input a story idea and receive a structured scene breakdown
- Clarifying questions flow works end-to-end with skip option
- Streaming response shows Claude output in real-time
- Individual scenes can be regenerated, expanded, or merged
- Story consistency check flags narrative inconsistencies
Tech Notes
- Use Anthropic Claude API (claude-agent-sdk or direct API)
- Stream responses using SSE for real-time display
- Structure output as JSON for reliable scene parsing
Related
Part of the ai-video-editor Flutter application. Creates scenes fed into #62 (Scene Management), #63 (Scene Editor), and #67 (Voice & Audio).
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
ai-integrationAI/ML API integrationsAI/ML API integrationsai-video-editorAI Video Editor Flutter appAI Video Editor Flutter appfeatureNew feature implementationNew feature implementationflutterFlutter/Dart implementationFlutter/Dart implementationhigh-priorityHigh priority tasksHigh priority tasksphase-3Phase 3: AI IntelligencePhase 3: AI Intelligence