Skip to content

[Feature] AI Storytelling Agent - Claude-Powered Story-to-Scene Breakdown #68

@vzeman

Description

@vzeman

Summary

Implement an AI agent using the Anthropic Claude API that transforms a high-level story idea into a fully structured scene-by-scene breakdown. This is the intelligent core of the editor, allowing non-technical users to go from concept to production-ready scene plan in minutes.

Requirements

  • "Start with Story" mode: user enters a high-level idea (1-5 sentences)
  • Storytelling agent flow:
    1. Claude analyzes the idea and asks clarifying questions (genre, style, length, audience, tone)
    2. User answers (or skips)
    3. Claude generates a full scene breakdown
  • Scene breakdown output includes for each scene:
    • Scene title and number
    • Scene description (what happens)
    • Visual style notes (lighting, camera angle, color palette)
    • Shot type (wide, medium, close-up, extreme close-up)
    • Camera movement (pan, tilt, dolly, static, handheld)
    • Transition from previous scene (cut, dissolve, fade, wipe)
    • Mood and atmosphere
    • Suggested video model and parameters
    • Suggested duration
    • First frame description (for image generation)
    • Last frame description
    • Voice narration script
    • Suggested background music mood
  • User can review and edit the breakdown before creating scenes
  • "Regenerate scene X" to get alternative for specific scene
  • "Expand scene" to split one scene into multiple
  • "Merge scenes" to combine adjacent scenes
  • Streaming response display (show Claude's thinking in real-time)
  • Save story brief as part of project
  • Story consistency checker: Claude reviews all scenes for narrative coherence
  • Character consistency guidance: suggest reference images for recurring characters

Acceptance Criteria

  • User can input a story idea and receive a structured scene breakdown
  • Clarifying questions flow works end-to-end with skip option
  • Streaming response shows Claude output in real-time
  • Individual scenes can be regenerated, expanded, or merged
  • Story consistency check flags narrative inconsistencies

Tech Notes

  • Use Anthropic Claude API (claude-agent-sdk or direct API)
  • Stream responses using SSE for real-time display
  • Structure output as JSON for reliable scene parsing

Related

Part of the ai-video-editor Flutter application. Creates scenes fed into #62 (Scene Management), #63 (Scene Editor), and #67 (Voice & Audio).

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-integrationAI/ML API integrationsai-video-editorAI Video Editor Flutter appfeatureNew feature implementationflutterFlutter/Dart implementationhigh-priorityHigh priority tasksphase-3Phase 3: AI Intelligence

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions