[EPIC] AI Video Editor - Scene-by-Scene AI-Powered Video Generation Platform

[vzeman]

Vision

The AI Video Editor is a Dart/Flutter desktop and mobile application that reimagines video production by placing AI at the center of every creative decision. Instead of assembling pre-recorded footage, users compose videos scene by scene — each clip is generated from scratch by state-of-the-art AI video models. A built-in Claude-powered storytelling agent can transform a single sentence idea into a complete, production-ready scene breakdown in seconds. The result is a professional video creation platform accessible to anyone with an idea, regardless of technical or cinematographic expertise.

The application lives in the ai-video-editor folder of this repository and targets macOS, Windows, iOS, and Android via Flutter.

---

Architecture Overview

┌─────────────────────────────────────────────────────────────────┐
│                        Flutter App                               │
│  ┌──────────────┐  ┌──────────────────┐  ┌───────────────────┐  │
│  │   Project &  │  │   Scene Editor   │  │  Video Player &   │  │
│  │  Timeline UI │  │ (Prompt, Params) │  │  Preview Engine   │  │
│  └──────┬───────┘  └────────┬─────────┘  └─────────┬─────────┘  │
│         │                   │                       │            │
│  ┌──────▼───────────────────▼───────────────────────▼─────────┐  │
│  │              Application State (Riverpod / BLoC)            │  │
│  └──────┬───────────────────┬───────────────────────┬─────────┘  │
│         │                   │                       │            │
│  ┌──────▼───────┐  ┌────────▼──────────┐  ┌────────▼─────────┐  │
│  │  Replicate   │  │   Anthropic Claude │  │   ElevenLabs     │  │
│  │  API Client  │  │    API Client      │  │   API Client     │  │
│  │ (Video + Img)│  │ (Agent + Vision)   │  │   (TTS + Clone)  │  │
│  └──────────────┘  └───────────────────┘  └──────────────────┘  │
│                                                                   │
│  ┌─────────────────────────────────────────────────────────────┐  │
│  │        Local Storage: SQLite + path_provider + Keychain     │  │
│  └─────────────────────────────────────────────────────────────┘  │
│                                                                   │
│  ┌─────────────────────────────────────────────────────────────┐  │
│  │             FFmpeg Engine (ffmpeg_kit_flutter)               │  │
│  │         Concatenation · Transitions · Export Encoding        │  │
│  └─────────────────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────────────────┘

---

Tech Stack

Layer	Technology
Frontend	Flutter / Dart (desktop + mobile)
AI Video Generation	Replicate API (Kling, Wan 2.1, SVD, LTX, Runway, and more)
AI Image Generation	Replicate API (FLUX, SDXL, Ideogram, Recraft)
Voice Synthesis	ElevenLabs API (TTS, voice cloning, multilingual)
AI Storytelling & Quality	Anthropic Claude API (claude-agent-sdk)
Video Processing	FFmpeg via ffmpeg_kit_flutter
Local Storage	SQLite (sqflite) + path_provider
Secure Storage	flutter_secure_storage (OS Keychain backed)
State Management	Riverpod or BLoC
Project File Format	.aivp (JSON + assets in zip archive)

---

Getting Started

The Flutter application source code is located in the ai-video-editor/ folder of this repository.

# Navigate to the app directory
cd ai-video-editor

# Install Flutter dependencies
flutter pub get

# Run on macOS (recommended for development)
flutter run -d macos

# Run on iOS simulator
flutter run -d ios

# Build macOS release
flutter build macos --release

Before running, configure your API keys in the app Settings screen:

• Replicate API Key: required for video and image generation
• ElevenLabs API Key: required for voice narration
• Anthropic API Key: required for the storytelling agent and prompt enhancement

---

Sub-Issues Checklist

Phase 1: Core Infrastructure

• [Feature] UI/UX Design - Core App Layout, Navigation, and Design System #79 - [Feature] UI/UX Design - Core App Layout, Navigation, and Design System
• [Feature] Project Management - Create, Save, Open, and Organize Video Projects #61 - [Feature] Project Management - Create, Save, Open, and Organize Video Projects
• [Feature] Scene Management - Timeline, Ordering, and Scene Configuration #62 - [Feature] Scene Management - Timeline, Ordering, and Scene Configuration
• [Feature] Video Player & Preview - In-App Video Playback and Scene Preview #69 - [Feature] Video Player & Preview - In-App Video Playback and Scene Preview
• [Feature] Asset Library - Centralized Media Management for Projects #73 - [Feature] Asset Library - Centralized Media Management for Projects
• [Feature] Settings & Preferences - App-Wide Configuration and User Preferences #76 - [Feature] Settings & Preferences - App-Wide Configuration and User Preferences

Phase 2: Generation Engine

• [Feature] Cost Tracking & API Key Management - Monitor Spending Across All AI Services #75 - [Feature] Cost Tracking & API Key Management - Monitor Spending Across All AI Services
• [Feature] AI Video Generation - Replicate API Integration with All Supported Models #65 - [Feature] AI Video Generation - Replicate API Integration with All Supported Models
• [Feature] Text-to-Image Generation - Generate Keyframe Images via Replicate #66 - [Feature] Text-to-Image Generation - Generate Keyframe Images via Replicate
• [Feature] Scene Editor - Prompt Configuration and AI Model Parameters #63 - [Feature] Scene Editor - Prompt Configuration and AI Model Parameters
• [Feature] First & Last Frame Control - Keyframe Images for Scene Boundaries #64 - [Feature] First & Last Frame Control - Keyframe Images for Scene Boundaries
• [Feature] Voice & Audio Generation - ElevenLabs Integration and Audio Track Management #67 - [Feature] Voice & Audio Generation - ElevenLabs Integration and Audio Track Management

Phase 3: AI Intelligence

• [Feature] AI Storytelling Agent - Claude-Powered Story-to-Scene Breakdown #68 - [Feature] AI Storytelling Agent - Claude-Powered Story-to-Scene Breakdown
• [Feature] Iterative Generation Workflow - Step-by-Step and Auto-Run Modes #70 - [Feature] Iterative Generation Workflow - Step-by-Step and Auto-Run Modes

Phase 4: Polish & Export

• [Feature] Scene Continuation & Video Concatenation - Seamless Multi-Clip Assembly #71 - [Feature] Scene Continuation & Video Concatenation - Seamless Multi-Clip Assembly
• [Feature] Final Video Rendering & Export - Compose and Export the Complete Video #72 - [Feature] Final Video Rendering & Export - Compose and Export the Complete Video
• [Feature] Template System - Predefined Scene Templates and Project Templates #74 - [Feature] Template System - Predefined Scene Templates and Project Templates
• [Feature] Subtitle & Caption Generation - Auto-Generate and Edit Captions #78 - [Feature] Subtitle & Caption Generation - Auto-Generate and Edit Captions

Phase 5: Collaboration & Advanced

• [Feature] Undo/Redo & Version History - Non-Destructive Editing with Full History #77 - [Feature] Undo/Redo & Version History - Non-Destructive Editing with Full History
• [Feature] Cost Tracking & API Key Management - Monitor Spending Across All AI Services #75 - [Feature] Cost Tracking & API Key Management - Monitor Spending Across All AI Services
• [Feature] Collaboration & Sharing - Share Projects and Export for Review #80 - [Feature] Collaboration & Sharing - Share Projects and Export for Review

---

Milestone / Phase Breakdown

Phase 1: Core Infrastructure

Goal: A working shell of the app with project management, scene timeline, video playback, and settings. No AI generation yet — users can create projects, add scenes, and organize them.

Issues: #79, #61, #62, #69, #73, #76

---

Phase 2: Generation Engine

Goal: Full AI generation pipeline. Users can generate video clips, images, and voice narration from scene prompts. API keys are managed securely and costs are tracked.

Issues: #75, #65, #66, #63, #64, #67

---

Phase 3: AI Intelligence

Goal: Claude-powered storytelling agent transforms ideas into scene breakdowns. Iterative and auto-run generation workflows with AI quality checking.

Issues: #68, #70

---

Phase 4: Polish & Export

Goal: Seamless multi-clip assembly, transitions, final video export in all formats, template library, and subtitle/caption system.

Issues: #71, #72, #74, #78

---

Phase 5: Collaboration & Advanced

Goal: Version history, cost dashboard, and collaboration/sharing features. The app is feature-complete for professional use.

Issues: #77, #75, #80

---

Suggested Labels

• epic - This tracking issue
• ai-video-editor - All issues in this project
• flutter - Flutter/Dart implementation work
• ai-integration - Issues involving Replicate, ElevenLabs, or Anthropic APIs
• phase-1 through phase-5 - Phase assignment
• high-priority - Must-have for MVP
• enhancement - Feature requests and improvements

---

Notes

• All generated video and image files remain local — no cloud storage is used by the app itself
• API keys are never stored in plaintext; OS keychain is always used
• The .aivp project format is a zip archive containing project.json and an assets/ folder
• FFmpeg is required for video concatenation and export; ffmpeg_kit_flutter bundles it for all platforms