Major Architecture Refactor & Performance Improvements

[karimElmougi]

First of all, I would like to apologize for the gigantic PR. I could not find a way to split it up without ending in conflict-hell rewriting the history.

This PR represents a comprehensive refactoring of comically, restructuring the project into a workspace with three separate crates, reorganizing the image processing pipeline, and implementing significant performance optimizations. The result is a ~2.6x speedup in processing time.

Architecture Changes

Workspace Restructure

Split the monolithic crate into a workspace with three focused crates:

• comically - Core library with image processing, archive handling, and format conversion logic
• comically-tui - Terminal UI application (the original user interface)
• comically-cli - New command-line interface for automation and scripting

The CLI made performance profiling significantly easier, and this structure would allow for a GUI version of comically to reuse the core logic.

Module Organization

Within the comically library crate, the code is now organized into focused modules:

• image/ - Image processing pipeline split into:
  • decode.rs - Image decoding from various formats
  • encode.rs - Compression and encoding (JPEG/PNG/WebP)
  • transform.rs - Cropping, resizing, splitting, rotation, gamma correction
  • mod.rs - High-level batch processing API
• device.rs - Device presets and dimension handling (extracted from config)
• comic.rs - Core types (ComicFile, ComicConfig, ProcessedImage)
• archive.rs - Archive file reading (CBZ/CBR/ZIP/RAR)
• epub.rs - EPUB generation (now builds in memory)
• cbz.rs - CBZ generation
• mobi.rs - MOBI conversion (via kindlegen)

Performance Optimizations

Image Processing (~2.6x overall speedup)

1. Switched to fast_image_resize (from imageproc::imageops::resize)

   • Uses optimized SIMD algorithms with better cache locality
   • Lanczos3 for downscaling, CatmullRom for upscaling
   • Note: Output is not binary-identical to imageproc::resize, but visual quality is equivalent or better

2. Gamma lookup table

   • Pre-compute gamma correction values in a lookup table
   • Eliminates per-pixel floating-point calculations

3. Zero-copy image transformations

   • Introduced CroppedImage type for zero-copy cropping operations
   • Split double-page spreads without allocating new buffers
   • Only allocate when rotation or resize is required

4. Memory allocation optimizations

   • Use arrayvec for split images (1-3 images) to avoid heap allocation
   • Use Vec::with_capacity throughout to reduce reallocations
   • Reserve capacity for zip file operations based on expected size
   • Reuse 200MB build buffer across comics in batch processing

Parallelism Strategy Refinement

• Removed comic-level parallelism - Process one comic at a time
• Concentrate parallelism at image level - Use all threads for processing images within a comic

The rationale is that, with 100-200 images per manga volume, having multiple volumes in memory simultaneously creates excessive thread contention and memory pressure. I noticed significantly degraded performance when trying to process multiple volumes at the same time.

Archive Processing

• Parallel file reading with Rayon - Read multiple files from archives concurrently
• Collect archive files before processing - Enables accurate progress reporting and better parallelization

In-Memory EPUB Building

Build EPUB archives entirely in memory instead of using temporary files, reducing I/O overhead and improving performance.

Performance Results

Processing all the Naruto volumes (17.5 GiB) on my M2 Macbook Air:

• Upstream: 11m40s
• This PR: 4m32s
• Speedup: ~2.6x faster
• Throughput rate: Over 500 Mbps, getting close to capping out a SATA connection

I noticed it took around 3.5-4.2s per volume.

Output Compatibility

Until the fast_image_resize optimization:

• EPUB conversions produce essentially identical binary output compared to upstream
• Any differences are due to deterministic refactoring (no functional changes)

After the fast_image_resize optimization:

• Images are not binary-identical to imageproc::resize output
• Visual quality is equivalent or superior
• File sizes and compression ratios are similar
• Different resize algorithms produce slightly different pixel values

Files Worthy of Attention

New Crate: comically-cli/src/main.rs (355 lines, new file)

Complete command-line interface for batch processing comics. Includes:

• Argument parsing with clap
• Batch processing with progress reporting
• Configuration file support
• Parallelism controls

Core Processing: comically-tui/src/pipeline.rs (292 lines, new file)

Orchestrates the processing pipeline for the TUI:

• Batch comic processing with progress tracking
• Error handling and reporting
• Integration with comically library
• MOBI conversion coordination (kindlegen)

Progress Reporting: comically-tui/src/tui/progress.rs (139 lines, modified)

Improved progress tracking:

• Accurate file counts (collect files before processing)
• Per-comic progress reporting
• Stage tracking (Process/Package/Convert)
• Error state handling

Core Library: comically/ (entire crate, ~1,500 lines)

Complete rewrite/reorganization of the core processing logic:

comically/src/image/mod.rs (178 lines)

High-level image processing API:

• process_batch() - Parallel batch processing
• process_batch_with_progress() - With progress callbacks
• Split<T> - Stack-allocated container for split images (arrayvec)
• Public API for image formats and compression

comically/src/image/transform.rs (584 lines)

Core image transformations:

• Gamma correction with lookup table
• Auto-cropping with margin detection
• Brightness/contrast adjustments
• Rotation (90° clockwise/counter-clockwise)
• Resizing with fast_image_resize
• Double-page splitting (zero-copy)
• Split strategies (None/Split/Rotate/RotateAndSplit)

comically/src/image/encode.rs (141 lines)

Image compression:

• JPEG encoding with quality control
• PNG encoding with compression levels
• WebP encoding
• Parallel encoding with Rayon

comically/src/image/decode.rs (12 lines)

Simple image decoding wrapper around imageproc.

comically/src/comic.rs (172 lines)

Core types and configuration:

• ComicFile - Represents a comic archive file
• ComicConfig - Processing configuration
• ProcessedImage - Output image data
• SplitStrategy, OutputFormat enums
• Config file persistence

comically/src/device.rs (163 lines)

E-reader device presets:

• 11 device presets (Kindle, Kobo)
• Custom dimensions support
• Device dimension handling

comically/src/epub.rs (288 lines)

In-memory EPUB generation:

• Build EPUB entirely in memory
• XHTML page generation
• Content.opf manifest generation
• No temporary files

comically/src/archive.rs (32 lines, modified)

Archive file reading:

• ZIP/CBZ support
• RAR/CBR support (via unrar)
• Iterator-based API for streaming

comically/src/cbz.rs (33 lines)

CBZ archive generation.

comically/src/mobi.rs (12 lines, modified)

MOBI conversion coordination (delegates to kindlegen).

Dependencies Added

• fast_image_resize (5.0) - High-performance image resizing
• arrayvec (0.7) - Stack-allocated vectors for split images
• num_enum (0.7) - Enum derives for device presets
• parking_lot (0.12) - Better mutex implementation
• strum (0.27) - Enum utilities

Testing

All changes have been tested with:

• Full Naruto manga collection (17.5 GiB, 700+ volumes)
• All output formats (EPUB, MOBI, CBZ)
• All split strategies (None, Split, Rotate, RotateAndSplit)