VeraRetouch: A Lightweight Fully Differentiable Framework for Multi-Task Reasoning Photo Retouching

Yihong Guo¹  Youwei Lyu²  Jiajun Tang²  Yizhuo Zhou¹  Hongliang Wang³  Jinwei Chen²  Changqing Zou^1†  Qingnan Fan²

¹Zhejiang University, ²vivo BlueImage Lab, ³University of Chinese Academy of Sciences

📄 arXiv Paper  

🗓️ To Do List

• Release inference code.
• 🔴 Release model weights.
• 🔴 Release iOS toy deployment.

🚀 Quick Start

⚙️ Environment

# Clone the repository
git clone https://github.com/OpenVeraTeam/VeraRetouch.git
cd VeraRetouch

# Create and activate conda environment
conda create -n vera-retouch python=3.10
conda activate vera-retouch
pip install -r requirements.txt

☕ Pretrained Model

Download our pretrained weights from HuggingFace. You can put the pretrained model to ./checkpoints

🖥️ Inference

Our model supports three inference modes:

• Auto Retouch: Only an image is input.

python inference.py --mode auto \
                    --model-path ./checkpoints/VeraRetouch    # the pretrained model path \
                    --img_paths ./data_samples/input/sample_flower.jpg    # input image paths, multiple paths are supported \
                    --save_dir ./data_samples/output/    # output texts and images save path \
                    --chunk -1    # Enable when GPU memory is insufficient. The renderer will process large images in chunks. Recommended value: 262144 (512*512), enabling chunking will reduce inference speed. \
                    --batch_size 1    # Support batch inference

• Style Retouch: An image and user prompt are input.

python inference.py --mode style \
                    --prompt "I want a dreamy bright pink style."    # style user prompt(only 'style' mode used) \
                    --model-path ./checkpoints/VeraRetouch    # the pretrained model path \
                    --img_paths ./data_samples/input/sample_flower.jpg    # input image paths, multiple paths are supported \
                    --save_dir ./data_samples/output/    # output texts and images save path \
                    --chunk -1    # Enable when GPU memory is insufficient. The renderer will process large images in chunks. Recommended value: 262144 (512*512), enabling chunking will reduce inference speed. \
                    --batch_size 1    # Support batch inference

• Param Retouch: An image and retouching operator parameters are input.

python inference.py --mode style \
                    --instruction_path ./data_samples/param.json    # retourch operator parameters(only 'param' mode used) \
                    --model-path ./checkpoints/VeraRetouch    # the pretrained model path \
                    --img_paths ./data_samples/input/sample_flower.jpg    # input image paths, multiple paths are supported \
                    --save_dir ./data_samples/output/    # output texts and images save path \
                    --chunk -1    # Enable when GPU memory is insufficient. The renderer will process large images in chunks. Recommended value: 262144 (512*512), enabling chunking will reduce inference speed. \
                    --batch_size 1    # Support batch inference

📲 Toy IOS depolyment

Comming soon...