Skip to content
This repository was archived by the owner on Feb 7, 2025. It is now read-only.
This repository was archived by the owner on Feb 7, 2025. It is now read-only.

Add tutorial for ControlNets #362

@Warvito

Description

@Warvito

Add tutorial training a diffusion model on brain 2D images from the BRATS dataset. To obtain the 2D images, we could use the data tranformations from the anomaly detections examples. After having a model trained, train a controlnet that learns how to generate a brain given the bain mask. In this case, I guess it will be necessary just to fine tuning the controlnet paramenters (it might not be necessary to change the original diffusion model).
Finally, evaluate the performance of the model by generating the brains using the masks from the the validation set.

An possible extension, is show the use multicontrolnets and finetune a second network to generate brains given tumours. Then, in the evaluation combine the 2 controlnets (brain mask + tumour mask) to generate an image.

Metadata

Metadata

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions