Question about the base model

[nancy6o6]

You have explored a very interesting direction!
I have a few questions regarding the model. Why did you choose to use the DiT architecture? Is your proposed method specifically designed for DiT models, or does the DiT architecture provide some fundamental support for your approach?
If we apply LoRA fine-tuning on the SDXL inpainting model, would it be possible to achieve similar functionality?
I look forward to your response.

[Orannue]

Thank you for your interest!
The DiT architecture naturally provides the T2I models with in-context generation ability, allowing them to generate images conditioned not only on text but also on additional visual inputs. By fine-tuning with minimal data, this ability can be further enhanced for specific tasks.
In contrast, U-Net–based models lack this capability, so it may be difficult to achieve in-context generation through LoRA fine-tuning alone.

You have explored a very interesting direction! I have a few questions regarding the model. Why did you choose to use the DiT architecture? Is your proposed method specifically designed for DiT models, or does the DiT architecture provide some fundamental support for your approach? If we apply LoRA fine-tuning on the SDXL inpainting model, would it be possible to achieve similar functionality? I look forward to your response.