How can I change alpha of LoRA?

[hysts]

I'm not that familiar with the implementation detail of LoRA and wondering how I can change alpha in inference time.
I found a field scale here:

diffusers/src/diffusers/models/cross_attention.py

Line 260 in 0856137

self.scale = 1.0

so I though I can change it by implementing something like tune_lora_scale in the original implementation:
https://github.com/cloneofsimo/lora/blob/e19f6ae22c3fa4d6da252c5b7443fc0942088cd7/lora_diffusion/lora.py#L597-L600
but actually it doesn't seem to be used:

diffusers/src/diffusers/models/cross_attention.py

Line 285 in 0856137

self, attn: CrossAttention, hidden_states, encoder_hidden_states=None, attention_mask=None, scale=1.0

It looks like scale has to be passed as an argument to this module, but how can I do that?

[patrickvonplaten]

You can change the alpha by doing:

unet(sample, timestep, cross_attention_kwargs={"scale"=0.5}

This will change the scale here:

diffusers/src/diffusers/models/cross_attention.py

Line 285 in 0856137

self, attn: CrossAttention, hidden_states, encoder_hidden_states=None, attention_mask=None, scale=1.0

The self.scale attribute can be deleted.
Also we probably should add a cross_attention_kwargs argument to the StableDiffusionPipeline.__call__ function :-)

I'll open a PR for this :-)

[patrickvonplaten]

#2129 should solve it :-)

[hysts]

@patrickvonplaten Thank you for the answer and PR!