Zimage lora support issue

[christopher5106]

Describe the bug

Error message:

lora_conversion_utils.py", line 2566, in _convert_non_diffusers_z_image_lora_to_diffusers
    raise ValueError(f"`state_dict` should be empty at this point but has {state_dict.keys()=}")
ValueError: `state_dict` should be empty at this point but has state_dict.keys()=dict_keys(['context.refiner.0.attention.to_out.0.alpha', 'context.refiner.0.attention.out.lora.down.weight', 'context.refiner.0.attention.out.lora.up.weight', 'context.refiner.0.attention.qkv.alpha', 'context.refiner.0.attention.qkv.lora.down.weight', 'context.refiner.0.attention.qkv.lora.up.weight', 'context.refiner.0.attention.to_k.alpha', 'context.refiner.0.attention.to.k.lora.down.weight', 'context.refiner.0.attention.to.k.lora.up.weight', 'context.refiner.0.attention.to_out.0.lora.down.weight', 'context.refiner.0.attention.to_out.0.lora.up.weight', 'context.refiner.0.attention.to_q.alpha', 'context.refiner.0.attention.to.q.lora.down.weight', 'context.refiner.0.attention.to.q.lora.up.weight', 'context.refiner.0.attention.to_v.alpha', 'context.refiner.0.attention.to.v.lora.down.weight', 'context.refiner.0.attention.to.v.lora.up.weight', 'context.refiner.1.attention.to_out.0.alpha', 'context.refiner.1.attention.out.lora.down.weight', 'context.refiner.1.attention.out.lora.up.weight', 'context.refiner.1.attention.qkv.alpha', 'context.refiner.1.attention.qkv.lora.down.weight', 'context.refiner.1.attention.qkv.lora.up.weight', 'context.refiner.1.attention.to_k.alpha', 'context.refiner.1.attention.to.k.lora.down.weight', 'context.refiner.1.attention.to.k.lora.up.weight', 'context.refiner.1.attention.to_out.0.lora.down.weight', 'context.refiner.1.attention.to_out.0.lora.up.weight', 'context.refiner.1.attention.to_q.alpha', 'context.refiner.1.attention.to.q.lora.down.weight', 'context.refiner.1.attention.to.q.lora.up.weight', 'context.refiner.1.attention.to_v.alpha', 'context.refiner.1.attention.to.v.lora.down.weight', 'context.refiner.1.attention.to.v.lora.up.weight', 'layers.0.attention.to_out.0.alpha', 'layers.0.attention.out.lora.down.weight', 'layers.0.attention.out.lora.up.weight', 'layers.0.attention.qkv.alpha', 'layers.0.attention.qkv.lora.down.weight', 'layers.0.attention.qkv.lora.up.weight', 'layers.0.attention.to_k.alpha', 'layers.0.attention.to.k.lora.down.weight', 'layers.0.attention.to.k.lora.up.weight', 'layers.0.attention.to_out.0.lora.down.weight', 'layers.0.attention.to_out.0.lora.up.weight', 'layers.0.attention.to_q.alpha', 'layers.0.attention.to.q.lora.down.weight', 'layers.0.attention.to.q.lora.up.weight', 'layers.0.attention.to_v.alpha', 'layers.0.attention.to.v.lora.down.weight', 'layers.0.attention.to.v.lora.up.weight', 'layers.10.attention.to_out.0.alpha', 'layers.10.attention.out.lora.down.weight', 'layers.10.attention.out.lora.up.weight', 'layers.10.attention.qkv.alpha', 'layers.10.attention.qkv.lora.down.weight', 'layers.10.attention.qkv.lora.up.weight', 'layers.10.attention.to_k.alpha', 'layers.10.attention.to.k.lora.down.weight', 'layers.10.attention.to.k.lora.up.weight', 'layers.10.attention.to_out.0.lora.down.weight', 'layers.10.attention.to_out.0.lora.up.weight', 'layers.10.attention.to_q.alpha', 'layers.10.attention.to.q.lora.down.weight', 'layers.10.attention.to.q.lora.up.weight', 'layers.10.attention.to_v.alpha', 'layers.10.attention.to.v.lora.down.weight', 'layers.10.attention.to.v.lora.up.weight', 'layers.11.attention.to_out.0.alpha', 'layers.11.attention.out.lora.down.weight', 'layers.11.attention.out.lora.up.weight', 'layers.11.attention.qkv.alpha', 'layers.11.attention.qkv.lora.down.weight', 'layers.11.attention.qkv.lora.up.weight', 'layers.11.attention.to_k.alpha', 'layers.11.attention.to.k.lora.down.weight', 'layers.11.attention.to.k.lora.up.weight', 'layers.11.attention.to_out.0.lora.down.weight', 'layers.11.attention.to_out.0.lora.up.weight', 'layers.11.attention.to_q.alpha', 'layers.11.attention.to.q.lora.down.weight', 'layers.11.attention.to.q.lora.up.weight', 'layers.11.attention.to_v.alpha', 'layers.11.attention.to.v.lora.down.weight', 'layers.11.attention.to.v.lora.up.weight', 'layers.12.attention.to_out.0.alpha', 'layers.12.attention.out.lora.down.weight', 'layers.12.attention.out.lora.up.weight', 'layers.12.attention.qkv.alpha', 'layers.12.attention.qkv.lora.down.weight', 'layers.12.attention.qkv.lora.up.weight', 'layers.12.attention.to_k.alpha', 'layers.12.attention.to.k.lora.down.weight', 'layers.12.attention.to.k.lora.up.weight', 'layers.12.attention.to_out.0.lora.down.weight', 'layers.12.attention.to_out.0.lora.up.weight', 'layers.12.attention.to_q.alpha', 'layers.12.attention.to.q.lora.down.weight', 'layers.12.attention.to.q.lora.up.weight', 'layers.12.attention.to_v.alpha', 'layers.12.attention.to.v.lora.down.weight', 'layers.12.attention.to.v.lora.up.weight', 'layers.13.attention.to_out.0.alpha', 'layers.13.attention.out.lora.down.weight', 'layers.13.attention.out.lora.up.weight', 'layers.13.attention.qkv.alpha', 'layers.13.attention.qkv.lora.down.weight', 'layers.13.attention.qkv.lora.up.weight', 'layers.13.attention.to_k.alpha', 'layers.13.attention.to.k.lora.down.weight', 'layers.13.attention.to.k.lora.up.weight', 'layers.13.attention.to_out.0.lora.down.weight', 'layers.13.attention.to_out.0.lora.up.weight', 'layers.13.attention.to_q.alpha', 'layers.13.attention.to.q.lora.down.weight', 'layers.13.attention.to.q.lora.up.weight', 'layers.13.attention.to_v.alpha', 'layers.13.attention.to.v.lora.down.weight', 'layers.13.attention.to.v.lora.up.weight', 'layers.14.attention.to_out.0.alpha', 'layers.14.attention.out.lora.down.weight', 'layers.14.attention.out.lora.up.weight', 'layers.14.attention.qkv.alpha', 'layers.14.attention.qkv.lora.down.weight', 'layers.14.attention.qkv.lora.up.weight', 'layers.14.attention.to_k.alpha', 'layers.14.attention.to.k.lora.down.weight', 'layers.14.attention.to.k.lora.up.weight', 'layers.14.attention.to_out.0.lora.down.weight', 'layers.14.attention.to_out.0.lora.up.weight', 'layers.14.attention.to_q.alpha', 'layers.14.attention.to.q.lora.down.weight', 'layers.14.attention.to.q.lora.up.weight', 'layers.14.attention.to_v.alpha', 'layers.14.attention.to.v.lora.down.weight', 'layers.14.attention.to.v.lora.up.weight', 'layers.15.attention.to_out.0.alpha', 'layers.15.attention.out.lora.down.weight', 'layers.15.attention.out.lora.up.weight', 'layers.15.attention.qkv.alpha', 'layers.15.attention.qkv.lora.down.weight', 'layers.15.attention.qkv.lora.up.weight', 'layers.15.attention.to_k.alpha', 'layers.15.attention.to.k.lora.down.weight', 'layers.15.attention.to.k.lora.up.weight', 'layers.15.attention.to_out.0.lora.down.weight', 'layers.15.attention.to_out.0.lora.up.weight', 'layers.15.attention.to_q.alpha', 'layers.15.attention.to.q.lora.down.weight', 'layers.15.attention.to.q.lora.up.weight', 'layers.15.attention.to_v.alpha', 'layers.15.attention.to.v.lora.down.weight', 'layers.15.attention.to.v.lora.up.weight', 'layers.16.attention.to_out.0.alpha', 'layers.16.attention.out.lora.down.weight', 'layers.16.attention.out.lora.up.weight', 'layers.16.attention.qkv.alpha', 'layers.16.attention.qkv.lora.down.weight', 'layers.16.attention.qkv.lora.up.weight', 'layers.16.attention.to_k.alpha', 'layers.16.attention.to.k.lora.down.weight', 'layers.16.attention.to.k.lora.up.weight', 'layers.16.attention.to_out.0.lora.down.weight', 'layers.16.attention.to_out.0.lora.up.weight', 'layers.16.attention.to_q.alpha', 'layers.16.attention.to.q.lora.down.weight', 'layers.16.attention.to.q.lora.up.weight', 'layers.16.attention.to_v.alpha', 'layers.16.attention.to.v.lora.down.weight', 'layers.16.attention.to.v.lora.up.weight', 'layers.17.attention.to_out.0.alpha', 'layers.17.attention.out.lora.down.weight', 'layers.17.attention.out.lora.up.weight', 'layers.17.attention.qkv.alpha', 'layers.17.attention.qkv.lora.down.weight', 'layers.17.attention.qkv.lora.up.weight', 'layers.17.attention.to_k.alpha', 'layers.17.attention.to.k.lora.down.weight', 'layers.17.attention.to.k.lora.up.weight', 'layers.17.attention.to_out.0.lora.down.weight', 'layers.17.attention.to_out.0.lora.up.weight', 'layers.17.attention.to_q.alpha', 'layers.17.attention.to.q.lora.down.weight', 'layers.17.attention.to.q.lora.up.weight', 'layers.17.attention.to_v.alpha', 'layers.17.attention.to.v.lora.down.weight', 'layers.17.attention.to.v.lora.up.weight', 'layers.18.attention.to_out.0.alpha', 'layers.18.attention.out.lora.down.weight', 'layers.18.attention.out.lora.up.weight', 'layers.18.attention.qkv.alpha', 'layers.18.attention.qkv.lora.down.weight', 'layers.18.attention.qkv.lora.up.weight', 'layers.18.attention.to_k.alpha', 'layers.18.attention.to.k.lora.down.weight', 'layers.18.attention.to.k.lora.up.weight', 'layers.18.attention.to_out.0.lora.down.weight', 'layers.18.attention.to_out.0.lora.up.weight', 'layers.18.attention.to_q.alpha', 'layers.18.attention.to.q.lora.down.weight', 'layers.18.attention.to.q.lora.up.weight', 'layers.18.attention.to_v.alpha', 'layers.18.attention.to.v.lora.down.weight', 'layers.18.attention.to.v.lora.up.weight', 'layers.19.attention.to_out.0.alpha', 'layers.19.attention.out.lora.down.weight', 'layers.19.attention.out.lora.up.weight', 'layers.19.attention.qkv.alpha', 'layers.19.attention.qkv.lora.down.weight', 'layers.19.attention.qkv.lora.up.weight', 'layers.19.attention.to_k.alpha', 'layers.19.attention.to.k.lora.down.weight', 'layers.19.attention.to.k.lora.up.weight', 'layers.19.attention.to_out.0.lora.down.weight', 'layers.19.attention.to_out.0.lora.up.weight', 'layers.19.attention.to_q.alpha', 'layers.19.attention.to.q.lora.down.weight', 'layers.19.attention.to.q.lora.up.weight', 'layers.19.attention.to_v.alpha', 'layers.19.attention.to.v.lora.down.weight', 'layers.19.attention.to.v.lora.up.weight', 'layers.1.attention.to_out.0.alpha', 'layers.1.attention.out.lora.down.weight', 'layers.1.attention.out.lora.up.weight', 'layers.1.attention.qkv.alpha', 'layers.1.attention.qkv.lora.down.weight', 'layers.1.attention.qkv.lora.up.weight', 'layers.1.attention.to_k.alpha', 'layers.1.attention.to.k.lora.down.weight', 'layers.1.attention.to.k.lora.up.weight', 'layers.1.attention.to_out.0.lora.down.weight', 'layers.1.attention.to_out.0.lora.up.weight', 'layers.1.attention.to_q.alpha', 'layers.1.attention.to.q.lora.down.weight', 'layers.1.attention.to.q.lora.up.weight', 'layers.1.attention.to_v.alpha', 'layers.1.attention.to.v.lora.down.weight', 'layers.1.attention.to.v.lora.up.weight', 'layers.20.attention.to_out.0.alpha', 'layers.20.attention.out.lora.down.weight', 'layers.20.attention.out.lora.up.weight', 'layers.20.attention.qkv.alpha', 'layers.20.attention.qkv.lora.down.weight', 'layers.20.attention.qkv.lora.up.weight', 'layers.20.attention.to_k.alpha', 'layers.20.attention.to.k.lora.down.weight', 'layers.20.attention.to.k.lora.up.weight', 'layers.20.attention.to_out.0.lora.down.weight', 'layers.20.attention.to_out.0.lora.up.weight', 'layers.20.attention.to_q.alpha', 'layers.20.attention.to.q.lora.down.weight', 'layers.20.attention.to.q.lora.up.weight', 'layers.20.attention.to_v.alpha', 'layers.20.attention.to.v.lora.down.weight', 'layers.20.attention.to.v.lora.up.weight', 'layers.21.attention.to_out.0.alpha', 'layers.21.attention.out.lora.down.weight', 'layers.21.attention.out.lora.up.weight', 'layers.21.attention.qkv.alpha', 'layers.21.attention.qkv.lora.down.weight', 'layers.21.attention.qkv.lora.up.weight', 'layers.21.attention.to_k.alpha', 'layers.21.attention.to.k.lora.down.weight', 'layers.21.attention.to.k.lora.up.weight', 'layers.21.attention.to_out.0.lora.down.weight', 'layers.21.attention.to_out.0.lora.up.weight', 'layers.21.attention.to_q.alpha', 'layers.21.attention.to.q.lora.down.weight', 'layers.21.attention.to.q.lora.up.weight', 'layers.21.attention.to_v.alpha', 'layers.21.attention.to.v.lora.down.weight', 'layers.21.attention.to.v.lora.up.weight', 'layers.22.attention.to_out.0.alpha', 'layers.22.attention.out.lora.down.weight', 'layers.22.attention.out.lora.up.weight', 'layers.22.attention.qkv.alpha', 'layers.22.attention.qkv.lora.down.weight', 'layers.22.attention.qkv.lora.up.weight', 'layers.22.attention.to_k.alpha', 'layers.22.attention.to.k.lora.down.weight', 'layers.22.attention.to.k.lora.up.weight', 'layers.22.attention.to_out.0.lora.down.weight', 'layers.22.attention.to_out.0.lora.up.weight', 'layers.22.attention.to_q.alpha', 'layers.22.attention.to.q.lora.down.weight', 'layers.22.attention.to.q.lora.up.weight', 'layers.22.attention.to_v.alpha', 'layers.22.attention.to.v.lora.down.weight', 'layers.22.attention.to.v.lora.up.weight', 'layers.23.attention.to_out.0.alpha', 'layers.23.attention.out.lora.down.weight', 'layers.23.attention.out.lora.up.weight', 'layers.23.attention.qkv.alpha', 'layers.23.attention.qkv.lora.down.weight', 'layers.23.attention.qkv.lora.up.weight', 'layers.23.attention.to_k.alpha', 'layers.23.attention.to.k.lora.down.weight', 'layers.23.attention.to.k.lora.up.weight', 'layers.23.attention.to_out.0.lora.down.weight', 'layers.23.attention.to_out.0.lora.up.weight', 'layers.23.attention.to_q.alpha', 'layers.23.attention.to.q.lora.down.weight', 'layers.23.attention.to.q.lora.up.weight', 'layers.23.attention.to_v.alpha', 'layers.23.attention.to.v.lora.down.weight', 'layers.23.attention.to.v.lora.up.weight', 'layers.24.attention.to_out.0.alpha', 'layers.24.attention.out.lora.down.weight', 'layers.24.attention.out.lora.up.weight', 'layers.24.attention.qkv.alpha', 'layers.24.attention.qkv.lora.down.weight', 'layers.24.attention.qkv.lora.up.weight', 'layers.24.attention.to_k.alpha', 'layers.24.attention.to.k.lora.down.weight', 'layers.24.attention.to.k.lora.up.weight', 'layers.24.attention.to_out.0.lora.down.weight', 'layers.24.attention.to_out.0.lora.up.weight', 'layers.24.attention.to_q.alpha', 'layers.24.attention.to.q.lora.down.weight', 'layers.24.attention.to.q.lora.up.weight', 'layers.24.attention.to_v.alpha', 'layers.24.attention.to.v.lora.down.weight', 'layers.24.attention.to.v.lora.up.weight', 'layers.25.attention.to_out.0.alpha', 'layers.25.attention.out.lora.down.weight', 'layers.25.attention.out.lora.up.weight', 'layers.25.attention.qkv.alpha', 'layers.25.attention.qkv.lora.down.weight', 'layers.25.attention.qkv.lora.up.weight', 'layers.25.attention.to_k.alpha', 'layers.25.attention.to.k.lora.down.weight', 'layers.25.attention.to.k.lora.up.weight', 'layers.25.attention.to_out.0.lora.down.weight', 'layers.25.attention.to_out.0.lora.up.weight', 'layers.25.attention.to_q.alpha', 'layers.25.attention.to.q.lora.down.weight', 'layers.25.attention.to.q.lora.up.weight', 'layers.25.attention.to_v.alpha', 'layers.25.attention.to.v.lora.down.weight', 'layers.25.attention.to.v.lora.up.weight', 'layers.26.attention.to_out.0.alpha', 'layers.26.attention.out.lora.down.weight', 'layers.26.attention.out.lora.up.weight', 'layers.26.attention.qkv.alpha', 'layers.26.attention.qkv.lora.down.weight', 'layers.26.attention.qkv.lora.up.weight', 'layers.26.attention.to_k.alpha', 'layers.26.attention.to.k.lora.down.weight', 'layers.26.attention.to.k.lora.up.weight', 'layers.26.attention.to_out.0.lora.down.weight', 'layers.26.attention.to_out.0.lora.up.weight', 'layers.26.attention.to_q.alpha', 'layers.26.attention.to.q.lora.down.weight', 'layers.26.attention.to.q.lora.up.weight', 'layers.26.attention.to_v.alpha', 'layers.26.attention.to.v.lora.down.weight', 'layers.26.attention.to.v.lora.up.weight', 'layers.27.attention.to_out.0.alpha', 'layers.27.attention.out.lora.down.weight', 'layers.27.attention.out.lora.up.weight', 'layers.27.attention.qkv.alpha', 'layers.27.attention.qkv.lora.down.weight', 'layers.27.attention.qkv.lora.up.weight', 'layers.27.attention.to_k.alpha', 'layers.27.attention.to.k.lora.down.weight', 'layers.27.attention.to.k.lora.up.weight', 'layers.27.attention.to_out.0.lora.down.weight', 'layers.27.attention.to_out.0.lora.up.weight', 'layers.27.attention.to_q.alpha', 'layers.27.attention.to.q.lora.down.weight', 'layers.27.attention.to.q.lora.up.weight', 'layers.27.attention.to_v.alpha', 'layers.27.attention.to.v.lora.down.weight', 'layers.27.attention.to.v.lora.up.weight', 'layers.28.attention.to_out.0.alpha', 'layers.28.attention.out.lora.down.weight', 'layers.28.attention.out.lora.up.weight', 'layers.28.attention.qkv.alpha', 'layers.28.attention.qkv.lora.down.weight', 'layers.28.attention.qkv.lora.up.weight', 'layers.28.attention.to_k.alpha', 'layers.28.attention.to.k.lora.down.weight', 'layers.28.attention.to.k.lora.up.weight', 'layers.28.attention.to_out.0.lora.down.weight', 'layers.28.attention.to_out.0.lora.up.weight', 'layers.28.attention.to_q.alpha', 'layers.28.attention.to.q.lora.down.weight', 'layers.28.attention.to.q.lora.up.weight', 'layers.28.attention.to_v.alpha', 'layers.28.attention.to.v.lora.down.weight', 'layers.28.attention.to.v.lora.up.weight', 'layers.29.attention.to_out.0.alpha', 'layers.29.attention.out.lora.down.weight', 'layers.29.attention.out.lora.up.weight', 'layers.29.attention.qkv.alpha', 'layers.29.attention.qkv.lora.down.weight', 'layers.29.attention.qkv.lora.up.weight', 'layers.29.attention.to_k.alpha', 'layers.29.attention.to.k.lora.down.weight', 'layers.29.attention.to.k.lora.up.weight', 'layers.29.attention.to_out.0.lora.down.weight', 'layers.29.attention.to_out.0.lora.up.weight', 'layers.29.attention.to_q.alpha', 'layers.29.attention.to.q.lora.down.weight', 'layers.29.attention.to.q.lora.up.weight', 'layers.29.attention.to_v.alpha', 'layers.29.attention.to.v.lora.down.weight', 'layers.29.attention.to.v.lora.up.weight', 'layers.2.attention.to_out.0.alpha', 'layers.2.attention.out.lora.down.weight', 'layers.2.attention.out.lora.up.weight', 'layers.2.attention.qkv.alpha', 'layers.2.attention.qkv.lora.down.weight', 'layers.2.attention.qkv.lora.up.weight', 'layers.2.attention.to_k.alpha', 'layers.2.attention.to.k.lora.down.weight', 'layers.2.attention.to.k.lora.up.weight', 'layers.2.attention.to_out.0.lora.down.weight', 'layers.2.attention.to_out.0.lora.up.weight', 'layers.2.attention.to_q.alpha', 'layers.2.attention.to.q.lora.down.weight', 'layers.2.attention.to.q.lora.up.weight', 'layers.2.attention.to_v.alpha', 'layers.2.attention.to.v.lora.down.weight', 'layers.2.attention.to.v.lora.up.weight', 'layers.3.attention.to_out.0.alpha', 'layers.3.attention.out.lora.down.weight', 'layers.3.attention.out.lora.up.weight', 'layers.3.attention.qkv.alpha', 'layers.3.attention.qkv.lora.down.weight', 'layers.3.attention.qkv.lora.up.weight', 'layers.3.attention.to_k.alpha', 'layers.3.attention.to.k.lora.down.weight', 'layers.3.attention.to.k.lora.up.weight', 'layers.3.attention.to_out.0.lora.down.weight', 'layers.3.attention.to_out.0.lora.up.weight', 'layers.3.attention.to_q.alpha', 'layers.3.attention.to.q.lora.down.weight', 'layers.3.attention.to.q.lora.up.weight', 'layers.3.attention.to_v.alpha', 'layers.3.attention.to.v.lora.down.weight', 'layers.3.attention.to.v.lora.up.weight', 'layers.4.attention.to_out.0.alpha', 'layers.4.attention.out.lora.down.weight', 'layers.4.attention.out.lora.up.weight', 'layers.4.attention.qkv.alpha', 'layers.4.attention.qkv.lora.down.weight', 'layers.4.attention.qkv.lora.up.weight', 'layers.4.attention.to_k.alpha', 'layers.4.attention.to.k.lora.down.weight', 'layers.4.attention.to.k.lora.up.weight', 'layers.4.attention.to_out.0.lora.down.weight', 'layers.4.attention.to_out.0.lora.up.weight', 'layers.4.attention.to_q.alpha', 'layers.4.attention.to.q.lora.down.weight', 'layers.4.attention.to.q.lora.up.weight', 'layers.4.attention.to_v.alpha', 'layers.4.attention.to.v.lora.down.weight', 'layers.4.attention.to.v.lora.up.weight', 'layers.5.attention.to_out.0.alpha', 'layers.5.attention.out.lora.down.weight', 'layers.5.attention.out.lora.up.weight', 'layers.5.attention.qkv.alpha', 'layers.5.attention.qkv.lora.down.weight', 'layers.5.attention.qkv.lora.up.weight', 'layers.5.attention.to_k.alpha', 'layers.5.attention.to.k.lora.down.weight', 'layers.5.attention.to.k.lora.up.weight', 'layers.5.attention.to_out.0.lora.down.weight', 'layers.5.attention.to_out.0.lora.up.weight', 'layers.5.attention.to_q.alpha', 'layers.5.attention.to.q.lora.down.weight', 'layers.5.attention.to.q.lora.up.weight', 'layers.5.attention.to_v.alpha', 'layers.5.attention.to.v.lora.down.weight', 'layers.5.attention.to.v.lora.up.weight', 'layers.6.attention.to_out.0.alpha', 'layers.6.attention.out.lora.down.weight', 'layers.6.attention.out.lora.up.weight', 'layers.6.attention.qkv.alpha', 'layers.6.attention.qkv.lora.down.weight', 'layers.6.attention.qkv.lora.up.weight', 'layers.6.attention.to_k.alpha', 'layers.6.attention.to.k.lora.down.weight', 'layers.6.attention.to.k.lora.up.weight', 'layers.6.attention.to_out.0.lora.down.weight', 'layers.6.attention.to_out.0.lora.up.weight', 'layers.6.attention.to_q.alpha', 'layers.6.attention.to.q.lora.down.weight', 'layers.6.attention.to.q.lora.up.weight', 'layers.6.attention.to_v.alpha', 'layers.6.attention.to.v.lora.down.weight', 'layers.6.attention.to.v.lora.up.weight', 'layers.7.attention.to_out.0.alpha', 'layers.7.attention.out.lora.down.weight', 'layers.7.attention.out.lora.up.weight', 'layers.7.attention.qkv.alpha', 'layers.7.attention.qkv.lora.down.weight', 'layers.7.attention.qkv.lora.up.weight', 'layers.7.attention.to_k.alpha', 'layers.7.attention.to.k.lora.down.weight', 'layers.7.attention.to.k.lora.up.weight', 'layers.7.attention.to_out.0.lora.down.weight', 'layers.7.attention.to_out.0.lora.up.weight', 'layers.7.attention.to_q.alpha', 'layers.7.attention.to.q.lora.down.weight', 'layers.7.attention.to.q.lora.up.weight', 'layers.7.attention.to_v.alpha', 'layers.7.attention.to.v.lora.down.weight', 'layers.7.attention.to.v.lora.up.weight', 'layers.8.attention.to_out.0.alpha', 'layers.8.attention.out.lora.down.weight', 'layers.8.attention.out.lora.up.weight', 'layers.8.attention.qkv.alpha', 'layers.8.attention.qkv.lora.down.weight', 'layers.8.attention.qkv.lora.up.weight', 'layers.8.attention.to_k.alpha', 'layers.8.attention.to.k.lora.down.weight', 'layers.8.attention.to.k.lora.up.weight', 'layers.8.attention.to_out.0.lora.down.weight', 'layers.8.attention.to_out.0.lora.up.weight', 'layers.8.attention.to_q.alpha', 'layers.8.attention.to.q.lora.down.weight', 'layers.8.attention.to.q.lora.up.weight', 'layers.8.attention.to_v.alpha', 'layers.8.attention.to.v.lora.down.weight', 'layers.8.attention.to.v.lora.up.weight', 'layers.9.attention.to_out.0.alpha', 'layers.9.attention.out.lora.down.weight', 'layers.9.attention.out.lora.up.weight', 'layers.9.attention.qkv.alpha', 'layers.9.attention.qkv.lora.down.weight', 'layers.9.attention.qkv.lora.up.weight', 'layers.9.attention.to_k.alpha', 'layers.9.attention.to.k.lora.down.weight', 'layers.9.attention.to.k.lora.up.weight', 'layers.9.attention.to_out.0.lora.down.weight', 'layers.9.attention.to_out.0.lora.up.weight', 'layers.9.attention.to_q.alpha', 'layers.9.attention.to.q.lora.down.weight', 'layers.9.attention.to.q.lora.up.weight', 'layers.9.attention.to_v.alpha', 'layers.9.attention.to.v.lora.down.weight', 'layers.9.attention.to.v.lora.up.weight', 'noise.refiner.0.attention.to_out.0.alpha', 'noise.refiner.0.attention.out.lora.down.weight', 'noise.refiner.0.attention.out.lora.up.weight', 'noise.refiner.0.attention.qkv.alpha', 'noise.refiner.0.attention.qkv.lora.down.weight', 'noise.refiner.0.attention.qkv.lora.up.weight', 'noise.refiner.0.attention.to_k.alpha', 'noise.refiner.0.attention.to.k.lora.down.weight', 'noise.refiner.0.attention.to.k.lora.up.weight', 'noise.refiner.0.attention.to_out.0.lora.down.weight', 'noise.refiner.0.attention.to_out.0.lora.up.weight', 'noise.refiner.0.attention.to_q.alpha', 'noise.refiner.0.attention.to.q.lora.down.weight', 'noise.refiner.0.attention.to.q.lora.up.weight', 'noise.refiner.0.attention.to_v.alpha', 'noise.refiner.0.attention.to.v.lora.down.weight', 'noise.refiner.0.attention.to.v.lora.up.weight', 'noise.refiner.1.attention.to_out.0.alpha', 'noise.refiner.1.attention.out.lora.down.weight', 'noise.refiner.1.attention.out.lora.up.weight', 'noise.refiner.1.attention.qkv.alpha', 'noise.refiner.1.attention.qkv.lora.down.weight', 'noise.refiner.1.attention.qkv.lora.up.weight', 'noise.refiner.1.attention.to_k.alpha', 'noise.refiner.1.attention.to.k.lora.down.weight', 'noise.refiner.1.attention.to.k.lora.up.weight', 'noise.refiner.1.attention.to_out.0.lora.down.weight', 'noise.refiner.1.attention.to_out.0.lora.up.weight', 'noise.refiner.1.attention.to_q.alpha', 'noise.refiner.1.attention.to.q.lora.down.weight', 'noise.refiner.1.attention.to.q.lora.up.weight', 'noise.refiner.1.attention.to_v.alpha', 'noise.refiner.1.attention.to.v.lora.down.weight', 'noise.refiner.1.attention.to.v.lora.up.weight'])

Reproduction

Code to reproduce the error:

import torch
from diffusers import ZImagePipeline

pipe = ZImagePipeline.from_pretrained("Tongyi-MAI/Z-Image-Turbo", torch_dtype=torch.bfloat16)

pipe.load_lora_weights(
    "strangerzonehf/Anime-Z", weight_name="Anime-Z.safetensors",
)

System Info

Latest commit of Diffusers (e7fe4ce)

Who can help?

@sayakpaul @asomoza

[tdvipp]

When comparing the state dict formats of these two:

• https://huggingface.co/suayptalha/Z-Image-Turbo-Realism-LoRA/blob/main/pytorch_lora_weights.safetensors
• https://huggingface.co/strangerzonehf/Anime-Z/blob/main/Anime-Z.safetensors

I noticed that Anime-Z.safetensors contains keys like lora_unet_context_refiner_0_attention_out, which is unusual for a Z-Image-style model since Z-Image Turbo doesn’t use a UNet-based architecture. That suggests the Anime-Z LoRA might have been trained on a different base model rather than Z-Image-Turbo.

[sayakpaul]

@MrTuanDao it's common in many trainers to use the lora_unet prefix for historical reasons.

Anyway, I will have look into this issue.