Qwen Edit Plus + Qwen Edit pipelines fail (TypeError: argument of type 'NoneType' is not iterable)

[christopher5106]

Describe the bug

When I take the example given at the beginning of the file, I get the following error:

  File "/workspace/diffusers/test_loras/qwenimageplus.py", line 6, in <module>
    pipe = QwenImageEditPlusPipeline.from_pretrained("Qwen/Qwen-Image-Edit-2509", torch_dtype=torch.bfloat16)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/huggingface_hub/utils/_validators.py", line 89, in _inner_fn
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/src/diffusers/pipelines/pipeline_utils.py", line 1043, in from_pretrained
    loaded_sub_model = load_sub_model(
                       ^^^^^^^^^^^^^^^
  File "/workspace/diffusers/src/diffusers/pipelines/pipeline_loading_utils.py", line 885, in load_sub_model
    loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/processing_utils.py", line 1402, in from_pretrained
    args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, processor_dict, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/processing_utils.py", line 1523, in _get_arguments_from_pretrained
    sub_processor = auto_processor_class.from_pretrained(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/models/auto/video_processing_auto.py", line 360, in from_pretrained
    video_processor_class = video_processor_class_from_name(video_processor_class)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/models/auto/video_processing_auto.py", line 96, in video_processor_class_from_name
    if class_name in extractors:
       ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

Reproduction

import torch
from PIL import Image
from diffusers import QwenImageEditPlusPipeline
from diffusers.utils import load_image

pipe = QwenImageEditPlusPipeline.from_pretrained("Qwen/Qwen-Image-Edit-2509", torch_dtype=torch.bfloat16)
pipe.to("cuda")
image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/yarn-art-pikachu.png"
).convert("RGB")
prompt = (
    "Make Pikachu hold a sign that says 'Qwen Edit is awesome', yarn art style, detailed, vibrant colors"
)
# Depending on the variant being used, the pipeline call will slightly vary.
# Refer to the pipeline documentation for more details.
image = pipe(image, prompt, num_inference_steps=50).images[0]
image.save("qwenimage_edit_plus.png")

Logs

System Info

Diffusers@c3a4cd14b82dd84ea8dd803bdafc82bf12fd337f

Who can help?

@sayakpaul

[christopher5106]

I rebased on Diffusers latest commit 2843b3d and got the same error.

[christopher5106]

Same problem with Qwen-Edit simple:

import torch
from PIL import Image
from diffusers import QwenImageEditPipeline
from diffusers.utils import load_image

pipe = QwenImageEditPipeline.from_pretrained("Qwen/Qwen-Image-Edit", torch_dtype=torch.bfloat16)
pipe.to("cuda")
image = load_image(
    "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/yarn-art-pikachu.png"
).convert("RGB")
prompt = (
    "Make Pikachu hold a sign that says 'Qwen Edit is awesome', yarn art style, detailed, vibrant colors"
)
# Depending on the variant being used, the pipeline call will slightly vary.
# Refer to the pipeline documentation for more details.
image = pipe(image, prompt, num_inference_steps=50).images[0]
image.save("qwenimage_edit.png")

2 weeks ago it was working well.

[sayakpaul]

Seems to be coming from transformers. Can you reproduce with just transformers?

[christopher5106]

The transformer is loading correctly, but the Qwen2VLProcessor not:

>>> from transformers import Qwen2VLProcessor
>>> Qwen2VLProcessor.from_pretrained("Qwen/Qwen-Image-Edit", torch_dtype=torch.bfloat16, subfolder="processor")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/processing_utils.py", line 1402, in from_pretrained
    args = cls._get_arguments_from_pretrained(pretrained_model_name_or_path, processor_dict, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/processing_utils.py", line 1523, in _get_arguments_from_pretrained
    sub_processor = auto_processor_class.from_pretrained(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/models/auto/video_processing_auto.py", line 360, in from_pretrained
    video_processor_class = video_processor_class_from_name(video_processor_class)
                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/diffusers/venv/lib/python3.11/site-packages/transformers/models/auto/video_processing_auto.py", line 96, in video_processor_class_from_name
    if class_name in extractors:
       ^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'NoneType' is not iterable

Reproduced error.

[sayakpaul]

Then the issue should go to the transformers repo. Not here :)

[christopher5106]

Seems to be already being fixed huggingface/transformers#43986

[christopher5106]

I'm closing because it's unrelated to diffusers and installing torchvision solves the issue