Skip to content

[AMD CI] Fix test failures across important models #44632

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Abdennacer-Badaoui merged 8 commits into from
Mar 17, 2026
Merged

[AMD CI] Fix test failures across important models #44632

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
81d6ce3
qwen2
Abdennacer-Badaoui Mar 3, 2026
eba6b69
merge
Abdennacer-Badaoui Mar 11, 2026
47b9cfb
qwen_2_audio
Abdennacer-Badaoui Mar 3, 2026
e8dcd2f
fix more
Abdennacer-Badaoui Mar 10, 2026
ec59b94
qwen2_5_vl
Abdennacer-Badaoui Mar 11, 2026
4a2142d
style
Abdennacer-Badaoui Mar 12, 2026
014f4b9
fix
Abdennacer-Badaoui Mar 12, 2026
62fa178
Merge branch 'main' into qwens
Abdennacer-Badaoui Mar 17, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docker/transformers-pytorch-amd-gpu/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ RUN git clone https://github.com/ROCm/flash-attention/ -b tridao && \
GPU_ARCHS="gfx942" python setup.py install
# GPU_ARCHS builds for MI300, MI325 but not MI355: we would need to add `;gfx950` but it takes too long to build.

RUN python3 -m pip install --no-cache-dir einops
RUN python3 -m pip install --no-cache-dir einops blobfile num2words

# timm is required for many vision models tests
RUN python3 -m pip install --no-cache-dir timm
6 changes: 3 additions & 3 deletions src/transformers/models/qwen2_5_omni/modeling_qwen2_5_omni.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2404,9 +2404,9 @@ def prepare_inputs_for_generation(
):
model_inputs = super().prepare_inputs_for_generation(
input_ids,
past_key_values,
attention_mask,
inputs_embeds,
past_key_values=past_key_values,
attention_mask=attention_mask,
inputs_embeds=inputs_embeds,
Comment on lines +2407 to +2409
Copy link
Copy Markdown
Collaborator

@ydshieh ydshieh Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you share more info about what was wrong when passing as pos. arguments?

I am fine for the change, but just prefer some details 🙏

Copy link
Copy Markdown
Member Author

@Abdennacer-Badaoui Abdennacer-Badaoui Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parent GenerationMixin.prepare_inputs_for_generation added next_sequence_length as the 2nd positional arg (right after input_ids). So when the talker passed past_key_values positionally in that slot, it got bound to next_sequence_length instead. Then next_sequence_length also came in through **kwargs from the caller → duplicate value → TypeError.

use_cache=use_cache,
thinker_reply_part=thinker_reply_part,
input_text_ids=input_text_ids,
Expand Down
6 changes: 3 additions & 3 deletions src/transformers/models/qwen2_5_omni/modular_qwen2_5_omni.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2238,9 +2238,9 @@ def prepare_inputs_for_generation(
):
model_inputs = super().prepare_inputs_for_generation(
input_ids,
past_key_values,
attention_mask,
inputs_embeds,
past_key_values=past_key_values,
attention_mask=attention_mask,
inputs_embeds=inputs_embeds,
use_cache=use_cache,
thinker_reply_part=thinker_reply_part,
input_text_ids=input_text_ids,
Expand Down
2 changes: 1 addition & 1 deletion tests/models/llava/test_modeling_llava.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -579,7 +579,7 @@ def test_tokenizer_integration(self):
fast_tokenizer.add_tokens("<image>", True)

prompt = "<|im_start|>system\nAnswer the questions.<|im_end|><|im_start|>user\n<image>\nWhat is shown in this image?<|im_end|><|im_start|>assistant\n"
EXPECTED_OUTPUT = ['<|im_start|>', 'sy', 'st', 'em', '\n', 'An', 'sw', 'er', ' ', 'the', ' ', 'qu', 'est', 'ions', '.', '<|im_end|>', '<|im_start|>', 'us', 'er', '\n', '<image>', '\n', 'What', ' ', 'is', ' ', 'sh', 'own', ' ', 'in', ' ', 'th', 'is', ' ', 'im', 'age', '?', '<|im_end|>', '<|im_start|>', 'ass', 'ist', 'ant', '\n'] # fmt: skip
EXPECTED_OUTPUT = ['<|im_start|>', 'system', '\n', 'Answer', '▁the', '▁questions', '.', '<|im_end|>', '<|im_start|>', 'user', '\n', '<image>', '\n', 'What', '▁is', '▁shown', '▁in', '▁this', '▁image', '?', '<|im_end|>', '<|im_start|>', 'ass', 'istant', '\n'] # fmt: skip
Copy link
Copy Markdown
Collaborator

@ydshieh ydshieh Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to use Expectation class here.

(I don't know if this updated values are good across different hardware however : A10, AMD etc.)

Copy link
Copy Markdown
Member Author

@Abdennacer-Badaoui Abdennacer-Badaoui Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are good for A10 as well : https://github.com/huggingface/transformers/actions/runs/23034484105/job/66899818920

self.assertEqual(slow_tokenizer.tokenize(prompt), EXPECTED_OUTPUT)
self.assertEqual(fast_tokenizer.tokenize(prompt), EXPECTED_OUTPUT)

Expand Down
8 changes: 3 additions & 5 deletions tests/models/qwen2/test_modeling_qwen2.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -183,11 +183,9 @@ def test_model_450m_long_prompt_sdpa(self):

@slow
def test_speculative_generation(self):
EXPECTED_TEXT_COMPLETION = Expectations({
(None, None): "My favourite condiment is 100% natural and organic, and I love to use it to make my own sauces.",
("xpu", 3): "My favourite condiment is 100% natural, organic, gluten-free, vegan, and vegetarian. I have been making",
}) # fmt: off

EXPECTED_TEXT_COMPLETION = (
"My favourite condiment is 100% natural, organic, gluten-free, vegan, and vegetarian. I have been making"
Copy link
Copy Markdown
Collaborator

@ydshieh ydshieh Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same

Copy link
Copy Markdown
Member Author

@Abdennacer-Badaoui Abdennacer-Badaoui Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good for A10 as well :
https://github.com/huggingface/transformers/actions/runs/23034484105/job/66899814875

)
prompt = "My favourite condiment is "
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2-7B", use_fast=False)
model = Qwen2ForCausalLM.from_pretrained("Qwen/Qwen2-0.5B", device_map="auto", dtype=torch.float16)
Expand Down
33 changes: 24 additions & 9 deletions tests/models/qwen2_5_vl/test_modeling_qwen2_5_vl.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -508,14 +508,22 @@ def test_small_model_integration_test_batch(self):
# it should not matter whether two images are the same size or not
output = model.generate(**inputs, max_new_tokens=30, do_sample=False)

EXPECTED_DECODED_TEXT = [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
] # fmt: skip
expected_decoded_texts = Expectations(
{
(None, None): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
],
("rocm", (9, 4)): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is often reflected',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is often reflected'
],
}
).get_expectation() # fmt: skip

self.assertEqual(
self.processor.batch_decode(output, skip_special_tokens=True),
EXPECTED_DECODED_TEXT,
expected_decoded_texts,
)

@slow
Expand Down Expand Up @@ -563,6 +571,10 @@ def test_small_model_integration_test_batch_wo_image(self):
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWho are you?\nassistant\nI am Qwen, an AI language model created by Alibaba Cloud. I am designed to assist with various tasks such as answering questions, providing information,'
],
("rocm", (9, 4)): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWho are you?\nassistant\nI am Qwen, a large language model created by Alibaba Cloud. I am designed to assist with a wide range of tasks, from answering questions and'
],
("xpu", None): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWho are you?\nassistant\nI am Qwen, a large language model created by Alibaba Cloud. I am designed to assist with a wide range of tasks, from answering questions and'
Expand Down Expand Up @@ -604,8 +616,8 @@ def test_small_model_integration_test_batch_different_resolutions(self):
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is evident in',
],
("rocm", None): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\n addCriterion\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is often reflected',
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is evident in'
],
("xpu", None): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
Expand Down Expand Up @@ -643,7 +655,7 @@ def test_small_model_integration_test_batch_flashatt2(self):

expected_decoded_text = Expectations({
("cuda", None): "system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in",
("rocm", (9, 4)): "system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in",
("rocm", (9, 4)): "system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and gentle nature, which is evident in",
("xpu", None): "system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in",
}).get_expectation() # fmt: skip

Expand Down Expand Up @@ -685,7 +697,7 @@ def test_small_model_integration_test_batch_wo_image_flashatt2(self):
],
("rocm", (9, 4)): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
"system\nYou are a helpful assistant.\nuser\nWho are you?\nassistant\nI am Qwen, a large language model created by Alibaba Cloud. I am designed to answer a wide range of questions and provide information on various topics",
'system\nYou are a helpful assistant.\nuser\nWho are you?\nassistant\nI am Qwen, a large language model created by Alibaba Cloud. I am designed to assist with a wide range of tasks, from answering questions and'
],
("xpu", None): [
'system\nYou are a helpful assistant.\nuser\nWhat kind of dog is this?\nassistant\nThe dog in the picture appears to be a Labrador Retriever. Labradors are known for their friendly and energetic nature, which is evident in',
Expand Down Expand Up @@ -741,6 +753,9 @@ def test_small_model_integration_test_with_video(self):
(None, None): [
'system\nYou are a helpful assistant.\nuser\nWhat is shown in this video?\nassistant\nThe video shows an indoor tennis court with a person standing on the service line, preparing to serve. The individual is wearing athletic attire, including a white',
],
("rocm", (9, 4)): [
'system\nYou are a helpful assistant.\nuser\nWhat is shown in this video?\nassistant\nThe video shows an indoor tennis court with a person standing on the service line, preparing to serve. The individual appears to be practicing or warming up,',
],
("xpu", None): [
'system\nYou are a helpful assistant.\nuser\nWhat is shown in this video?\nassistant\nThe video shows an indoor tennis court with a person standing on the service line, preparing to serve. The individual appears to be practicing or warming up,',
],
Expand Down
Loading