Conversation
|
[For maintainers] Suggested jobs to run (before merge) run-slow: qwen2 |
|
run-slow: qwen2 |
|
This comment contains models: ["models/qwen2"] |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
| # Expected mean on dim = -1 | ||
| EXPECTED_MEAN = Expectations({ | ||
| (None, None): torch.tensor([[-2.2121, -1.6335, -1.4816, -1.5035, -1.9110, -1.8979, -1.9682, -2.1980]]), | ||
| (None, None): torch.tensor([[-2.2463, -1.6463, -1.4748, -1.4913, -1.9213, -1.9016, -1.9969, -2.1761]]), |
There was a problem hiding this comment.
@Abdennacer-Badaoui Maybe this will fail on AMD, if you want to double check. Although maybe I should use ("cuda", 8) and ("rocm", 9). But let's have double check anyway.
| # slicing logits[0, 0, 0:30] | ||
| EXPECTED_SLICE = Expectations({ | ||
| (None, None): torch.tensor([2.7344, 4.2812, 4.1562, 2.3906, 1.1875, 2.1562, 3.1719, 3.1406, 1.2891, 3.6094, 3.3125, 1.8203, 2.9219, 3.2344, 1.5938, 6.2500, 7.4062, 7.2188, 6.5938, 6.0312, 6.1562, 5.3750, 5.9688, 5.5938, 6.1250, 1.2656, 1.6016, 3.4062, 1.7891, 3.6406]), | ||
| (None, None): torch.tensor([2.6406, 4.3125, 3.9531, 2.2656, 1.0000, 2.0312, 3.2344, 2.9219, 1.1953, 3.3750, 3.1875, 2.0156, 2.8281, 3.2656, 1.4453, 6.0625, 7.2188, 7.0312, 6.4375, 5.8750, 6.0312, 5.2500, 5.8438, 5.5000, 5.9688, 1.2734, 1.5312, 3.2344, 1.6406, 3.4375]), |
There was a problem hiding this comment.
@Abdennacer-Badaoui Maybe this will fail on AMD, if you want to double check. Although maybe I should use ("cuda", 8) and ("rocm", 9). But let's have double check anyway.
fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
Thanks for pinging @ydshieh. I don't have a fixed GPU, i use available cloud gpu's, I remember using A10 only that day, but I don't recall for sure. I just tested with A10 and it's giving the expected output that's fixed here. So, this should be the correct expected text. |
|
Thanks for the fix! for #43794 I did run the GPU tests but it looks like the result was a false positive |
fix Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
4 participants
What does this PR do?
Fix
Qwen2IntegrationTest:test_speculative_generation:0c89522f: Enforce manual seed to reduce flakiness #43794 changed seed, so the actual output changed, but the expected output is not updated (cc @tarekziade more attention next time 😄 )af93d384: [AMD CI] Fix test failures across important models #44632 changed the expected output to correct value but with wrong formata8732d55: Fix failingT5ModelIntegrationTest#44934 changed the expected output to wrong value but with correct format (@Sai-Suraj-27 Are you using A10?)test_model_450m_logits:6217adc6: Default auto 🚨 🚨 #42805 changed model dtype, but didn't update the expected outputs and failed several testsc154b021: [CI] Fixing some AMD failures #42879 updated the expected outputs, but the one fortest_model_450m_logitsnot matching the actual output onA10. (@Abdennacer-Badaoui , maybe we can double check together the runner you are using is the good one)