Conversation
ydshieh
changed the title
test_eager_matches sdpa generate and update an integration test for blip-like models
|
[For maintainers] Suggested jobs to run (before merge) run-slow: blip_2, instructblip, instructblipvideo |
|
run-slow: blip_2, instructblip, instructblipvideo |
|
This comment contains run-slow, running the specified jobs: models: ['models/blip_2', 'models/instructblip', 'models/instructblipvideo'] |
| generated_text = processor.batch_decode(outputs, skip_special_tokens=True)[0] | ||
|
|
||
| expected_outputs = [0, 37, 7225, 1023, 9850, 7, 3, 9, 388, 3575, 53, 4954, 30, 8, 223, 13, 3, 9, 4459, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 388, 19, 5119, 3, 9, 4459, 8677, 28, 46, 3575, 53, 1476, 5223, 12, 34, 6, 15495, 24, 3, 88, 19, 692, 112, 293, 10428, 44, 234, 1066, 145, 338, 3, 9, 50, 1106, 3522, 144, 42, 2192, 7919, 31, 7, 5, 37, 1023, 92, 1267, 3, 9, 381, 13, 119, 3203, 16, 8, 2458, 6, 379, 14264, 6, 9256, 7, 6, 11, 11718, 7, 5, 1] # fmt: skip | ||
| expected_outputs = [0, 37, 1023, 9850, 7, 3, 9, 388, 3575, 53, 4954, 30, 8, 223, 13, 3, 9, 4459, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 388, 19, 5119, 3, 9, 4459, 8677, 28, 3, 9, 4459, 6177, 6, 11, 3, 88, 19, 338, 46, 3575, 53, 1476, 5223, 12, 8, 223, 13, 8, 4049, 5, 37, 1023, 19, 7225, 16, 24, 34, 1267, 3, 9, 388, 3575, 53, 4954, 30, 8, 223, 13, 3, 9, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 388, 19, 338, 46, 3575, 53, 1476, 5223, 12, 8, 223, 13, 3, 9, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 388, 19, 338, 46, 3575, 53, 1476, 5223, 12, 8, 223, 13, 3, 9, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 1023, 19, 7225, 16, 24, 34, 1267, 3, 9, 388, 3575, 53, 4954, 30, 8, 223, 13, 3, 9, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 37, 388, 19, 338, 46, 3575, 53, 1476, 5223, 12, 8, 223, 13, 3, 9, 4049, 16, 8, 2214, 13, 3, 9, 3164, 690, 2815, 5, 1] # fmt: skip |
There was a problem hiding this comment.
[core] Large/full refactor of from_pretrained (#36033) change the values here on T4.
The commit before would pass with the previous value on T4, but now we also change to A10, and that previous commit also fail on A10.
Let's just update the values.
There was a problem hiding this comment.
I don't know how much this is expected 😅
There was a problem hiding this comment.
The output is not too far off from the previous one, but it's indeed a bit strange why that PR would change the output values.
There was a problem hiding this comment.
cc @Cyrilvallez if you want to dive into this (if so, remember to check on T4)
There was a problem hiding this comment.
Could be also the dtype, I remember having some issues in prev versions before refactor. For ex: if a backbone needs to be loaded in fp32, somehow it was still converted first to fp16 and then casted back to fp32, can't remember the reason
There was a problem hiding this comment.
Yeah, that's indeed a likely reason!
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
|
The failing slow CI jobs are all in multi-gpu runners, which are not treated at this moment. |
zucchini-nlp
left a comment
zucchini-nlp
left a comment
There was a problem hiding this comment.
Thanks, these have to be skipped in the end anyway, unless we make it multimodal-friendly. I am adding a check for all backbones in #38974 :)
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
…t for blip-like models (huggingface#39248) * skip * skip --------- Co-authored-by: ydshieh <ydshieh@users.noreply.github.com>
4 participants
What does this PR do?
For skip, see this comment. Let's make the CI report clean though for now.