fix(x_clip): fix 8 failed test cases#45394
Conversation
Fixed 8 test(s): - tests/models/x_clip/test_modeling_x_clip.py::XCLIPVisionModelTest::test_flash_attn_2_inference_equivalence - tests/models/x_clip/test_modeling_x_clip.py::XCLIPVisionModelTest::test_flash_attn_2_inference_equivalence_right_padding - tests/models/x_clip/test_modeling_x_clip.py::XCLIPVisionModelTest::test_model_parallelism - tests/models/x_clip/test_modeling_x_clip.py::XCLIPModelTest::test_flash_attn_2_inference_equivalence - tests/models/x_clip/test_modeling_x_clip.py::XCLIPModelTest::test_flash_attn_2_inference_equivalence_right_padding - tests/models/x_clip/test_modeling_x_clip.py::XCLIPModelTest::test_model_parallelism - tests/models/x_clip/test_modeling_x_clip.py::XCLIPModelIntegrationTest::test_inference - tests/models/x_clip/test_modeling_x_clip.py::XCLIPModelIntegrationTest::test_inference_interpolate_pos_encoding
|
This PR fixes 8 failed test cases for x_clip model: |
kaixuanliu
changed the title
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
kaixuanliu
changed the title
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
kaixuanliu
changed the title
|
@zucchini-nlp Hi, can you help review? Thx! |
| @unittest.skip( | ||
| reason="X-CLIP's hidden_states are nested in sub-outputs (text_model_output, vision_model_output), not at root level" | ||
| ) | ||
| def test_flash_attn_2_inference_equivalence(self): | ||
| pass | ||
|
|
There was a problem hiding this comment.
can change this part to check if output has llogits_per_video, ig xclip has no images which is why it fails now
transformers/tests/test_modeling_common.py
Lines 3357 to 3366 in 69448db
There was a problem hiding this comment.
I tried this but still failed for this case. As the value of logits_per_video is very sensitive to the vision_model_output's hidden_states, the tolerance is not enough.
There was a problem hiding this comment.
Oops, on CUDA it is OK, I will update the code here.
| @unittest.skip(reason="X-CLIP needs batch size to match frames, can't crop and create new dummy inputs") | ||
| def test_flash_attn_2_inference_equivalence(self): | ||
| pass | ||
|
|
There was a problem hiding this comment.
i suppose already failing on main, but not sure we want to just skip it. @ydshieh to review
Signed-off-by: Liu, Kaixuan <kaixuan.liu@intel.com>
|
[For maintainers] Suggested jobs to run (before merge) run-slow: x_clip |
| def __call__(self, images=None, text=None, videos=None, **kwargs): | ||
| # X-CLIP uses the image_processor for video frames. Map videos to images | ||
| # so the base class processes them through image_processor. | ||
| if videos is not None and images is None: | ||
| images = videos | ||
| return super().__call__(images=images, text=text, **kwargs) |
There was a problem hiding this comment.
cc @yonigozlan and @zucchini-nlp if you remember the history and could judge the change here.
There was a problem hiding this comment.
yea, I made those changes. Actually x-clip processes videos in old-style via an image processor, so this is a valid fix
I don't mind it, so we don't have to override the whole __call__
|
run inside CI runner, all ✅ |
3 participants
No description provided.