fix: prevent accelerate from splitting vision encoder by setting _no_…

[CodersAcademy006]

This PR resolves the model parallelism crash in PeVideo and PeAudioVideo models by adding _no_split_modules = ["TimmWrapperForImageClassification"] to their configuration. Currently, accelerate naively splits the timm-based vision encoder layer-by-layer across devices, breaking internal residual connections and causing a RuntimeError during distributed training. By explicitly marking the wrapper as a non-splittable module, we ensure the vision encoder remains atomic on a single device, restoring stability for FSDP and model parallelism workflows as verified by the now-passing test_model_parallelism unit tests.

Fixes #42918

[ArthurZucker]

Thanks

[ArthurZucker]

can you fix the consistency (python utils/modular_model_converter.py)

[CodersAcademy006]

@ArthurZucker, sorry for being late. Fixed, updated modular_pe_video.py and modular_pe_audio_video.py to declare _no_split_modules explicitly in the PreTrainedModel classes (needed because the converter resolves parent class from the installed package). Ran modular_model_converter.py to regenerate both modeling files and removed the @unittest.skip decorators from test_model_parallelism in both test files. All four files should now be consistent.

[github-actions]

[For maintainers] Suggested jobs to run (before merge)

run-slow: pe_audio, pe_audio_video, pe_video

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.