make sure model.save_pretrained has the correct is_main_process

[winglian]

What does this PR do?

Now that multiple processes can be involved in saving due to parallelism (see #39693), there are intermittent race conditions since the default to save_pretrained is is_main_process=True. Stacktrace below that intermittently happens on later checkpoints

[rank1]:   File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/trainer.py", line 3237, in _save_checkpoint
[rank1]:     self.save_model(output_dir, _internal_call=True)                                                                                                                    
[rank1]:   File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/trainer.py", line 3959, in save_model
[rank1]:     self._save(output_dir)   
[rank1]:   File "/workspace/axolotl/src/axolotl/core/trainers/mixins/distributed_parallel.py", line 20, in _save
[rank1]:     super()._save(output_dir, state_dict=state_dict)
[rank1]:   File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/trainer.py", line 4091, in _save
[rank1]:     self.model.save_pretrained(                                                                                                                                         
[rank1]:   File "/root/miniconda3/envs/py3.11/lib/python3.11/site-packages/transformers/modeling_utils.py", line 4062, in save_pretrained
[rank1]:     os.remove(full_filename)                                                   
[rank1]: FileNotFoundError: [Errno 2] No such file or directory: './model-out/checkpoint-34/model-00001-of-00004.safetensors'

@SunMarc @ArthurZucker @S1ro1

Fixes # (issue)

Before submitting

• This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
• Did you read the contributor guideline,
  Pull Request section?
• Was this discussed/approved via a Github issue or the forum? Please add a link
  to it if that's the case.
• Did you make sure to update the documentation with your changes? Here are the
  documentation guidelines, and
  here are tips on formatting docstrings.
• Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

[ArthurZucker]

LGTM let's add a test! 🤗