Some huggingface models are saved as sharded checkpoints, we should support loading from and saving as HF sharded checkpoint. An exmaple would be https://huggingface.co/facebook/opt-13b/blob/main/pytorch_model.bin.index.json
Some huggingface models are saved as sharded checkpoints, we should support loading from and saving as HF sharded checkpoint.
An exmaple would be https://huggingface.co/facebook/opt-13b/blob/main/pytorch_model.bin.index.json