Skip to content

[FEATURE]: Refactor checkpointIO API and add safetensors support #3426

@FrankLeeeee

Description

@FrankLeeeee

Describe the feature

A better module structure can be something like below for the checkpoint io module:

- index_file.py
- checkpoint_io_base.py
- general_checkpoint_io.py

Meanwhile, we can add support for huggingface safetensors.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

Status

✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions