[FEATURE]: shardformer to support interleaved pipeline parallel #4833

Closed

Labels

opened

on Sep 28, 2023

Describe the feature

shardformer currently only supports non-interleaved 1f1b pipeline parallel

Metadata

Assignees

No one assigned

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests