[FEATURE]: Add Ulysses Sequence Parallelism support for Command-R, Qwen2 and ChatGLM #5853

Closed

Labels

enhancementshardformer

opened

on Jun 25, 2024

Describe the feature

Please add Ulysses Sequence Parallelism support for Command-R, Qwen2 and ChatGLM

Metadata

Assignees

No one assigned

Labels

enhancementshardformer

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests