[shardformer] align BERT value #3906

Closed

Assignees

Labels

shardformertesting

opened

A sharded BERT has been implemented in our shardformer branch. We need to make sure its numerical values can be aligned with the non-sharded version, including the output and gradients.

Metadata

Assignees

FoolPlayer

Labels

shardformertesting

Type

No type

Projects

Status

✅ Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests