Skip to content

[pipeline,shardformer] Fix p2p efficiency in pipeline, allow skipping loading weight not in weight_map when strict=False, fix llama flash attention forward, add flop estimation by megatron in llama benchmark#5017

Merged
ver217 merged 15 commits intohpcaitech:mainfrom
zeyugao:main
Nov 16, 2023

Commits

Commits on Nov 5, 2023

Commits on Nov 6, 2023

Commits on Nov 7, 2023

Commits on Nov 8, 2023

Commits on Nov 13, 2023

Commits on Nov 15, 2023