[feature] support no master weights option for low level zero plugin#4816

Merged

KKZ20 merged 14 commits intohpcaitech:mainfrom

KKZ20:feature/no_master_weights_llz

Oct 13, 2023

Commits on Sep 27, 2023

[feature] support no master weights for low level zero plugin
Zhongkai Zhao
committed
[feature] support no master weights for low level zero plugin, remove data copy when no master weights
Zhongkai Zhao
committed
remove data copy and typecasting when no master weights
Zhongkai Zhao
committed
not load weights to cpu when using no master weights
Zhongkai Zhao
committed

Commits on Oct 11, 2023

fix grad: use fp16 grad when no master weights
Zhongkai Zhao
committed
fix code complexity
Zhongkai Zhao
committed
retry fix code complexity
Zhongkai Zhao
committed

Commits on Oct 12, 2023

do not update working param
Zhongkai Zhao
committed
only do not update working param when no master weights
Zhongkai Zhao
committed
fix: only do not update working param when no master weights
Zhongkai Zhao
committed
fix: passing params in dict format in hybrid plugin
Zhongkai Zhao
committed
Merge branch 'main' into feature/no_master_weights_llz
KKZ20
authored

Commits on Oct 13, 2023

fix: remove extra params (tp_process_group) in hybrid_parallel_plugin
Zhongkai Zhao
committed
add a comment
KKZ20
committed