TP-experiments

Repo contains ongoing experiments with TP + DP with or without accelerate and FSDP.

Name		Name	Last commit message	Last commit date
Latest commit History 14 Commits
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
hf_trainer_accelerate_tp_dp.py		hf_trainer_accelerate_tp_dp.py
hf_trainer_ddp.py		hf_trainer_ddp.py
init.sh		init.sh
torch_tp_dp_llama.py		torch_tp_dp_llama.py
torch_tp_dp_mlp.py		torch_tp_dp_mlp.py

Provide feedback