[BUG]: Llama2 HybridParallelPlugin train failed when pp_size>1

[xs1997zju]

🐛 Describe the bug

Llama2 HybridParallelPlugin train failed when pp_size>1
we modified the llama training example to use hybridparallel plugin but encountered such an error.
The hidden_states is None in colossalai/shardformer/modeling/llama.py:61 in llama_model_forward

Environment

torch-'1.13.1+cu117', transformers-4.32.0, colossalAI-release-v0.3.2

[flybird11111]

Hi, The example of using the hybrid parallel plugin in llama2 is still in development and has not been merged. If you have any issues with using it, you can join our WeChat group, and we can help you take a look.

[xs1997zju]

Thanks, but in colossalAI PRs, you put SharFormer example of llama, and says support llama1/2. This exaggerates the functionality of the codebase, causing confusion and increasing debugging costs for users. We hope you can control the level of exaggeration in your PRs. Your project is indeed excellent, but the promotion is too excessive.

[flybird11111]

Thanks, but in colossalAI PRs, you put SharFormer example of llama, and says support llama1/2. This exaggerates the functionality of the codebase, causing confusion and increasing debugging costs for users. We hope you can control the level of exaggeration in your PRs. Your project is indeed excellent, but the promotion is too excessive.

LlamaV2 is also supported.

[xs1997zju]

?? I just report the bug of useing pp_size=2 in the PR's example. Does we discuss the same thing?

[Fridge003]

🐛 Describe the bug

Llama2 HybridParallelPlugin train failed when pp_size>1 we modified the llama training example to use hybridparallel plugin but encountered such an error. The hidden_states is None in colossalai/shardformer/modeling/llama.py:61 in llama_model_forward

Environment

torch-'1.13.1+cu117', transformers-4.32.0, colossalAI-release-v0.3.2

Hi, this is a typical bug. When enabling pipeline parallel, please do not do forward/backward through model(input)/loss.backward(), rather, please call booster.execute_pipeline to do forward/backward. For concrete usage you can refer to ColossalAI/examples/language/bert/finetune.py.

Tutorials on the usage of HybridParallelPlugin and PipelineStageManager will be posted to our official website in the next few days.

The example of LlaMa2 is still under development, we apologize for that, and it will be released within a few days. Thank you for your issue !

[flybird11111]

?? I just report the bug of useing pp_size=2 in the PR's example. Does we discuss the same thing?

I understand, and we will keep an eye on this issue.

[xs1997zju]

Okay, thank you for the prompt response, appreciate it.