[BUG]: Chat GPT inference still gives error

[sdvies]

🐛 Describe the bug

[Referring to the #3061 bug]

I've obtained the actor_checkpoint_prompts.pt again with the new version of the train_prompts.py code and I'm still getting the same error:

RuntimeError: Error(s) in loading state_dict for BloomForCausalLM:
	Unexpected key(s) in state_dict: "transformer.h.2.input_layernorm.weight", "transformer.h.2.input_layernorm.bias", "transformer.h.2.self_attention.query_key_value.weight", "transformer.h.2.self_attention.query_key_value.bias", "transformer.h.2.self_attention.query_key_value.lora_A", "transformer.h.2.self_attention.query_key_value.lora_B" ...

Thank you!

Environment

No response

[ht-zhou]

Thanks for your feedback.
Please show your command to help us to reproduce the problem.

[ht-zhou]

You can follow the code above to train and inference.

[sdvies]

I've reproduced the ChatGPT Stages described in https://github.com/hpcaitech/ColossalAI/tree/main/applications/ChatGPT/examples#inference-exampleafter-stage3.

I've obtained the rm_checkpoint.pt from Stage 2 and I could obtain the actor_checkpoint_prompts.pt as well from Stage 3 (based on rm_checkpoint.pt).

The problem occurs when I try to run the inference process because the following error raises:

RuntimeError: Error(s) in loading state_dict for BloomForCausalLM:
	Unexpected key(s) in state_dict: "transformer.h.2.input_layernorm.weight", "transformer.h.2.input_layernorm.bias", "transformer.h.2.self_attention.query_key_value.weight", "transformer.h.2.self_attention.query_key_value.bias", "transformer.h.2.self_attention.query_key_value.lora_A", "transformer.h.2.self_attention.query_key_value.lora_B" ...

To reproduce

In order to obtain the rm_checkpoint.pt I executed the following:
python train_reward_model.py --pretrain bigscience/bloom-560m --model bloom --batch_size 1 --lora_rank 16 --strategy naive

The actor_checkpoint_prompts.pt was obtain by executing:

python train_prompts.py prompts.csv --pretrain bigscience/bloom-560m --strategy naive --model bloom --train_batch_size 1 --lora_rank 16 --experience_batch_size 1 --max_epochs

(I've used #3011 too in order to obtain the actor_checkpoint_prompts.pt using my own rm_checkpoint.py)

And finally I tried to do the inference by:
python inference.py --model_path actor_checkpoint_prompts.pt --pretrain bigscience/bloom-560m --model bloom

Thank you very much!

Environment

I include the environment information as follows:

• CUDA: 11.1
• cuDNN: 8
• Python: 3.8.8
• PyTorch: 1.13.1

[sarrahbbh]

@sdvies hii
Have you managed to fix the error? :/
I still get missing keys from the saved state_dict + the size mismatch between it and the pretrained model