[BUG]: inference.py

[JingxinLee]

🐛 Describe the bug

python inference.py --model_path ./actor_checkpoint_prompts.pt --pretrain bloom-560m --model bloom 

 size mismatch for transformer.ln_f.weight: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
 size mismatch for transformer.ln_f.bias: copying a param with shape torch.Size([768]) from checkpoint, the shape in current model is torch.Size([1024]).
 size mismatch for lm_head.weight: copying a param with shape torch.Size([50257, 768]) from checkpoint, the shape in current model is torch.Size([250880, 1024]).

Environment

No response

[ht-zhou]

Thanks for your feedback, please update your code to our newest pr, this problem has been solved.

[JingxinLee]

Thanks for your feedback, please update your code to our newest pr, this problem has been solved.

I am sure I used the latest code. Problem still exists.

[ht-zhou]

There may be some problems with the old ckpt.
I suggest you to train a new one for few epochs to test.

[1a2cjitenfei]

There may be some problems with the old ckpt. I suggest you to train a new one for few epochs to test.

How to realize stage1 of chatGPT training? Can you explain it in detail

[ht-zhou]

? Can you explain it in detail

You can use a human-writing dataset to fine tune a pretrained language model like Bloom/Lamma.

[Issues-translate-bot]

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿

---

? Can you explain it in detail

You can use a human-writing dataset to fine tune a pretrained language model like Bloom/Lamma.

[JingxinLee]

There may be some problems with the old ckpt. I suggest you to train a new one for few epochs to test.

I trained new one. And this problem still exist.

[1a2cjitenfei]

? Can you explain it in detail

You can use a human-writing dataset to fine tune a pretrained language model like Bloom/Lamma.

How to operate it? Using the hugging face? However, the GPU video memory that can be supported by using the hugging face is ColossalAI 1/10. This problem will come back. It is recommended to add stage1

[ht-zhou]

There may be some problems with the old ckpt. I suggest you to train a new one for few epochs to test.

I trained new one. And this problem still exist.
You can refer to our test_ci in https://github.com/hpcaitech/ColossalAI/blob/main/applications/ChatGPT/examples/test_ci.sh
to try. We have already completed the ci process of this application, so it may not have this problem because the code has passed this ci.

[ht-zhou]

? Can you explain it in detail

You can use a human-writing dataset to fine tune a pretrained language model like Bloom/Lamma.

How to operate it? Using the hugging face? However, the GPU video memory that can be supported by using the hugging face is ColossalAI 1/10. This problem will come back. It is recommended to add stage1

Ok, we will support stage1 in our example soon.

[sarrahbbh]

@JingxinLee Hello, sorry to disturb
have you found a solution to this? I'm facing the same issue for days now.. (+ missing keys from the PPO save state_dict)