[DOC]: optimize readme of ChatGPT

[ver217]

📚 The doc issue

A new user is hard to start training after reading the readme.

I list some FAQs which users may concern:

• How to prepare training dataset? What does the dataset look like?
• How to save models?
• How to use the fine-tuned models to generate?
• How to use LoRA? How does it affect training?

[feifeibear]

The biggest question in my mind is:
How to initialize optimizer states when finetuning a pre-trained model?

[Gy-Lu]

Hi, I found this picture a little confusing. I think if there is some description, it would be much better.

[feifeibear]

Hi, I found this picture a little confusing. I think if there is some description, it would be much better.

Yes, at least tell the readers: the green part is the trainable parameters and the rest parts are frozen parameters.

[Fazziekey]

I think we need a detail blog to tell user what is reward model, critic model, sft-model

[Gy-Lu]

This picture is from InstructGPT, maybe there should be copyright information?

[Fazziekey]

I think instructs LM is better than chatgpt, chatgpt is a serving, instruct LM is an algorithm

[ht-zhou]

This picture is from InstructGPT, maybe there should be copyright information?

Thanks for your remind, and we mention the reference here.

[binmakeswell]

##Todo

• implement PPO training and fine-tuning

We should also mentioned PPO fine-tuning. This looks a lot like we haven't implemented finetune function yet, because we plan to implement PPO-ptx fine-tuning.

[binmakeswell]

Clear error message or doc to remind user 'pip install .'

[binmakeswell]

TODO ckpt
Seriously affect the actual use of the user, need to be fixed as a priority

[ht-zhou]

Hi, I found this picture a little confusing. I think if there is some description, it would be much better.

Yes, at least tell the readers: the green part is the trainable parameters and the rest parts are frozen parameters.

I've already completed a detailed doc to explain about the training process, it will be released soon.