Skip to content

Ra#13

Merged
jamesthesnake merged 30 commits intomainfrom
ra
Apr 8, 2023
Merged

Ra#13
jamesthesnake merged 30 commits intomainfrom
ra

Conversation

@jamesthesnake
Copy link
Copy Markdown
Owner

@jamesthesnake jamesthesnake commented Apr 8, 2023

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

Camille7777 and others added 30 commits April 3, 2023 10:11
@Camille7777
[chatgpt] add pre-trained model RoBERTa for RLHF stage 2 & 3 (hpcaite…
3041286 
…ch#3223)

* Add RoBERTa for RLHF Stage 2 & 3 (test)

RoBERTa for RLHF Stage 2 & 3 (still in testing)

* Revert "Add RoBERTa for RLHF Stage 2 & 3 (test)"

This reverts commit 06741d8.

* Add RoBERTa for RLHF stage 2 & 3

1. add roberta folder under model folder
2. add  roberta option in train_reward_model.py
3. add some test in testci

* add test for reward model training

* Update test_ci.sh

* Revert "Update test_ci.sh"

This reverts commit 9c7352b.

* Add RoBERTa for RLHF Stage 2 & 3 (test)

RoBERTa for RLHF Stage 2 & 3 (still in testing)

* Revert "Add RoBERTa for RLHF Stage 2 & 3 (test)"

This reverts commit 06741d8.

* Add RoBERTa for RLHF stage 2 & 3

1. add roberta folder under model folder
2. add  roberta option in train_reward_model.py
3. add some test in testci

* Update test_ci.sh

* Revert "Update test_ci.sh"

This reverts commit 9c7352b.

* update roberta with coati
@FrankLeeeee
[test] fixed gemini plugin test (hpcaitech#3411)
638a07a 
* [test] fixed gemini plugin test

* polish code

* polish code
@chengeharrison
[chat]fix sft training for bloom, gpt and opt (hpcaitech#3418)
b09adff 
fix sft training for bloom, gpt and opt
@ver217
[zero] reorganize zero/gemini folder structure (hpcaitech#3424)
26b7aac 
* [zero] refactor low-level zero folder structure

* [zero] fix legacy zero import path

* [zero] fix legacy zero import path

* [zero] remove useless import

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor gemini folder structure

* [zero] refactor legacy zero import path

* [zero] fix test import path

* [zero] fix test

* [zero] fix circular import

* [zero] update import
@FrankLeeeee
[checkpoint] refactored the API and added safetensors support (hpcait…
1beb85c 
…ech#3427)

* [checkpoint] refactored the API and added safetensors support

* polish code
@chengeharrison
fix save_model inin naive and ddp strategy (hpcaitech#3436)
773955a 
Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>
@ver217
[example] update examples related to zero/gemini (hpcaitech#3431)
573af84 
* [zero] update legacy import

* [zero] update examples

* [example] fix opt tutorial

* [example] fix opt tutorial

* [example] fix opt tutorial

* [example] fix opt tutorial

* [example] fix import
@YuliangLiu0306
[autoparallel]integrate auto parallel feature with new tracer (hpcait…
ffcdbf0 
…ech#3408)

* [autoparallel] integrate new analyzer in module level

* unify the profiling method

* polish

* fix no codegen bug

* fix pass bug

* fix liveness test

* polish
@chengeharrison
fix save_model indent error in ppo trainer (hpcaitech#3450)
b923139 
Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>
@nijkah
[format] Run lint on colossalai.engine (hpcaitech#3367)
46c009d
@Camille7777
[Chat] fix the tokenizer "int too big to convert" error in SFT traini…
72cb4dd 
…ng (hpcaitech#3453)

* Add RoBERTa for RLHF Stage 2 & 3 (test)

RoBERTa for RLHF Stage 2 & 3 (still in testing)

* Revert "Add RoBERTa for RLHF Stage 2 & 3 (test)"

This reverts commit 06741d8.

* Add RoBERTa for RLHF stage 2 & 3

1. add roberta folder under model folder
2. add  roberta option in train_reward_model.py
3. add some test in testci

* Update test_ci.sh

* Revert "Update test_ci.sh"

This reverts commit 9c7352b.

* Add RoBERTa for RLHF Stage 2 & 3 (test)

RoBERTa for RLHF Stage 2 & 3 (still in testing)

* Revert "Add RoBERTa for RLHF Stage 2 & 3 (test)"

This reverts commit 06741d8.

* Add RoBERTa for RLHF stage 2 & 3

1. add roberta folder under model folder
2. add  roberta option in train_reward_model.py
3. add some test in testci

* Update test_ci.sh

* Revert "Update test_ci.sh"

This reverts commit 9c7352b.

* update roberta with coati

* chat ci update

* Revert "chat ci update"

This reverts commit 17ae7ae.

* [Chat] fix the tokenizer "int too big to convert" error in SFT training

fix the tokenizer error during SFT training using Bloom and OPT
@ver217
[test] reorganize zero/gemini tests (hpcaitech#3445)
933048a
@yhna940
Fix typo (hpcaitech#3448)
8f740de
@FrankLeeeee
[booster] fixed the torch ddp plugin with the new checkpoint api (hpc…
7d8d825 
…aitech#3442)
@kingkingofall
[chat]fix readme (hpcaitech#3429)
57a3c4d 
* fix stage 2

fix stage 2

* add torch
@Dr-Corgi
[chat]fix save_model(hpcaitech#3377)
73afb63 
The function save_model should be a part of PPOTrainer.
@yynil
[Chat]Add Peft support & fix the ptx bug (hpcaitech#3433)
62f4e2e 
* Update ppo.py

Fix the bug of fetching wrong batch data

* Add peft model support in SFT and Prompts training

In stage-1 and stage-3, the peft model supports are added. So the trained artifacts will be only a small lora additions instead of the whole bunch of files.

* Delete test_prompts.txt

* Delete test_pretrained.txt

* Move the peft stuffs to a community folder.

* Move the demo sft to community

* delete dirty files

* Add instructions to install peft using source

* Remove Chinese comments

* remove the Chinese comments
@FrankLeeeee
[test] refactor tests with spawn (hpcaitech#3452)
80eba05 
* [test] added spawn decorator

* polish code

* polish code

* polish code

* polish code

* polish code

* polish code
@Fazziekey
add community example dictionary (hpcaitech#3465)
6afeb12
@flybird11111
[checkpoint] support huggingface style sharded checkpoint (hpcaitech#…
52a933e 
…3461)

* [checkpoint] support huggingface style sharded checkpoint

* [checkpoint] support huggingface style sharded checkpoint

* [checkpoint] support huggingface style sharded checkpoint

* [checkpoint] support huggingface style sharded checkpoint

* [checkpoint] support huggingface style sharded checkpoint

---------

Co-authored-by: luchen <luchen@luchendeMBP.lan>
@FrankLeeeee
[doc] updated contributor list (hpcaitech#3474)
4e99893
@NatalieC323
[dreambooth] fixing the incompatibity in requirements.txt (hpcaitech#…
c701b77 
…3190) (hpcaitech#3378)

* Update requirements.txt

* Update environment.yaml

* Update README.md

* Update environment.yaml

* Update README.md

* Update README.md

* Delete requirements_colossalai.txt

* Update requirements.txt

* Update README.md
@binmakeswell
[chat] fix stage3 PPO sample sh command (hpcaitech#3477)
891b8e7
@NatalieC323
Revert "[dreambooth] fixing the incompatibity in requirements.txt (hp…
fb8fae6 
…caitech#3190) (hpcaitech#3378)" (hpcaitech#3481)
@mandoxzhang
[example] update roberta with newer ColossalAI (hpcaitech#3472)
ab5fd12 
* update roberta example

* update roberta example
@mandoxzhang
[example] remove redundant texts & update roberta (hpcaitech#3493)
8f2c55f 
* update roberta example

* update roberta example

* modify conflict & update roberta
@gongel
[coati] Fix LlamaCritic (hpcaitech#3475)
a7ca297 
* mv LlamaForCausalLM to LlamaModel

* rm unused imports

---------

Co-authored-by: gongenlei <gongenlei@baidu.com>
@jamesthesnake
Merge pull request #10 from hpcaitech/main
634f32f 
k
@jamesthesnake
Merge pull request #11 from jamesthesnake/l
25e423f 
L
@jamesthesnake
Merge pull request #12 from jamesthesnake/co
74d9714 
Co
@jamesthesnake jamesthesnake merged commit f1bab70 into main Apr 8, 2023
jamesthesnake pushed a commit that referenced this pull request Jun 7, 2023
@ver217 @CsRic
[chat] add distributed PPO trainer (hpcaitech#3740)
b5f0566 
* Detached ppo (#9)

* run the base

* working on dist ppo

* sync

* detached trainer

* update detached trainer. no maker update function

* facing init problem

* 1 maker 1 trainer detached run. but no model update

* facing cuda problem

* fix save functions

* verified maker update

* nothing

* add ignore

* analyize loss issue

* remove some debug codes

* facing 2m1t stuck issue

* 2m1t verified

* do not use torchrun

* working on 2m2t

* working on 2m2t

* initialize strategy in ray actor env

* facing actor's init order issue

* facing ddp model update issue (need unwarp ddp)

* unwrap ddp actor

* checking 1m2t stuck problem

* nothing

* set timeout for trainer choosing. It solves the stuck problem!

* delete some debug output

* rename to sync with upstream

* rename to sync with upstream

* coati rename

* nothing

* I am going to detach the replaybuffer from trainer and make it a Ray Actor. Two benefits: 1. support TP trainer. 2. asynchronized buffer operations

* experience_maker_holder performs target-revolving _send_experience() instead of length comparison.

* move code to ray subfolder

* working on pipeline inference

* apply comments

* working on pipeline strategy. in progress.

* remove pipeline code. clean this branch

* update remote parameters by state_dict. no test

* nothing

* state_dict sharding transfer

* merge debug branch

* gemini _unwrap_model fix

* simplify code

* simplify code & fix LoRALinear AttributeError

* critic unwrapped state_dict

---------

Co-authored-by: csric <richcsr256@gmail.com>

* [chat] add perfomance evaluator and fix bugs (#10)

* [chat] add performance evaluator for ray

* [chat] refactor debug arg

* [chat] support hf config

* [chat] fix generation

* [chat] add 1mmt dummy example

* [chat] fix gemini ckpt

* split experience to send (#11)

Co-authored-by: csric <richcsr256@gmail.com>

* [chat] refactor trainer and maker (#12)

* [chat] refactor experience maker holder

* [chat] refactor model init

* [chat] refactor trainer args

* [chat] refactor model init

* [chat] refactor trainer

* [chat] refactor experience sending logic and training loop args (#13)

* [chat] refactor experience send logic

* [chat] refactor trainer

* [chat] refactor trainer

* [chat] refactor experience maker

* [chat] refactor pbar

* [chat] refactor example folder (#14)

* [chat] support quant (#15)

* [chat] add quant

* [chat] add quant example

* prompt example (#16)

* prompt example

* prompt load csv data

* remove legacy try

---------

Co-authored-by: csric <richcsr256@gmail.com>

* [chat] add mmmt dummy example and refactor experience sending (#17)

* [chat] add mmmt dummy example

* [chat] refactor naive strategy

* [chat] fix struck problem

* [chat] fix naive strategy

* [chat] optimize experience maker sending logic

* [chat] refactor sending assignment

* [chat] refactor performance evaluator (#18)

* Prompt Example & requires_grad state_dict & sharding state_dict (#19)

* prompt example

* prompt load csv data

* remove legacy try

* maker models require_grad set to False

* working on zero redundancy update

* mmmt_prompt example; naive strategy requires_grad state_dict & sharding; maker model requires_no_grad.

* remove legacy examples

* remove legacy examples

* remove replay buffer tp state. bad design

---------

Co-authored-by: csric <richcsr256@gmail.com>

* state_dict sending adapts to new unwrap function (#20)

* prompt example

* prompt load csv data

* remove legacy try

* maker models require_grad set to False

* working on zero redundancy update

* mmmt_prompt example; naive strategy requires_grad state_dict & sharding; maker model requires_no_grad.

* remove legacy examples

* remove legacy examples

* remove replay buffer tp state. bad design

* opt benchmark

* better script

* nothing

* [chat] strategy refactor unwrap model

* [chat] strategy refactor save model

* [chat] add docstr

* [chat] refactor trainer save model

* [chat] fix strategy typing

* [chat] refactor trainer save model

* [chat] update readme

* [chat] fix unit test

* working on lora reconstruction

* state_dict sending adapts to new unwrap function

* remove comments

---------

Co-authored-by: csric <richcsr256@gmail.com>
Co-authored-by: ver217 <lhx0217@gmail.com>

* [chat-ray] add readme (#21)

* add readme

* transparent graph

* add note background

---------

Co-authored-by: csric <richcsr256@gmail.com>

* [chat] get images from url (#22)

* Refactor/chat ray (#23)

* [chat] lora add todo

* [chat] remove unused pipeline strategy

* [chat] refactor example structure

* [chat] setup ci for ray

* [chat-ray] Support LoRA trainer. LoRA weights reconstruction. (#24)

* lora support prototype

* lora support

* 1mmt lora & remove useless code

---------

Co-authored-by: csric <richcsr256@gmail.com>

* [chat] fix test ci for ray

* [chat] fix test ci requirements for ray

* [chat] fix ray runtime env

* [chat] fix ray runtime env

* [chat] fix example ci docker args

* [chat] add debug info in trainer

* [chat] add nccl debug info

* [chat] skip ray test

* [doc] fix typo

---------

Co-authored-by: csric <59389055+CsRic@users.noreply.github.com>
Co-authored-by: csric <richcsr256@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.