Ra by jamesthesnake · Pull Request #48 · jamesthesnake/ColossalAI

jamesthesnake · 2023-05-27T15:49:04Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>

* [test] fix flop tensor test * [test] fix autochunk test * [test] fix lazyinit test * [devops] update torch version of CI * [devops] enable testmon * [devops] fix ci * [devops] fix ci * [test] fix checkpoint io test * [test] fix cluster test * [test] fix timm test * [devops] fix ci * [devops] fix ci * [devops] fix ci * [devops] fix ci * [devops] force sync to test ci * [test] skip fsdp test

Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>

…h#3668)

…3742) * fix typo applications/ and colossalai/ date 5.11 * fix typo colossalai/

* [devops] make build on PR run automatically * [devops] update build on pr condition

* [doc] add test info * [devops] update doc check ci * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] add debug info * [devops] remove debug info and update invalid doc * [devops] add essential comments

Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>

* [booster] update booster tutorials#3717 * [booster] update booster tutorials#3717, fix * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, update setup doc * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, rename colossalai booster.md * [booster] update booster tutorials#3717, fix * [booster] update booster tutorials#3717, fix * [booster] update tutorials#3717, update booster api doc * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, modify file * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3717, fix reference link * [booster] update tutorials#3713 * [booster] update tutorials#3713, modify file

* [mixed_precison] add naive amp demo * [mixed_precison] add naive amp demo

…h#3775) * [plugin] torch ddp plugin add save sharded model * [test] fix torch ddp ckpt io test * [test] fix torch ddp ckpt io test * [test] fix low level zero plugin test * [test] fix low level zero plugin test * [test] add debug info * [test] add debug info * [test] add debug info * [test] add debug info * [test] add debug info * [test] fix low level zero plugin test * [test] fix low level zero plugin test * [test] remove debug info

* [doc] add en cluster utils doc * [doc] add zh cluster utils doc * [doc] add cluster utils doc in sidebar

* [doc] add en booster plugins doc * [doc] add booster plugins doc in sidebar * [doc] add zh booster plugins doc * [doc] fix zh booster plugin translation * [doc] reoganize tutorials order of basic section * [devops] force sync to test ci

* [doc] add checkpoint related docstr for booster * [doc] add en checkpoint doc * [doc] add zh checkpoint doc * [doc] add booster checkpoint doc in sidebar * [doc] add cuation about ckpt for plugins * [doc] add doctest placeholder * [doc] add doctest placeholder * [doc] add doctest placeholder

…ch#3780) * [test] refactor torch ddp checkpoint test * [plugin] update low level zero optim checkpoint * [plugin] update gemini optim checkpoint

* [doc] add docstr for booster methods * [doc] fix autodoc

* Update colotensor_concept.md * Update configure_parallelization.md * Update define_your_config.md * Update engine_trainer.md * Update initialize_features.md * Update model_checkpoint.md * Update colotensor_concept.md * Update configure_parallelization.md * Update define_your_config.md * Update engine_trainer.md * Update initialize_features.md * Update model_checkpoint.md

* [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping document * [doc] update gradient clipping doc, fix sidebars.json * [doc] update gradient clipping doc, fix doc test

hpcaitech#3787) Co-authored-by: github-actions <github-actions@github.com>

* [docker] Add ARG VERSION to determine the Tag * [workflow] fixed the version in the release docker workflow --------- Co-authored-by: liuzeming <liuzeming@4paradigm.com>

* [workflow] fixed the docker build workflow * polish code

* [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, add sidebars * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, fix * [doc]update gradient accumulation, resolve comments * [doc]update gradient accumulation, resolve comments * fix

…tech#3783) * [mixed_precison] add naive amp demo * [mixed_precison] add naive amp demo * [api] add docstrings and initialization to apex amp, naive amp * [api] add docstring to apex amp/ naive amp * [api] add docstring to apex amp/ naive amp * [api] add docstring to apex amp/ naive amp * [api] add docstring to apex amp/ naive amp * [api] add docstring to apex amp/ naive amp * [api] add docstring to apex amp/ naive amp * [api] fix * [api] fix

* fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc.

[doc] update amp document

…#3808)

…ech#3825) * [workflow] changed to doc build to be on schedule and release * polish code

* add functions for gpt evaluation * add automatic eval Update eval.py * using jload and modify the type of answers1 and answers2 * Update eval.py Update eval.py * Update evaluator.py * support gpt evaluation * update readme.md update README.md update READNE.md modify readme.md * add Chinese example for config, battle prompt and evaluation prompt file * remove GPT-4 config * remove sample folder --------- Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com> Co-authored-by: Camille Zhong <44392324+Camille7777@users.noreply.github.com>

* fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc. * fix typo colossalai/auto_parallel autochunk fx/passes etc. * fix typo docs/ * change placememt_policy to placement_policy in docs/ and examples/

* [workflow] fixed testmon cache in build CI * polish code

* [doc] update meet_gemini.md * [doc] update meet_gemini.md * [doc] fix parentheses * [doc] fix parentheses * [doc] fix doc test * [doc] fix doc test * [doc] fix doc

* fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc. * fix typo colossalai/auto_parallel autochunk fx/passes etc. * fix typo docs/ * change placememt_policy to placement_policy in docs/ and examples/ * fix typo colossalai/ applications/

raa

digger-yu and others added 30 commits May 15, 2023 11:46

[NFC] fix typo applications/ and colossalai/ (hpcaitech#3735)

ad6460c

[booster] support torch fsdp plugin in booster (hpcaitech#3697)

b37797e

Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>

[booster] removed models that don't support fsdp (hpcaitech#3744)

6050f37

Co-authored-by: 纪少敏 <jishaomin@jishaomindeMBP.lan>

[fix] Add init to fix import error when importing _analyzer (hpcaitec…

7386c66

…h#3668)

[NFC] fix typo with colossalai/auto_parallel/tensor_shard (hpcaitech#…

1baeb39

…3742) * fix typo applications/ and colossalai/ date 5.11 * fix typo colossalai/

[devops] make build on PR run automatically (hpcaitech#3748)

c03bd7c

* [devops] make build on PR run automatically * [devops] update build on pr condition

[chat] fix bugs in stage 3 training (hpcaitech#3759)

0575983

Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>

[auto] fix install cmd (hpcaitech#3772)

15024e4

[doc] update hybrid parallelism doc (hpcaitech#3770)

48bd056

[amp] Add naive amp demo (hpcaitech#3774)

2703a37

* [mixed_precison] add naive amp demo * [mixed_precison] add naive amp demo

[doc] add tutorial for cluster utils (hpcaitech#3763)

5ce6c9d

* [doc] add en cluster utils doc * [doc] add zh cluster utils doc * [doc] add cluster utils doc in sidebar

[NFC] fix typo colossalai/amp auto_parallel autochunk (hpcaitech#3756)

32f81f1

[devops] fix doc test on pr (hpcaitech#3782)

b4788d6

[chat] add performance and tutorial (hpcaitech#3786)

ad2cf58

[plugin] a workaround for zero plugins' optimizer checkpoint (hpcaite…

3c07a28

…ch#3780) * [test] refactor torch ddp checkpoint test * [plugin] update low level zero optim checkpoint * [plugin] update gemini optim checkpoint

[doc] add booster docstring and fix autodoc (hpcaitech#3789)

72688ad

* [doc] add docstr for booster methods * [doc] fix autodoc

[format] applied code formatting on changed files in pull request 3786 (

62c7e67

hpcaitech#3787) Co-authored-by: github-actions <github-actions@github.com>

Fix/docker action (hpcaitech#3266)

4d29c0f

* [docker] Add ARG VERSION to determine the Tag * [workflow] fixed the version in the release docker workflow --------- Co-authored-by: liuzeming <liuzeming@4paradigm.com>

[workflow] fixed the docker build workflow (hpcaitech#3794)

788e07d

* [workflow] fixed the docker build workflow * polish code

fixed the example docstring for booster (hpcaitech#3795)

f5c425c

[workflow] enable testing for develop & feature branch (hpcaitech#3801)

ad93c73

FrankLeeeee and others added 29 commits May 23, 2023 11:57

[test] fixed lazy init test import error (hpcaitech#3799)

615e2e5

[NFC]fix typo colossalai/auto_parallel nn utils etc. (hpcaitech#3779)

9265f2d

* fix typo colossalai/autochunk auto_parallel amp * fix typo colossalai/auto_parallel nn utils etc.

[doc] update amp document

8c62e50

[doc] update amp document

1167bf5

[doc] update amp document

a520610

[doc] add removed warning

75272ef

[doc] add removed change of config.py

c425a69

[booster] torch fsdp fix ckpt (hpcaitech#3788)

6b305a9

[doc] add warning about fsdp plugin (hpcaitech#3813)

19d1530

[workflow] enblaed doc build from a forked repo (hpcaitech#3815)

1e3b64f

[doc]fix

8aa1fb2

[doc]fix

278fcbc

Merge pull request hpcaitech#3810 from jiangmingyan/amp

725365f

[doc] update amp document

fix typo colossalai/auto_parallel autochunk fx/passes etc. (hpcaitech…

7f8203a

…#3808)

[Docker] Fix a couple of build issues (hpcaitech#3691)

269150b

[workflow] changed to doc build to be on schedule and release (hpcait…

05b8a8d

…ech#3825) * [workflow] changed to doc build to be on schedule and release * polish code

fix typo docs/

e90fdb1

[workflow] fixed testmon cache in build CI (hpcaitech#3806)

84500b7

* [workflow] fixed testmon cache in build CI * polish code

[booster] add warning for torch fsdp plugin doc (hpcaitech#3833)

3229f93

[workflow] supported test on CUDA 10.2 (hpcaitech#3841)

54e97ed

[doc] update document of gemini instruction. (hpcaitech#3842)

a64df3f

* [doc] update meet_gemini.md * [doc] update meet_gemini.md * [doc] fix parentheses * [doc] fix parentheses * [doc] fix doc test * [doc] fix doc test * [doc] fix doc

[release] bump to v0.3.0 (hpcaitech#3830)

d42b1be

[workflow] fixed workflow check for docker build (hpcaitech#3849)

ae959a7

[doc] update nvme offload documents. (hpcaitech#3850)

b047487

Merge pull request #47 from hpcaitech/main

815f9bb

raa

jamesthesnake merged commit 9fd12c6 into main May 27, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ra#48

Ra#48
jamesthesnake merged 59 commits intomainfrom
ra

jamesthesnake commented May 27, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

Conversation

jamesthesnake commented May 27, 2023

📌 Checklist before creating the PR

🚨 Issue number

📝 What does this PR do?

💥 Checklist before requesting a review

⭐️ Do you enjoy contributing to Colossal-AI?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants