fz by jamesthesnake · Pull Request #62 · jamesthesnake/ColossalAI

jamesthesnake · 2023-06-13T03:58:38Z

📌 Checklist before creating the PR

I have created an issue for this PR for traceability
The title follows the standard format: [doc/gemini/tensor/...]: A concise description
I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

I have linked my PR to an issue (instruction)
My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
I have performed a self-review of my code
I have added thorough tests.
I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

🌝 Yes, I do.
🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

* [dtensor] polish sharding spec docstring * [dtensor] polish sharding spec example docstring

[sync] update develop with main

[sync] sync feature/dtensor with develop

* [doc] add lazy init en doc * [doc] add lazy init zh doc * [doc] add lazy init doc in sidebar * [doc] add lazy init doc test * [doc] fix lazy init doc link

[feature] updated device mesh and dtensor

* init shardformer code structure * add implement of sharder (inject and replace) * add implement of replace layer to colossal layer * separate different layer policy, add some notion * implement 1d and 2d slicer, can tell col or row * fix bug when slicing and inject model * fix some bug; add inference test example

) * init shardformer code structure * add implement of sharder (inject and replace) * add implement of replace layer to colossal layer * separate different layer policy, add some notion * implement 1d and 2d slicer, can tell col or row * fix bug when slicing and inject model * fix some bug; add inference test example * add share weight and train example * add train * add docstring and readme * add docstring for other files * pre-commit

* [shardformer] refactored the user api * polish code

* update readme with modules content * remove img

) * add dropout layer, add dropout test * modify seed manager as context manager * add a copy of col_nn.layer * add dist_crossentropy loss; separate module test * polish the code * fix dist crossentropy loss

…3883) * add gpt2 policy and modify shard and slicer to support * remove unused code * polish code

Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>

[example] Adding an example of training dreambooth with the new booster API

[sync] sync feature/shardformer with develop

…hardformer Revert "[sync] sync feature/shardformer with develop"

ver217 and others added 30 commits May 25, 2023 13:09

[dtensor] polish sharding spec docstring (#3838)

7c9f2ed

* [dtensor] polish sharding spec docstring * [dtensor] polish sharding spec example docstring

Modify torch version requirement to adapt torch 2.0

46503c3

Add a new example of Dreambooth training using the booster API

60ec33b

roll back

42e3232

modify path

25447d4

update performance evaluation

176010f

update shell file

b56c7f4

fixing insecure hash function

1c1f71c

change directory

b29e1f0

fixed model saving bugs

d3379f0

fixed port

79c9f77

fixed port

b4437e8

modify file path

4fc8bc6

Merge pull request #3915 from FrankLeeeee/update/develop

c622bb3

[sync] update develop with main

Merge pull request #3916 from FrankLeeeee/sync/dtensor-with-develop

d51e83d

[sync] sync feature/dtensor with develop

modify shell for check

c94a335

[doc] add lazy init tutorial (#3922)

12c90db

* [doc] add lazy init en doc * [doc] add lazy init zh doc * [doc] add lazy init doc in sidebar * [doc] add lazy init doc test * [doc] fix lazy init doc link

[nfc] fix typo colossalai/zero (#3923)

de0d7df

[devops] update torch version in compability test (#3919)

9166988

[dtensor] updated api and doc (#3845)

eb39154

modify shell for check

cf4792c

[example] update opt example using booster api (#3918)

e417dd0

modify shell for check

039854b

modify shell for check

49567d5

modify shell for check

730a092

fix typo examples/community/roberta (#3925)

407aa48

Merge pull request #3926 from hpcaitech/feature/dtensor

a98e16e

[feature] updated device mesh and dtensor

modify shell for check

9b5e7ce

FrankLeeeee and others added 15 commits June 8, 2023 15:01

[shardformer] updated readme (#3827)

bc19024

[shardformer] refactored the user api (#3828)

537a52b

* [shardformer] refactored the user api * polish code

[shardformer] update readme with modules implement doc (#3834)

997544c

* update readme with modules content * remove img

[shardformer] add Dropout layer support different dropout pattern (#3856

21a3915

) * add dropout layer, add dropout test * modify seed manager as context manager * add a copy of col_nn.layer * add dist_crossentropy loss; separate module test * polish the code * fix dist crossentropy loss

update README (#3909)

6370a93

[shardformer] add gpt2 policy and modify shard and slicer to support (#…

ef15377

…3883) * add gpt2 policy and modify shard and slicer to support * remove unused code * polish code

fix typo examples and docs (#3932)

33eef71

support UniEval and add CHRF metric (#3924)

21c4c0b

Co-authored-by: Yuanchen Xu <yuanchen.xu00@gmail.com>

Merge pull request #3905 from MaruyamaAya/dreambooth

e277534

[example] Adding an example of training dreambooth with the new booster API

Merge pull request #3931 from FrankLeeeee/sync/develop-to-shardformer

24651fd

[sync] sync feature/shardformer with develop

Revert "[sync] sync feature/shardformer with develop"

ddcf58c

Merge pull request #3942 from hpcaitech/revert-3931-sync/develop-to-s…

bd2c7c3

…hardformer Revert "[sync] sync feature/shardformer with develop"

fix typo tests/ (#3936)

e61ffc7

fix typo .github/workflows/scripts/ (#3946)

1aadeed

[example] update ViT example using booster api (#3940)

b3ab7fb

jamesthesnake merged commit eabae7a into jamesthesnake:l Jun 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fz#62

fz#62
jamesthesnake merged 45 commits intojamesthesnake:lfrom
hpcaitech:main

jamesthesnake commented Jun 13, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

Conversation

jamesthesnake commented Jun 13, 2023

📌 Checklist before creating the PR

🚨 Issue number

📝 What does this PR do?

💥 Checklist before requesting a review

⭐️ Do you enjoy contributing to Colossal-AI?

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants