Skip to content

merged#2

Merged
varshith-Git merged 83 commits intovarshith-Git:mainfrom
hpcaitech:main
Aug 7, 2024
Merged

merged#2
varshith-Git merged 83 commits intovarshith-Git:mainfrom
hpcaitech:main

Conversation

@varshith-Git
Copy link
Copy Markdown
Owner

@varshith-Git varshith-Git commented Aug 6, 2024

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs
  • I have installed pre-commit: pip install pre-commit && pre-commit install

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

ver217 and others added 30 commits July 18, 2024 15:33
@ver217
[plugin] support all-gather overlap for hybrid parallel (#5919)
e861279 
* [plugin] fixed all-gather overlap support for hybrid parallel
@YeAnbang
add kto
09d5ffc
@YeAnbang
Merge branch 'main' of https://github.com/hpcaitech/ColossalAI into kto
845ea72
@YeAnbang
fix style, add kto data sample
544b7a3
@Edenzzzz
[Examples] Add lazy init to OPT and GPT examples (#5924)
8cc8f64 
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
@TongLi3701
[ColossalChat] Hotfix for ColossalChat (#5910)
f585d4e 
* add ignore and tiny llama

* fix path issue

* run style

* fix issue

* update bash

* add ignore and tiny llama

* fix path issue

* run style

* fix issue

* update bash

* fix ddp issue

* add Qwen 1.5 32B
@TongLi3701
Merge branch 'main' into kto
d08c99b
@YeAnbang
refactor tokenization
d49550f
@YeAnbang
Merge branch 'kto' of https://github.com/hpcaitech/ColossalAI into kto
150505c
@zhurunhua @pre-commit-ci
[FIX BUG] UnboundLocalError: cannot access local variable 'default_co…
4ec17a7 
…nversation' where it is not associated with a value (#5931)

* cannot access local variable 'default_conversation' where it is not associated with a value

set default value for 'default_conversation'

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@YeAnbang
fix test data
c5f582f
@YeAnbang
refactor evaluation
12fe8b5
@YeAnbang
remove real data path
b0e15d5
@YeAnbang
remove real data path
9688e19
@insujang
Add n_fused as an input from native_module (#5894)
a521ffc
@flymin
[FIX BUG] convert env param to int in (#5934)
5fb958c
@Edenzzzz
[Hotfix] Fix ZeRO typo #5936
2069472 
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
@zhurunhua @pre-commit-ci
[Feature] Add a switch to control whether the model checkpoint needs …
ad35a98 
…to be saved after each epoch ends (#5941)

* Add a switch to control whether the model checkpoint needs to be saved after each epoch ends

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@YeAnbang
fix style
8a3ff4f
@YeAnbang
fix style
de1bf08
@YeAnbang
fix style
6fd9e86
@YeAnbang
Merge pull request #5922 from hpcaitech/kto
c8332b9 
[Chat] Add KTO
@ver217
[shardformer] hotfix attn mask (#5945)
9664b1b
@ver217
[shardformer] hotfix attn mask (#5947)
7b38964
@LRY89757
[Feat] Distrifusion Acceleration Support for Diffusion Inference (#5895)
bcf0181 
* Distrifusion Support source

* comp comm overlap optimization

* sd3 benchmark

* pixart distrifusion bug fix

* sd3 bug fix and benchmark

* generation bug fix

* naming fix

* add docstring, fix counter and shape error

* add reference

* readme and requirement
@ver217
[zero] hotfix update master params (#5951)
0608921
@ver217
[release] update version (#5952)
09c5f72
@YeAnbang
[Chat] Fix lora (#5946)
30f4e31 
* fix merging

* remove filepath

* fix style
@YeAnbang
Update README.md (#5958)
66fbf2e
@TongLi3701
[hotfix] Remove unused plan section (#5957)
1aeb5e8 
* remove readme

* fix readme

* update
botbw and others added 29 commits August 1, 2024 10:06
@botbw @ver217
[moe] finalize test (no pp)
7077d38
@botbw @ver217
[moe] full test for deepseek and mixtral (pp + sp to fix)
803878b
@botbw @ver217
[chore] minor fix after rebase
46037c2
@pre-commit-ci @ver217
[pre-commit.ci] auto fixes from pre-commit.com hooks
52d346f 
for more information, see https://pre-commit.ci
@botbw @ver217
[chore] solve moe ckpt test failure and some other arg pass failure
70c9924
@botbw @ver217
[moe] remove ops
74b03de
@botbw @ver217
[test] fix test: test_zero1_2
067e18f
@botbw @ver217
[bug] fix: somehow logger hangs the program
96d0fbc
@Hz188 @ver217
[moe] deepseek moe sp support
b2952a5
@botbw @ver217
[test] add check
6c39f0b
@botbw @ver217
[deepseek] replace attn (a workaround for bug in transformers)
c3dc9b4
@Hz188 @ver217
[misc] skip redunant test
59bcf56
@Hz188 @ver217
[misc] remove debug/print code
034020b
@botbw @ver217
[moe] refactor mesh assignment
cb01c0d
@botbw @ver217
Revert "[moe] implement submesh initialization"
5b4c123 
This reverts commit 2f9bce6.
@botbw @ver217
[chore] change moe_pg_mesh to private
606b089
@Hz188 @ver217
[misc] remove incompatible test config
12d043c
@Hz188 @ver217
[misc] fix ci failure: change default value to false in moe plugin
70793ce
@Hz188 @ver217
[misc] remove useless condition
7e737df
@botbw @ver217
[chore] docstring
f7c5485
@botbw @ver217
[moe] remove force_overlap_comm flag and add warning instead
7bedd03
@botbw @ver217
[doc] add MoeHybridParallelPlugin docstring
65daa87
@botbw @ver217
[moe] solve dp axis issue
d1d1ab8
@botbw @ver217
[chore] remove redundant test case, print string & reduce test tokens
62cdac6
@TongLi3701 @MichelleMa8 @pre-commit-ci
[feat] Dist Loader for Eval (#5950)
19d1510 
* support auto distributed data loader

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* support auto distributed data loader

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

* fix tp error

* remove unused parameters

* remove unused

* update inference

* update docs

* update inference

---------

Co-authored-by: Michelle <qianranma8@gmail.com>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
@wangbluo
[lora] lora support hybrid parallel plugin (#5956)
75c9636 
* lora support hybrid plugin

* fix

* fix

* fix

* fix
@YeAnbang
Support overall loss, update KTO logging
0b2d55c
@YeAnbang
Merge pull request #5962 from hpcaitech/colossalchat
fe71917 
[Chat] Support overall loss, update KTO logging
@Edenzzzz
[Docs] clarify launch port
9179d40 
Co-authored-by: Edenzzzz <wtan45@wisc.edu>
@varshith-Git varshith-Git merged commit e07e3e6 into varshith-Git:main Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.