Skip to content

Feature/npu#10

Closed
FrankLeeeee wants to merge 17 commits intomainfrom
feature/npu
Closed

Feature/npu#10
FrankLeeeee wants to merge 17 commits intomainfrom
feature/npu

Conversation

@FrankLeeeee
Copy link
Copy Markdown
Owner

@FrankLeeeee FrankLeeeee commented Jan 18, 2024

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

FrankLeeeee and others added 17 commits November 30, 2023 13:25
@FrankLeeeee
[accelerator] init the accelerator module (hpcaitech#5129)
f4e72c9 
* [accelerator] init the accelerator module

* polish code

* polish code

* polish code

* polish code
@oahzxl
[npu] support triangle attention for llama (hpcaitech#5130)
d6df19b 
* update fused attn

* update spda

* tri attn

* update triangle

* import

* fix

* fix
@oahzxl
[npu] use extension for op builder (hpcaitech#5172)
dd2c28a 
* update extension

* update cpu adam

* update is

* add doc for cpu adam

* update kernel

* update commit

* update flash

* update memory efficient

* update flash attn

* update flash attention loader

* update api

* fix

* update doc

* update example time limit

* reverse change

* fix doc

* remove useless kernel

* fix

* not use warning

* update

* update
@ver217 @oahzxl
[npu] change device to accelerator api (hpcaitech#5239)
d202cc2 
* update accelerator

* fix timer

* fix amp

* update

* fix

* update bug

* add error raise

* fix autocast

* fix set device

* remove doc accelerator

* update doc

* update doc

* update doc

* use nullcontext

* update cpu

* update null context

* change time limit for example

* udpate

* update

* update

* update

* [npu] polish accelerator code

---------

Co-authored-by: Xuanlei Zhao <xuanlei.zhao@gmail.com>
Co-authored-by: zxl <43881818+oahzxl@users.noreply.github.com>
@FrankLeeeee
[hotfix] removed unused flag (hpcaitech#5242)
9102d65
@FrankLeeeee
[ci] fixed ddp test (hpcaitech#5254)
2b83418 
* [ci] fixed ddp test

* polish
@digger-yu
fix typo in applications/ColossalEval/README.md (hpcaitech#5250)
756c400
@flybird11111 @cwher
[ci] fix shardformer tests. (hpcaitech#5255)
e830ef9 
* fix ci

fix

* revert: revert p2p

* feat: add enable_metadata_cache option

* revert: enable t5 tests

---------

Co-authored-by: Wenhao Chen <cwher@outlook.com>
@binmakeswell
[doc] fix doc typo (hpcaitech#5256)
c174c4f 
* [doc] fix annotation display

* [doc] fix llama2 doc
@cwher
[hotfix]: add pp sanity check and fix mbs arg (hpcaitech#5268)
ef4f0ee 
* fix: fix misleading mbs arg

* feat: add pp sanity check

* fix: fix 1f1b sanity check
@FrankLeeeee
[workflow] fixed incomplete bash command (hpcaitech#5272)
04244aa
@FrankLeeeee
[workflow] fixed oom tests (hpcaitech#5275)
d69cd2e 
* [workflow] fixed oom tests

* polish

* polish

* polish
@flybird11111 @cwher
[ci] fix test_hybrid_parallel_plugin_checkpoint_io.py (hpcaitech#5276)
2a0558d 
* fix ci

fix

* fix test

* revert: revert p2p

* feat: add enable_metadata_cache option

* revert: enable t5 tests

* fix

---------

Co-authored-by: Wenhao Chen <cwher@outlook.com>
@flybird11111
[shardformer] hybridparallelplugin support gradients accumulation. (h…
46e0916 
…pcaitech#5246)

* support gradients acc

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

fix

* fix

fix

* fix

fix

fix
@KKZ20
[hotfix] Fix ShardFormer test execution path when using sequence para…
5d9a0ae 
…llelism (hpcaitech#5230)
@ver217
Merge branch 'main' into sync/npu
1484693
@FrankLeeeee
Merge pull request hpcaitech#5278 from ver217/sync/npu
d66e698 
[sync] sync npu branch with main
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@FrankLeeeee @oahzxl @ver217 @digger-yu @flybird11111 @binmakeswell @cwher @KKZ20