Skip to content

mchange bug#188

Merged
jamesthesnake merged 13 commits intojamesthesnake:mostfrom
hpcaitech:main
Oct 25, 2023
Merged

mchange bug#188
jamesthesnake merged 13 commits intojamesthesnake:mostfrom
hpcaitech:main

Conversation

@jamesthesnake
Copy link
Copy Markdown
Owner

@jamesthesnake jamesthesnake commented Oct 25, 2023

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

Link this PR to your issue with words like fixed to automatically close the linked issue upon merge

e.g. fixed #1234, closed #1234, resolved #1234

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

Xu-Kai and others added 13 commits October 16, 2023 11:28
@Xu-Kai
[inference] Add smmoothquant for llama (#4904)
611a5a8 
* [inference] add int8 rotary embedding kernel for smoothquant (#4843)

* [inference] add smoothquant llama attention (#4850)

* add smoothquant llama attention

* remove uselss code

* remove useless code

* fix import error

* rename file name

* [inference] add silu linear fusion for smoothquant llama mlp  (#4853)

* add silu linear

* update skip condition

* catch smoothquant cuda lib exception

* prcocess exception for tests

* [inference] add llama mlp for smoothquant (#4854)

* add llama mlp for smoothquant

* fix down out scale

* remove duplicate lines

* add llama mlp check

* delete useless code

* [inference] add smoothquant llama (#4861)

* add smoothquant llama

* fix attention accuracy

* fix accuracy

* add kv cache and save pretrained

* refactor example

* delete smooth

* refactor code

* [inference] add smooth function and delete useless code for smoothquant (#4895)

* add smooth function and delete useless code

* update datasets

* remove duplicate import

* delete useless file

* refactor codes (#4902)

* rafactor code

* add license

* add torch-int and smoothquant license
@Orion-Zheng
Update flash_attention_patch.py
7768afb 
To be compatible with the new change in the Transformers library, where a new argument 'padding_mask' was added to forward function of attention layer.
huggingface/transformers#25598
@ver217
[kernel] support pure fp16 for cpu adam and update gemini optim tests (…
4f68b3f 
…#4921)

* [kernel] support pure fp16 for cpu adam (#4896)

* [kernel] fix cpu adam kernel for pure fp16 and update tests (#4919)

* [kernel] fix cpu adam

* [test] update gemini optim test
@github-actions
[format] applied code formatting on changed files in pull request 4908 (
a41cf88 
#4918)

Co-authored-by: github-actions <github-actions@github.com>
[gemini] support gradient accumulation (#4869)
21ba89c 
* add test

* fix no_sync bug in low level zero plugin

* fix test

* add argument for grad accum

* add grad accum in backward hook for gemini

* finish implementation, rewrite tests

* fix test

* skip stuck model in low level zero test

* update doc

* optimize communication & fix gradient checkpoint

* modify doc

* cleaning codes

* update cpu adam fp16 case
@ver217
[hotfix] fix torch 2.0 compatibility (#4936)
1f5d2e8 
* [hotfix] fix launch

* [test] fix test gemini optim

* [shardformer] fix vit
@KKZ20
[test] add no master test for low level zero plugin (#4934)
c7aa319
@github-actions
[format] applied code formatting on changed files in pull request 4820 (
486d06a 
#4886)

Co-authored-by: github-actions <github-actions@github.com>
@digger-yu
[nfc] fix some typo with colossalai/ docs/ etc. (#4920)
1100910
@tiandiao123 @lcq336 @CjhHa1
[Refactor] Integrated some lightllm kernels into token-attention (#4946)
3a41e83 
* add some req for inference

* clean codes

* add codes

* add some lightllm deps

* clean codes

* hello

* delete rms files

* add some comments

* add comments

* add doc

* add lightllm deps

* add lightllm cahtglm2 kernels

* add lightllm cahtglm2 kernels

* replace rotary embedding with lightllm kernel

* add some commnets

* add some comments

* add some comments

* add

* replace fwd kernel att1

* fix a arg

* add

* add

* fix token attention

* add some comments

* clean codes

* modify comments

* fix readme

* fix bug

* fix bug

---------

Co-authored-by: cuiqing.li <lixx336@gmail.com>
Co-authored-by: CjhHa1 <cjh18671720497@outlook.com>
@ver217
[test] merge old components to test to model zoo (#4945)
b8e770c 
* [test] add custom models in model zoo

* [test] update legacy test

* [test] update model zoo

* [test] update gemini test

* [test] remove components to test
@Xu-Kai
[inference] add reference and fix some bugs (#4937)
785802e 
* add reference and fix some bugs

* update gptq init

---------

Co-authored-by: Xu Kai <xukai16@foxamil.com>
@CjhHa1
[Inference]ADD Bench Chatglm2 script (#4963)
c6cd629 
* add bench chatglm

* fix bug and make utils

---------

Co-authored-by: CjhHa1 <cjh18671720497outlook.com>
@jamesthesnake jamesthesnake merged commit a000b61 into jamesthesnake:most Oct 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@jamesthesnake @Xu-Kai @Orion-Zheng @ver217 @KKZ20 @digger-yu @tiandiao123 @CjhHa1