[Feature] LoRA rebased to main branch#5622

Merged

linsj20 merged 225 commits intohpcaitech:feature/lorafrom

linsj20:rebase/lora

Apr 23, 2024

Commits on Oct 24, 2023

[Inference]ADD Bench Chatglm2 script (#4963)
CjhHa1
authored

Commits on Oct 27, 2023

Commits on Oct 30, 2023

[Inference] Dynamic Batching Inference, online and offline (#4953)

authored
[Kernels]Updated Triton kernels into 2.1.0 and adding flash-decoding for llama token attention (#4965)

tiandiao123
and
lcq336
authored

Commits on Nov 1, 2023

Commits on Nov 2, 2023

Commits on Nov 3, 2023

[hotfix] Add layer norm gradients all-reduce for sequence parallel (#4926)

littsk
and
KKZ20
authored

Commits on Nov 6, 2023

[format] applied code formatting on changed files in pull request 4926 (#5007)

github-actions[bot]
and
github-actions
authored

Commits on Nov 7, 2023

[Inference] Fix bug in ChatGLM2 Tensor Parallelism (#5014)
CjhHa1
authored

Commits on Nov 8, 2023

Commits on Nov 9, 2023

Commits on Nov 10, 2023

Commits on Nov 14, 2023

fix wrong EOS token in ColossalChat
Orion-Zheng
committed

Commits on Nov 17, 2023

[hotfix]: modify create_ep_hierarchical_group and add test (#5032)
cwher
authored

Commits on Nov 18, 2023

[exampe] fix llama example' loss error when using gemini plugin (#5060)
flybird11111
authored

Commits on Nov 19, 2023

[inference] Refactor inference architecture (#5057)

authored

Commits on Nov 20, 2023

Commits on Nov 21, 2023

Commits on Nov 23, 2023

Commits on Nov 24, 2023

fix typo change lazy_iniy to lazy_init (#5099)
digger-yu
authored

Commits on Nov 27, 2023

[nfc] fix typo change directoty to directory (#5111)
digger-yu
authored

Commits on Nov 28, 2023

Commits on Dec 1, 2023

[doc] fix colossalqa document (#5146)
MichelleMa8
authored

Commits on Dec 5, 2023

fix (#5158)
flybird11111
authored

Commits on Dec 7, 2023

[Colossal-Llama-2] Add finetuning Colossal-Llama-2 example (#4878)

authored

Commits on Dec 8, 2023

[gemini] hotfix NaN loss while using Gemini + tensor_parallel (#5150)
flybird11111
authored

Commits on Dec 11, 2023

[colossalqa] fix pangu api (#5170)
MichelleMa8
authored

Commits on Dec 12, 2023

Commits on Dec 15, 2023

Fix ColossalEval (#5186)

chengeharrison
and
Xu Yuanchen
authored
[doc] update pytorch version in documents. (#5177)
flybird11111
authored

Commits on Dec 20, 2023

polish readme in application/chat (#5194)
ht-zhou
authored

Commits on Dec 22, 2023

Commits on Dec 27, 2023

[doc] Update required third-party library list for testing and torch comptibility checking (#5207)
KKZ20
authored

Commits on Dec 29, 2023

support linear accumulation fusion (#5199)
flybird11111
authored

Commits on Jan 2, 2024

[pipeline]: support arbitrary batch size in forward_only mode (#5201)
cwher
authored

Commits on Jan 4, 2024

[nfc] fix typo colossalai/shardformer/ (#5133)
digger-yu
authored

Commits on Jan 5, 2024

[Colossal-LLaMA-2] Release Colossal-LLaMA-2-13b-base model (#5224)
TongLi3701
authored

Commits on Jan 6, 2024

Commits on Jan 7, 2024

Commits on Jan 8, 2024

Commits on Jan 9, 2024

Commits on Jan 10, 2024

Commits on Jan 15, 2024

[hotfix]: add pp sanity check and fix mbs arg (#5268)
cwher
authored

Commits on Jan 16, 2024

Commits on Jan 17, 2024

Commits on Jan 18, 2024

Commits on Jan 19, 2024

Commits on Jan 22, 2024

Commits on Jan 25, 2024

Commits on Jan 30, 2024

Commits on Jan 31, 2024

[extension] fixed exception catch (#5342)
FrankLeeeee
authored

Commits on Feb 1, 2024

Commits on Feb 2, 2024

[fix] remove unnecessary dp_size assert (#5351)
cwher
authored

Commits on Feb 4, 2024

[gemini] fix param op hook when output is tuple (#5355)
ver217
authored

Commits on Feb 20, 2024

[hotfix] Fix wrong import in meta_registry (#5392)
stephankoe
authored

Commits on Feb 26, 2024

[extension] hotfix jit extension setup (#5402)
ver217
authored

Commits on Feb 28, 2024

update requirements (#5407)
TongLi3701
authored

Commits on Feb 29, 2024

[workflow] added pypi channel (#5412)
FrankLeeeee
authored
[doc] fix blog link

binmakeswell
authored and
liwenjuna
committed
[doc] fix blog link

binmakeswell
authored and
liwenjuna
committed

Commits on Mar 1, 2024

fix sft single turn inference example (#5416)
Camille7777
authored

Commits on Mar 4, 2024

[example]add gpt2 benchmark example script. (#5295)

authored

Commits on Mar 5, 2024

Commits on Mar 7, 2024

Commits on Mar 11, 2024

fix tensor data update for gemini loss caluculation (#5442)
Camille7777
authored

Commits on Mar 12, 2024

[hotfix] fix typo s/keywrods/keywords etc. (#5429)
digger-yu
authored

Commits on Mar 13, 2024

[devops] fix compatibility (#5444)
ver217
authored

Commits on Mar 18, 2024

Commits on Mar 20, 2024

[doc] update open-sora demo (#5479)
binmakeswell
authored

Commits on Mar 21, 2024

[example] add grok-1 inference (#5485)
ver217
authored

Commits on Mar 22, 2024

[release] grok-1 314b inference (#5490)
binmakeswell
authored

Commits on Mar 24, 2024

[example] update Grok-1 inference (#5495)
yuanheng-zhao
authored

Commits on Mar 25, 2024

Commits on Mar 27, 2024

Commits on Mar 28, 2024

[Fix] Grok-1 use tokenizer from the same pretrained path (#5532)
yuanheng-zhao
authored

Commits on Mar 29, 2024

[ColossalChat] Update RLHF V2 (#5286)

YeAnbang
and
TongLi3701
authored

Commits on Apr 1, 2024

[shardformer, pipeline] add `gradient_checkpointing_ratio` and heterogenous shard policy for llama (#5508)
cwher
authored

Commits on Apr 2, 2024

fix incorrect sharding without zero (#5545)
Edenzzzz
authored

Commits on Apr 3, 2024

[shardformer] Sequence Parallelism Optimization (#5533)

KKZ20
and
linsj20
authored

Commits on Apr 7, 2024

Commits on Apr 8, 2024

[devops] remove post commit ci (#5566)

ver217
and
pre-commit-ci[bot]
authored

Commits on Apr 15, 2024

[doc] fix ColossalMoE readme (#5599)

Camille7777
and
pre-commit-ci[bot]
authored

Commits on Apr 16, 2024

[zero] support multiple (partial) backward passes (#5596)
ver217
authored

Commits on Apr 18, 2024

Commits on Apr 22, 2024