From a302cb6f42c3d039a71b4ef43568785eb0a5deb3 Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Sun, 7 Jan 2024 21:10:08 +0800 Subject: [PATCH 1/5] [doc] SwiftInfer release --- applications/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/applications/README.md b/applications/README.md index 92096e5593d0..49a2900f1715 100644 --- a/applications/README.md +++ b/applications/README.md @@ -9,6 +9,7 @@ The list of applications include: - [X] [ColossalChat](./Chat/README.md): Replication of ChatGPT with RLHF. - [X] [FastFold](https://github.com/hpcaitech/FastFold): Optimizing AlphaFold (Biomedicine) Training and Inference on GPU Clusters. - [X] [ColossalQA](./ColossalQA/README.md): Document Retrieval Conversation System +- [X] [SwiftInfer](https://github.com/hpcaitech/SwiftInfer): Breaks the Length Limit of LLM Inference for Multi-Round Conversations > Please note that the `Chatbot` application is migrated from the original `ChatGPT` folder. From c972ff5559fe5191fe21f58fb8db49ed78286d94 Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Sun, 7 Jan 2024 21:10:16 +0800 Subject: [PATCH 2/5] [doc] SwiftInfer release --- docs/README-zh-Hans.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/README-zh-Hans.md b/docs/README-zh-Hans.md index a0330a62dd7a..0628b4b739e9 100644 --- a/docs/README-zh-Hans.md +++ b/docs/README-zh-Hans.md @@ -24,6 +24,7 @@ ## 新闻 +* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) * [2024/01] [Construct Refined 13B Private Model With Just $5000 USD, Upgraded Colossal-AI Llama-2 Open Source](https://hpc-ai.com/blog/colossal-llama-2-13b) * [2023/11] [Enhanced MoE Parallelism, Open-source MoE Model Training Can Be 9 Times More Efficient](https://www.hpc-ai.tech/blog/enhanced-moe-parallelism-open-source-moe-model-training-can-be-9-times-more-efficient) * [2023/09] [One Half-Day of Training Using a Few Hundred Dollars Yields Similar Results to Mainstream Large Models, Open-Source and Commercial-Free Domain-Specific LLM Solution](https://www.hpc-ai.tech/blog/one-half-day-of-training-using-a-few-hundred-dollars-yields-similar-results-to-mainstream-large-models-open-source-and-commercial-free-domain-specific-llm-solution) @@ -69,8 +70,9 @@
  • - 推理 (Energon-AI) 样例展示 + 推理
      +
    • SwiftInfer:打破LLM多轮对话的长度限制，推理加速46%
    • GPT-3
    • 1750亿参数OPT在线推理服务
    • 1760亿参数 BLOOM
      • @@ -344,7 +346,12 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的

      (返回顶端)

      -## 推理 (Energon-AI) 样例展示 +## 推理 +

      + +

      + +- [SwiftInfer](https://github.com/hpcaitech/SwiftInfer): Inference performance improved by 46%, open source solution breaks the length limit of LLM for multi-round conversations

      From 49ef509f76ef5eb319cf69a30fa1d8a110b5904c Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Sun, 7 Jan 2024 21:10:21 +0800 Subject: [PATCH 3/5] [doc] SwiftInfer release --- README.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 33aef6984568..99fc5e696beb 100644 --- a/README.md +++ b/README.md @@ -25,6 +25,7 @@ ## Latest News +* [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) * [2024/01] [Construct Refined 13B Private Model With Just $5000 USD, Upgraded Colossal-AI Llama-2 Open Source](https://hpc-ai.com/blog/colossal-llama-2-13b) * [2023/11] [Enhanced MoE Parallelism, Open-source MoE Model Training Can Be 9 Times More Efficient](https://www.hpc-ai.tech/blog/enhanced-moe-parallelism-open-source-moe-model-training-can-be-9-times-more-efficient) * [2023/09] [One Half-Day of Training Using a Few Hundred Dollars Yields Similar Results to Mainstream Large Models, Open-Source and Commercial-Free Domain-Specific LLM Solution](https://www.hpc-ai.tech/blog/one-half-day-of-training-using-a-few-hundred-dollars-yields-similar-results-to-mainstream-large-models-open-source-and-commercial-free-domain-specific-llm-solution) @@ -70,8 +71,9 @@

  • - Inference (Energon-AI) Demo + Inference
      +
    • SwiftInfer:Breaks the Length Limit of LLM for Multi-Round Conversations with 46% Acceleration
    • GPT-3
    • OPT-175B Online Serving for Text Generation
    • 176B BLOOM
      • @@ -350,7 +352,12 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt

      (back to top)

      -## Inference (Energon-AI) Demo +## Inference +

      + +

      + +- [SwiftInfer](https://github.com/hpcaitech/SwiftInfer): Inference performance improved by 46%, open source solution breaks the length limit of LLM for multi-round conversations

      From 80cd47bd8b3a94b7a60c34907761b45c70f12e9d Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Sun, 7 Jan 2024 21:12:53 +0800 Subject: [PATCH 4/5] [doc] SwiftInfer release --- docs/README-zh-Hans.md | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/README-zh-Hans.md b/docs/README-zh-Hans.md index 0628b4b739e9..0c438c726baa 100644 --- a/docs/README-zh-Hans.md +++ b/docs/README-zh-Hans.md @@ -117,8 +117,6 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的 - [PatrickStar](https://arxiv.org/abs/2108.05818) - 使用友好 - 基于参数文件的并行化 -- 推理 - - [Energon-AI](https://github.com/hpcaitech/EnergonAI)

      (返回顶端)

      @@ -214,7 +212,7 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的 - [DreamBooth微调](https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/dreambooth): 仅需3-5张目标主题图像个性化微调 -

      +

      From 1225980fc0a0aee293fd03c567d8a10524f90203 Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Sun, 7 Jan 2024 21:12:57 +0800 Subject: [PATCH 5/5] [doc] SwiftInfer release --- README.md | 5 +---- 1 file changed, 1 insertion(+), 4 deletions(-) diff --git a/README.md b/README.md index 99fc5e696beb..971f4375a289 100644 --- a/README.md +++ b/README.md @@ -123,9 +123,6 @@ distributed training and inference in a few lines. - Friendly Usage - Parallelism based on the configuration file -- Inference - - [Energon-AI](https://github.com/hpcaitech/EnergonAI) -

      (back to top)

      ## Colossal-AI in the Real World @@ -222,7 +219,7 @@ Acceleration of AIGC (AI-Generated Content) models such as [Stable Diffusion v1] - [DreamBooth Fine-tuning](https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/dreambooth): Personalize your model using just 3-5 images of the desired subject. -

      +