From 27cae4f05226720afb7666ff956739a355e282ae Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Mon, 25 Mar 2024 14:20:11 +0800 Subject: [PATCH 1/5] [release] grok-1 inference benchmark --- examples/language/grok-1/README.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/examples/language/grok-1/README.md b/examples/language/grok-1/README.md index 2e5ec52c514d..e8ecdca38773 100644 --- a/examples/language/grok-1/README.md +++ b/examples/language/grok-1/README.md @@ -1,9 +1,15 @@ # Grok-1 Inference -An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1. + - 314 Billion Parameter Grok-1 Inference Accelerated by 3.8x, an easy-to-use Python + PyTorch + HuggingFace version for Inference. + [[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1) -[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available) +[[blog]](https://hpc-ai.com/blog/314-billion-parameter-grok-1-inference-accelerated-by-3.8x-efficient-and-easy-to-use-pytorchhuggingface-version-is-here) [[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1) +[[ModelScope Grok-1 PyTorch model weights]](https://www.modelscope.cn/models/colossalai/grok-1-pytorch/summary) + +

+ +

## Installation From d390887a95caaeee5a105d0f84abf48013f08abc Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Mon, 25 Mar 2024 14:20:16 +0800 Subject: [PATCH 2/5] [release] grok-1 inference benchmark --- README.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index d79ec9a66618..36b637985b0c 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ ## Latest News -* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available) +* [2024/03] [314 Billion Parameter Grok-1 Inference Accelerated by 3.8x, Efficient and Easy-to-Use PyTorch+HuggingFace version is Here](https://hpc-ai.com/blog/314-billion-parameter-grok-1-inference-accelerated-by-3.8x-efficient-and-easy-to-use-pytorchhuggingface-version-is-here) * [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0) * [2024/03] [Open-Sora：Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora) * [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) @@ -368,10 +368,16 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt ## Inference ### Grok-1 -An easy-to-use Python + PyTorch + HuggingFace version of 314B Grok-1 Inference. + - 314 Billion Parameter Grok-1 Inference Accelerated by 3.8x, an easy-to-use Python + PyTorch + HuggingFace version for Inference. + [[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1) -[[blog]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available) +[[blog]](https://hpc-ai.com/blog/314-billion-parameter-grok-1-inference-accelerated-by-3.8x-efficient-and-easy-to-use-pytorchhuggingface-version-is-here) [[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1) +[[ModelScope Grok-1 PyTorch model weights]](https://www.modelscope.cn/models/colossalai/grok-1-pytorch/summary) + +

+ +

From b83a34d6344432f25ba5bfbb3cea826bb0ba168a Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Mon, 25 Mar 2024 14:22:48 +0800 Subject: [PATCH 3/5] [release] grok-1 inference benchmark --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 36b637985b0c..26776bdf6d9f 100644 --- a/README.md +++ b/README.md @@ -368,6 +368,10 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt ## Inference ### Grok-1 +

+ +

+ - 314 Billion Parameter Grok-1 Inference Accelerated by 3.8x, an easy-to-use Python + PyTorch + HuggingFace version for Inference. [[code]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1) @@ -375,10 +379,6 @@ Please visit our [documentation](https://www.colossalai.org/) and [examples](htt [[HuggingFace Grok-1 PyTorch model weights]](https://huggingface.co/hpcai-tech/grok-1) [[ModelScope Grok-1 PyTorch model weights]](https://www.modelscope.cn/models/colossalai/grok-1-pytorch/summary) -

- -

-

From e9fd4defde25cd52e28930f11796536e88bdb9e5 Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Mon, 25 Mar 2024 14:22:57 +0800 Subject: [PATCH 4/5] [release] grok-1 inference benchmark --- docs/README-zh-Hans.md | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/docs/README-zh-Hans.md b/docs/README-zh-Hans.md index 352564446cfe..6d243a80852d 100644 --- a/docs/README-zh-Hans.md +++ b/docs/README-zh-Hans.md @@ -24,7 +24,7 @@ ## 新闻 -* [2024/03] [Grok-1 of PyTorch + HuggingFace version is now available!](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available) +* [2024/03] [314 Billion Parameter Grok-1 Inference Accelerated by 3.8x, Efficient and Easy-to-Use PyTorch+HuggingFace version is Here](https://hpc-ai.com/blog/314-billion-parameter-grok-1-inference-accelerated-by-3.8x-efficient-and-easy-to-use-pytorchhuggingface-version-is-here) * [2024/03] [Open-Sora: Revealing Complete Model Parameters, Training Details, and Everything for Sora-like Video Generation Models](https://hpc-ai.com/blog/open-sora-v1.0) * [2024/03] [Open-Sora：Sora Replication Solution with 46% Cost Reduction, Sequence Expansion to Nearly a Million](https://hpc-ai.com/blog/open-sora) * [2024/01] [Inference Performance Improved by 46%, Open Source Solution Breaks the Length Limit of LLM for Multi-Round Conversations](https://hpc-ai.com/blog/Colossal-AI-SwiftInfer) @@ -361,10 +361,16 @@ Colossal-AI 为您提供了一系列并行组件。我们的目标是让您的 ## 推理 ### Grok-1 -方便易用的Python + PyTorch + HuggingFace Grok-1 推理 +

+ +

+ + - 3140亿参数Grok-1推理加速3.8倍，高效易用的PyTorch+HuggingFace版 + [[代码]](https://github.com/hpcaitech/ColossalAI/tree/main/examples/language/grok-1) -[[博客]](https://hpc-ai.com/blog/grok-1-of-pytorch-huggingface-version-is-now-available) -[[HuggingFace Grok-1 PyTorch模型权重]](https://huggingface.co/hpcai-tech/grok-1) +[[博客]](https://hpc-ai.com/blog/314-billion-parameter-grok-1-inference-accelerated-by-3.8x-efficient-and-easy-to-use-pytorchhuggingface-version-is-here) +[[HuggingFace Grok-1 PyTorch 模型权重]](https://huggingface.co/hpcai-tech/grok-1) +[[ModelScope Grok-1 PyTorch 模型权重]](https://www.modelscope.cn/models/colossalai/grok-1-pytorch/summary)

From df41226c126d2eb1235823881c79450379501bd2 Mon Sep 17 00:00:00 2001 From: binmakeswell Date: Mon, 25 Mar 2024 14:23:21 +0800 Subject: [PATCH 5/5] [release] grok-1 inference benchmark --- examples/language/grok-1/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/examples/language/grok-1/README.md b/examples/language/grok-1/README.md index e8ecdca38773..de56afc1d992 100644 --- a/examples/language/grok-1/README.md +++ b/examples/language/grok-1/README.md @@ -8,7 +8,7 @@ [[ModelScope Grok-1 PyTorch model weights]](https://www.modelscope.cn/models/colossalai/grok-1-pytorch/summary)

- +

## Installation