diff --git a/README.md b/README.md index 20a1f520a3b..3c9f2474384 100644 --- a/README.md +++ b/README.md @@ -16,11 +16,10 @@

-

特性 | - 安装 | - 快速开始 | + 服务器端 | + 端侧 | 社区交流

@@ -28,100 +27,70 @@ ## News 📢 -* 🔥 2022.6.30 晚20:30,⚡️FastDeploy天使用户邀测沟通会,与开发者共同讨论推理部署痛点问题,欢迎大家扫码报名入群获取会议链接。 -
- -
- -* 🔥 2022.6.27 [**⚡️FastDeploy v0.1.0**](https://github.com/PaddlePaddle/FastDeploy/releases/tag/release%2F0.1.0)测试版发布!🎉 - * 💎 发布40个重点模型在8种重点软硬件环境的支持的SDK - * 😊 支持网页端、pip包两种下载使用方式 - +* 🔥 2022.8.15 [**⚡️FastDeploy v0.2.0**](https://github.com/PaddlePaddle/FastDeploy/releases/tag/release%2F0.2.0)测试版发布!🎉 + + * 💎 升级服务器端(CPU/GPU/Jetson)SDK代码架构,速度SOTA + * 😊 支持PyTorch模型部署,如YOLOv5、YOLOv6、YOLOv7等热门模型 ## 特性 - ### 📦**开箱即用的推理部署工具链,支持云边端、多硬件、多平台部署** + - 网页端点选下载、PIP 安装一行命令,快速下载多种类型SDK安装包 - 云端(含服务器、数据中心): - - 支持一行命令启动 Serving 服务(含网页图形化展示) - - 支持一行命令启动图像、本地视频流、本地摄像头、网络视频流预测 - - 支持 Window、Linux 操作系统 - - 支持 Python、C++ 编程语言 + - 支持 Window、Linux 操作系统 + - 支持 Python、C++ 编程语言 - 边缘端: - - 支持 NVIDIA Jetson 等边缘设备,支持视频流预测服务 + - 支持 NVIDIA Jetson 等边缘设备 - 端侧(含移动端) - - 支持 iOS、Android 移动端 - - 支持 ARM CPU 端侧设备 + - 支持 iOS、Android 移动端 + - 支持 ARM CPU 端侧设备 - 支持主流硬件 - - 支持 Intel CPU 系列(含酷睿、至强等) - - 支持 ARM CPU 全系(含高通、MTK、RK等) - - 支持 NVIDIA GPU 全系(含 V100、T4、Jetson 等) - -### 🤗**丰富的预训练模型,轻松下载SDK搞定推理部署** - - - - -| 模型| 任务 | 大小(MB) | 端侧 | 移动端 | 移动端 |边缘端 |服务器+云端 | 服务器+云端 | 服务器+云端 | 服务器+云端 | -|---|---|---|---|---|---|---|---|---|---|---| -|----- | ---- |----- | Linux | Android | iOS | Linux | Linux | Linux | Windows | Windows | -|----- | ---- |--- | ARM CPU | ARM CPU | ARM CPU | Jetson | X86 CPU | GPU | X86 CPU | GPU | -| [PP-LCNet](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 11.9 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-LCNetv2](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 26.6 |✅|✅|✅|✅|✅|✅|✅|✅| -| [EfficientNet](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication |31.4 |✅|✅|✅|✅|✅|✅|✅|✅| -| [GhostNet](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 20.8 |✅|✅|✅|✅|✅|✅|✅|✅| -| [MobileNetV1](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 17 |✅|✅|✅|✅|✅|✅|✅|✅|✅| -| [MobileNetV2](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 14.2 |✅|✅|✅|✅|✅|✅|✅|✅| -| [MobileNetV3](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 22 |✅|✅|✅|✅|✅|✅|✅|✅| -| [ShuffleNetV2](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md)|Classfication | 9.2 |✅|✅|✅|✅|✅|✅|✅|✅| -| [SqueezeNetV1.1](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication |5 |✅|✅|✅|✅|✅|✅|✅|✅| -| [Inceptionv3](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication |95.5 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-HGNet](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 59 |✅|✅|✅|✅|✅|✅|✅|✅| -| [ResNet50_vd](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 102.5 |❌|❌|❌|✅|✅|✅|✅|✅| -| [SwinTransformer_224_win7](https://github.com/PaddlePaddle/PaddleClas/blob/release/2.3/docs/zh_CN/models_training/classification.md) |Classfication | 352.7 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-PicoDet_s_320_coco](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 4.1 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-PicoDet_s_320_lcnet](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 4.9 |✅|✅|✅|✅|✅|✅|✅|✅| -| [CenterNet](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection |4.8 |✅|✅|✅|✅ |✅ |✅|✅|✅| -| [YOLOv3_MobileNetV3](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 94.6 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-YOLO_tiny_650e_coco](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection |4.4 |✅|✅|✅|✅|✅|✅|✅|✅| -| [SSD_MobileNetV1_300_120e_voc](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 23.3 |✅|✅|✅|✅|✅|✅|✅|✅| -| [YOLOX_Nano_300e_coco](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 3.7 |❌|❌|❌|✅|✅ |✅|✅|✅| -| [PP-YOLO_ResNet50vd](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 188.5|✅ |✅ |✅ |✅ |✅ |✅|✅|✅| -| [PP-YOLOv2_ResNet50vd](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 218.7 |✅|✅|✅|✅|✅ |✅|✅|✅| -| [PP-YOLO_crn_l_300e_coco](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 209.1 |✅|✅|✅|✅|✅|✅|✅|✅| -| [YOLOv5s](https://github.com/ultralytics/yolov5) |Detection | 29.3|✅|✅|✅|✅|✅|✅|✅|✅| -| [Faster R-CNN_r50_fpn_1x_coco](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Detection | 167.2 |❌|❌|❌|✅|✅|✅|✅|✅| -| [BlazeFace](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Face Detection |1.5|✅|✅|✅|✅|✅|✅|✅|✅| -| [RetinaFace](https://github.com/biubug6/Pytorch_Retinaface) |Face Localisation |1.7| ✅|❌|❌|✅|✅|✅|✅|✅| -| [PP-TinyPose](https://github.com/PaddlePaddle/PaddleDetection/blob/develop/docs/tutorials/GETTING_STARTED_cn.md) |Keypoint Detection| 5.5 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-LiteSeg(STDC1)](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/configs/pp_liteseg/README.md)|Segmentation | 32.2|✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-HumanSeg-Lite](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README_cn.md) |Segmentation | 0.556|✅|✅|✅|✅|✅|✅|✅|✅| -| [HRNet-w18](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/train/train_cn.md) |Segmentation | 38.7|✅|✅|✅|❌|✅|✅|✅|✅| -| [Mask R-CNN_r50_fpn_1x_coco](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README_cn.md)|Segmentation| 107.2|❌|❌|❌|✅|✅|✅|✅|✅| -| [PP-HumanSeg-Server](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/contrib/PP-HumanSeg/README_cn.md)|Segmentation | 107.2|✅|✅|✅|✅|✅|✅|✅|✅| -| [Unet](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/train/train_cn.md) |Segmentation | 53.7|❌|✅|❌|❌|✅|✅|✅|❌| -| [Deeplabv3-ResNet50](https://github.com/PaddlePaddle/PaddleSeg/blob/develop/docs/train/train_cn.md)|Segmentation |156.5|❌|❌|❌|❌|✅|✅|✅|✅| -| [PP-OCRv1](https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.5/doc/doc_ch/ppocr_introduction.md) |OCR | 2.3+4.4 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-OCRv2](https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.5/doc/doc_ch/ppocr_introduction.md) |OCR | 2.3+4.4 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-OCRv3](https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.5/doc/doc_ch/PP-OCRv3_introduction.md) |OCR | 2.4+10.6 |✅|✅|✅|✅|✅|✅|✅|✅| -| [PP-OCRv3-tiny](https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.5/doc/doc_ch/models_list.md) |OCR |2.4+10.7 |✅|✅|✅|✅|✅|✅|✅|✅| - - - -## SDK安装 - -### 方式1:网页版下载安装 - -- 可以登录[EasyEdge网页端](https://ai.baidu.com/easyedge/app/openSource)下载SDK - -### 方式2:pip安装 + - 支持 Intel CPU 系列(含酷睿、至强等) + - 支持 ARM CPU 全系(含高通、MTK、RK等) + - 支持 NVIDIA GPU 全系(含 V100、T4、Jetson 等) + +## 服务器端 + +### 🤗**服务端:丰富的预训练模型,轻松下载SDK搞定推理部署** + + +| 任务场景 | 模型 | 大小(MB) | 边缘端 | 服务器/云端 | 服务器/云端 | 服务器/云端 | 服务器/云端 | +| ------------------------------- | --------------------------------------------------------- | --------------------- | --------------------------------------- | ------------------------------- | -------------------- | ------------------------- | --------------------- | +| ---- | ---- | ---- | [Jetson](./doc/Jetson.md) | [X86 CPU](./doc/) | [GPU]() | [X86 CPU]() | [GPU]() | +| ---- | ---- | ---- | Linux | Windows | Linux | Windows | Linux | +| Classfication | | | | | | | | +| Detection | [NanoDet-Plus](./model_zoo/vision/nanodet_plus/README.md) | 0.95~2.44 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOR](./model_zoo/vison/yolor/README.md) | | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOX](./model_zoo/vison/yolox/README.md) | | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [Scaled-YOLOv4](./model_zoo/vison/scaledyolov4/README.md) | 4.9 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOv5](./model_zoo/vison/yolov5/README.md) | | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOv5_Lite](./model_zoo/vison/yolov5lite/README.md) | 94.6 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOv6](./model_zoo/vison/yolov6/README.md) | 4.4 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOv7](./model_zoo/vison/yolov7/README.md) | 23.3 | ✅ | ✅ | ✅ | ✅ | ✅ | +| Face Detection | [UltraFace](./model_zoo/vison/ultraface/README.md) | 1.04~1.1 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [YOLOv5Face](./model_zoo/vison/yolov5face/README.md) | | ✅ | ✅ | ✅ | ✅ | ✅ | +| Face Localisation | [RetinaFace](./model_zoo/vison/retinaface/README.md) | 1.7M | ✅ | ✅ | ✅ | ✅ | ✅ | +| Face Recognition | [ArcFace](./model_zoo/vison/arcface/README.md) | 1.7 | ✅ | ✅ | ✅ | ✅ | ✅ | +| Keypoint Detection | [SCRFD](./model_zoo/vison/scrfd/README.md) | 5.5 | ✅ | ✅ | ✅ | ✅ | ✅ | +| Segmentation | [PP-Seg](./model_zoo/vison/ppseg/README.md) | 32.2 | ✅ | ✅ | ✅ | ✅ | ✅ | +| OCR | [PP-OCRv1](./model_zoo/vison/ppocrv1/README.md) | 2.3+4.4 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [PP-OCRv2](./model_zoo/vison/ppocrv2/README.md) | 2.3+4.4 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | [PP-OCRv3](./model_zoo/vison/ppocrv3/README.md) | 2.4+10.6 | ✅ | ✅ | ✅ | ✅ | ✅ | +| | | | | | | | | + + + +### 服务器端快速开始(CPU/GPU/Jetson) + + +### 方式:pip安装 开发者可以通过pip安装`fastdeploy-python`来获取最新的下载链接 - 环境依赖 - - python >= 3.6 + + python >= 3.6 - 安装方式 @@ -130,69 +99,105 @@ pip install fastdeploy-python --upgrade ``` - 使用方式 - - - 列出FastDeploy当前支持的所有模型 + + - 列出FastDeploy当前支持的所有模型 + ``` fastdeploy --list_models ``` - - 下载模型在具体平台和对应硬件上的部署SDK以及示例 + + - 下载模型在具体平台和对应硬件上的部署SDK以及示例 + ``` fastdeploy --download_sdk \ - --model PP-PicoDet-s_320 \ - --platform Linux \ - --soc x86 \ - --save_dir . + --model PP-PicoDet-s_320 \ + --platform Linux \ + --soc x86 \ + --save_dir . ``` - - - 参数说明 - - `list_models`: 列出FastDeploy当前最新支持的所有模型 - - `download_sdk`: 下载模型在具体平台和对应硬件上的部署SDK以及示例 - - `model`: 模型名,如"PP-PicoDet-s_320",可通过`list_models`查看所有的可选项 - - `platform`: 部署平台,支持 Windows/Linux/Android/iOS - - `soc`: 部署硬件,支持 x86/x86-NVIDIA-GPU/ARM/Jetson - - `save_dir`: SDK下载保存目录 - -## SDK使用 -### 1 云+服务器部署 - - Linux 系统(X86 CPU、NVIDIA GPU) - - [C++ Inference部署(含视频流)](./docs/Linux-CPP-SDK-Inference.md) - - [C++ 服务化部署](./docs/Linux-CPP-SDK-Serving.md) - - [Python Inference部署](./docs/Linux-Python-SDK-Inference.md) - - [Python 服务化部署](./docs/Linux-Python-SDK-Serving.md) - - Window系统(X86 CPU、NVIDIA GPU) - - [C++ Inference部署(含视频流)](./docs/Windows-CPP-SDK-Inference.md) - - [C++ 服务化部署](./docs/Windows-CPP-SDK-Serving.md) - - [Python Inference部署](./docs/Windows-Python-SDK-Inference.md) - - [Python 服务化部署](./docs/Windows-Python-SDK-Serving.md) - -### 2 边缘侧部署 - - ArmLinux 系统(NVIDIA Jetson Nano/TX2/Xavier) - - [C++ Inference部署(含视频流)](./docs/Jetson-Linux-CPP-SDK-Inference.md) - - [C++ 服务化部署](./docs/Jetson-Linux-CPP-SDK-Serving.md) - -### 3 端侧部署 - - ArmLinux 系统(ARM CPU) - - [C++ Inference部署(含视频流)](./docs/ARM-Linux-CPP-SDK-Inference.md) - - [C++ 服务化部署](./docs/ARM-Linux-CPP-SDK-Serving.md) - - [Python Inference部署](./docs/ARM-Linux-Python-SDK-Inference.md) - - [Python 服务化部署](./docs/ARM-Linux-Python-SDK-Serving.md) - -### 4 移动端部署 - - [iOS 系统部署](./docs/iOS-SDK.md) - - [Android 系统部署](./docs/Android-SDK.md) - -### 5 自定义模型部署 - - [快速实现个性化模型替换](./docs/Replace-Model-With-Anther-One.md) + + - 参数说明 + + - `list_models`: 列出FastDeploy当前最新支持的所有模型 + - `download_sdk`: 下载模型在具体平台和对应硬件上的部署SDK以及示例 + - `model`: 模型名,如"PP-PicoDet-s_320",可通过`list_models`查看所有的可选项 + - `platform`: 部署平台,支持 Windows/Linux/Android/iOS + - `soc`: 部署硬件,支持 x86/x86-NVIDIA-GPU/ARM/Jetson + - `save_dir`: SDK下载保存目录 + +## 端侧 + +### 🤗**移动端和端侧:丰富的预训练模型,轻松下载SDK搞定推理部署** + + +| 任务场景 | 模型 | 大小(MB) | 端侧 | 移动端 | 移动端 | +| ------------------ | ---------------------------- | --------------------- | --------------------- | ---------------------- | --------------------- | +| ---- | --- | --- | Linux | Android | iOS | +| ----- | ---- | --- | ARM CPU | ARM CPU | ARM CPU | +| Classfication | PP-LCNet | 11.9 | ✅ | ✅ | ✅ | +| | PP-LCNetv2 | 26.6 | ✅ | ✅ | ✅ | +| | EfficientNet | 31.4 | ✅ | ✅ | ✅ | +| | GhostNet | 20.8 | ✅ | ✅ | ✅ | +| | MobileNetV1 | 17 | ✅ | ✅ | ✅ | +| | MobileNetV2 | 14.2 | ✅ | ✅ | ✅ | +| | MobileNetV3 | 22 | ✅ | ✅ | ✅ | +| | ShuffleNetV2 | 9.2 | ✅ | ✅ | ✅ | +| | SqueezeNetV1.1 | 5 | ✅ | ✅ | ✅ | +| | Inceptionv3 | 95.5 | ✅ | ✅ | ✅ | +| | PP-HGNet | 59 | ✅ | ✅ | ✅ | +| | SwinTransformer_224_win7 | 352.7 | ✅ | ✅ | ✅ | +| Detection | PP-PicoDet_s_320_coco | 4.1 | ✅ | ✅ | ✅ | +| | PP-PicoDet_s_320_lcnet | 4.9 | ✅ | ✅ | ✅ | +| | CenterNet | 4.8 | ✅ | ✅ | ✅ | +| | YOLOv3_MobileNetV3 | 94.6 | ✅ | ✅ | ✅ | +| | PP-YOLO_tiny_650e_coco | 4.4 | ✅ | ✅ | ✅ | +| | SSD_MobileNetV1_300_120e_voc | 23.3 | ✅ | ✅ | ✅ | +| | PP-YOLO_ResNet50vd | 188.5 | ✅ | ✅ | ✅ | +| | PP-YOLOv2_ResNet50vd | 218.7 | ✅ | ✅ | ✅ | +| | PP-YOLO_crn_l_300e_coco | 209.1 | ✅ | ✅ | ✅ | +| | YOLOv5s | 29.3 | ✅ | ✅ | ✅ | +| Face Detection | BlazeFace | 1.5 | ✅ | ✅ | ✅ | +| Face Localisation | RetinaFace | 1.7 | ✅ | ❌ | ❌ | +| Keypoint Detection | PP-TinyPose | 5.5 | ✅ | ✅ | ✅ | +| Segmentation | PP-LiteSeg(STDC1) | 32.2 | ✅ | ✅ | ✅ | +| | PP-HumanSeg-Lite | 0.556 | ✅ | ✅ | ✅ | +| | HRNet-w18 | 38.7 | ✅ | ✅ | ✅ | +| | PP-HumanSeg-Server | 107.2 | ✅ | ✅ | ✅ | +| | Unet | 53.7 | ❌ | ✅ | ❌ | +| OCR | PP-OCRv1 | 2.3+4.4 | ✅ | ✅ | ✅ | +| | PP-OCRv2 | 2.3+4.4 | ✅ | ✅ | ✅ | +| | PP-OCRv3 | 2.4+10.6 | ✅ | ✅ | ✅ | +| | PP-OCRv3-tiny | 2.4+10.7 | ✅ | ✅ | ✅ | + +## + +## SDK使用(ARM CPU) + +### 1 端侧部署 + +- ArmLinux 系统(ARM CPU) + - [C++ Inference部署(含视频流)](./docs/ARM-Linux-CPP-SDK-Inference.md) + - [C++ 服务化部署](./docs/ARM-Linux-CPP-SDK-Serving.md) + - [Python Inference部署](./docs/ARM-Linux-Python-SDK-Inference.md) + - [Python 服务化部署](./docs/ARM-Linux-Python-SDK-Serving.md) + +### 2 移动端部署 + +- [iOS 系统部署](./docs/iOS-SDK.md) +- [Android 系统部署](./docs/Android-SDK.md) + +### 3 自定义模型部署 + +- [快速实现个性化模型替换](./docs/Replace-Model-With-Anther-One.md) ## 社区交流 - - **加入社区👬:** 微信扫描二维码后,填写问卷加入交流群,与开发者共同讨论推理部署痛点问题 + +- **加入社区👬:** 微信扫描二维码后,填写问卷加入交流群,与开发者共同讨论推理部署痛点问题
- - ## Acknowledge 本项目中SDK生成和下载使用了[EasyEdge](https://ai.baidu.com/easyedge/app/openSource)中的免费开放能力,再次表示感谢。