Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/api/vision_results/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# 视觉模型预测结果说明

FastDeploy根据视觉模型的任务类型,定义了不同的结构体(`csrcs/fastdeploy/vision/common/result.h`)来表达模型预测结果,具体如下表所示

| 结构体 | 文档 | 说明 | 相应模型 |
| :----- | :--- | :---- | :------- |
| ClassificationResult | [C++/Python文档](./classificiation_result.md) | 图像分类返回结果 | ResNet50、MobileNetV3等 |
| DetectionResult | [C++/Python文档](./detection_result.md) | 目标检测返回结果 | PPYOLOE、YOLOv7系列模型等 |
28 changes: 28 additions & 0 deletions docs/api/vision_results/classification_result.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# ClassifyResult 图像分类结果

ClassifyResult代码定义在`csrcs/fastdeploy/vision/common/result.h`中,用于表明图像的分类结果和置信度。

## C++ 结构体

`fastdeploy::vision::ClassifyResult`

```
struct ClassifyResult {
std::vector<int32_t> label_ids;
std::vector<float> scores;
void Clear();
std::string Str();
};
```

- **label_ids**: 成员变量,表示单张图片的分类结果,其个数根据在使用分类模型时传入的topk决定,例如可以返回top 5的分类结果
- **scores**: 成员变量,表示单张图片在相应分类结果上的置信度,其个数根据在使用分类模型时传入的topk决定,例如可以返回top 5的分类置信度
- **Clear()**: 成员函数,用于清除结构体中存储的结果
- **Str()**: 成员函数,将结构体中的信息以字符串形式输出(用于Debug)

## Python结构体

`fastdeploy.vision.ClassifyResult`

- **label_ids**(list of int): 成员变量,表示单张图片的分类结果,其个数根据在使用分类模型时传入的topk决定,例如可以返回top 5的分类结果
- **scores**(list of float): 成员变量,表示单张图片在相应分类结果上的置信度,其个数根据在使用分类模型时传入的topk决定,例如可以返回top 5的分类置信度
31 changes: 31 additions & 0 deletions docs/api/vision_results/detection_result.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# DetectionResult 目标检测结果

DetectionResult代码定义在`csrcs/fastdeploy/vision/common/result.h`中,用于表明图像检测出来的目标框、目标类别和目标置信度。

## C++ 结构体

`fastdeploy::vision::DetectionResult`

```
struct DetectionResult {
std::vector<std::array<float, 4>> boxes;
std::vector<float> scores;
std::vector<int32_t> label_ids;
void Clear();
std::string Str();
};
```

- **boxes**: 成员变量,表示单张图片检测出来的所有目标框坐标,`boxes.size()`表示框的个数,每个框以4个float数值依次表示xmin, ymin, xmax, ymax, 即左上角和右下角坐标
- **scores**: 成员变量,表示单张图片检测出来的所有目标置信度,其元素个数与`boxes.size()`一致
- **label_ids**: 成员变量,表示单张图片检测出来的所有目标类别,其元素个数与`boxes.size()`一致
- **Clear()**: 成员函数,用于清除结构体中存储的结果
- **Str()**: 成员函数,将结构体中的信息以字符串形式输出(用于Debug)

## Python结构体

`fastdeploy.vision.DetectionResult`

- **boxes**(list of list(float)): 成员变量,表示单张图片检测出来的所有目标框坐标。boxes是一个list,其每个元素为一个长度为4的list, 表示为一个框,每个框以4个float数值依次表示xmin, ymin, xmax, ymax, 即左上角和右下角坐标
- **scores**(list of float): 成员变量,表示单张图片检测出来的所有目标置信度
- **label_ids(list of int): 成员变量,表示单张图片检测出来的所有目标类别
21 changes: 21 additions & 0 deletions docs/quick_start/install.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# FastDeploy安装

## Python安装

首先安装FastDeploy的SDK管理工具

```
pip install fastdeploy-python
```

然后通过命令行管理工具,根据自己的硬件安装相应SDK

### 安装CPU Python SDK
```
fastdeploy --install cpu
```

### 安装GPU Python SDK
```
fastdeploy --install gpu
```
21 changes: 21 additions & 0 deletions docs/quick_start/requirements.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# FastDeploy环境要求

## 系统平台

- Linux x64/aarch64
- Windows 10 x64
- Mac OSX (x86 10.0以上/ arm64 12.0以上)

## 硬件

- Intel CPU
- Nvidia GPU

## 软件

- cmake >= 3.12
- gcc/g++ >= 8.2
- python >= 3.6
- Visual Studio 2019 (Windows平台)
- cuda >= 11.0
- cudnn >= 8.0
24 changes: 24 additions & 0 deletions new_examples/vision/detection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# 视觉模型部署

本目录下提供了各类视觉模型的部署,主要涵盖以下任务类型

| 任务类型 | 说明 | 预测结果结构体 |
| :------- | :----| :------------- |
| Detection | 目标检测,输入图像,检测图像中物体位置,并返回检测框坐标及类别和置信度 | DetectionResult |
| Segmentation | 语义分割,输入图像,给出图像中每个像素的分类及置信度 | SegmentationResult |
| Classification | 图像分类,输入图像,给出图像的分类结果和置信度 | ClassificationResult |


## FastDeploy API设计

视觉模型具有较有统一任务范式,在设计API时(包括C++/Python),FastDeploy将视觉模型的部署拆分为三个步骤

- 模型加载
- 图像预处理
- 模型推理
- 推理结果后处理

FastDeploy针对飞桨的视觉套件,以及外部热门模型,提供端到端的部署服务,用户只需准备模型,按以下步骤即可完成整个模型的部署

- 加载模型
- 调用`predict`接口
37 changes: 37 additions & 0 deletions new_examples/vision/detection/yolov7/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# YOLOv7部署

## 版本依赖

- [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1)


## 导出ONNX模型

```
#下载yolov7模型文件
wget https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt

# 导出onnx格式文件 (Tips: 对应 YOLOv7 release v0.1 代码)
python models/export.py --grid --dynamic --weights PATH/TO/yolov7.pt

# 如果您的代码版本中有支持NMS的ONNX文件导出,请使用如下命令导出ONNX文件(请暂时不要使用 "--end2end",我们后续将支持带有NMS的ONNX模型的部署)
python export.py --grid --dynamic --weights PATH/TO/yolov7.pt

# 移动onnx文件到demo目录
cp PATH/TO/yolov7.onnx PATH/TO/model_zoo/vision/yolov7/
```

## 预训练模型集合

为了方便开发者的测试,下面提供了YOLOv7导出的各系列模型,开发者可直接下载使用。

| 模型 | 大小 | 精度 |
| :--- | :--- | :--- |
| [YOLOv7](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7.onnx) | 141MB | 51.4% |
| [YOLOv7-x] | 10MB | 51.4% |


## 模型部署

- [Python部署](python)
- [C++部署](cpp)
14 changes: 14 additions & 0 deletions new_examples/vision/detection/yolov7/cpp/CMakeLists.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
PROJECT(infer_demo C CXX)
CMAKE_MINIMUM_REQUIRED (VERSION 3.12)

# 指定下载解压后的fastdeploy库路径
option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.")

include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake)

# 添加FastDeploy依赖头文件
include_directories(${FASTDEPLOY_INCS})

add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc)
# 添加FastDeploy库依赖
target_link_libraries(infer_demo ${FASTDEPLOY_LIBS})
65 changes: 65 additions & 0 deletions new_examples/vision/detection/yolov7/cpp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
# YOLOv7 C++部署示例

本目录下提供`infer.cc`快速完成YOLOv7在CPU/GPU,以及GPU上通过TensorRT加速部署的示例。

在部署前,需确认以下两个步骤

- 1. 软硬件环境满足要求,参考[FastDeploy环境要求](../../docs/quick_start/requirements.md)
- 2. 根据开发环境,下载预编译部署库,参考[FastDeploy预编译库](../../docs/compile/prebuild_libraries.md)

以Linux上CPU推理为例,在本目录执行如下命令即可完成编译测试

```
mkdir build
cd build
wget https://xxx.tgz
tar xvf fastdeploy-linux-x64-0.2.0.tgz
cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-0.2.0
make -j

./infer_demo yolov7.onnx 000001.jpg
```

## YOLOv7 C++接口

### YOLOv7类
```
fastdeploy::vision::wongkinyiu::YOLOv7(
const string& model_file,
const string& params_file = "",
const RuntimeOption& runtime_option = RuntimeOption(),
const Frontend& model_format = Frontend::ONNX)
```
YOLOv7模型加载和初始化,其中model_file为导出的ONNX模型格式。

**参数**

> * **model_file**(str): 模型文件路径
> * **params_file**(str): 参数文件路径,当模型格式为ONNX时,此参数传入空字符串即可
> * **runtime_option**(RuntimeOption): 后端推理配置,默认为None,即采用默认配置
> * **model_format**(Frontend): 模型格式,默认为ONNX格式

#### Predict函数
> ```
> YOLOv7::Predict(cv::Mat* im, DetectionResult* result,
> float conf_threshold = 0.25,
> float nms_iou_threshold = 0.5)
> ```
> 模型预测接口,输入图像直接输出检测结果。
>
> **参数**
>
> > * **im**: 输入图像,注意需为HWC,BGR格式
> > * **result**: 检测结果,包括检测框,各个框的置信度, DetectionResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/)
> > * **conf_threshold**: 检测框置信度过滤阈值
> > * **nms_iou_threshold**: NMS处理过程中iou阈值


### 类成员变量

> > * **size**(vector<int>): 通过此参数修改预处理过程中resize的大小,包含两个整型元素,表示[width, height], 默认值为[640, 640]


- [模型介绍](../../)
- [Python部署](../python)
- [视觉模型预测结果](../../../../../docs/api/vision_results/)
106 changes: 106 additions & 0 deletions new_examples/vision/detection/yolov7/cpp/infer.cc
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

#include "fastdeploy/vision.h"

void CpuInfer(const std::string& model_file, const std::string& image_file) {
auto model = fastdeploy::vision::detection::YOLOv7(model_file);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}

auto im = cv::imread(image_file);
auto im_bak = im.clone();

fastdeploy::vision::DetectionResult res;
if (!model.Predict(&im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}

auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

void GpuInfer() {
auto option = fastdeploy::RuntimeOption();
option.UseGpu();
auto model = fastdeploy::vision::detection::YOLOv7(model_file, "", option);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}

auto im = cv::imread(image_file);
auto im_bak = im.clone();

fastdeploy::vision::DetectionResult res;
if (!model.Predict(&im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}

auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

void TrtInfer() {
auto option = fastdeploy::RuntimeOption();
option.UseGpu();
option.UseTrtBackend();
option.SetTrtInputShape("image", {1, 3, 320, 320}, {1, 3, 640, 640},
{1, 3, 1280, 1280});
auto model = fastdeploy::vision::detection::YOLOv7(model_file, "", option);
if (!model.Initialized()) {
std::cerr << "Failed to initialize." << std::endl;
return;
}

auto im = cv::imread(image_file);
auto im_bak = im.clone();

fastdeploy::vision::DetectionResult res;
if (!model.Predict(&im, &res)) {
std::cerr << "Failed to predict." << std::endl;
return;
}

auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res);
cv::imwrite("vis_result.jpg", vis_im);
std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl;
}

int main(int argc, char* argv[]) {
if (argc < 4) {
std::cout << "Usage: infer_demo path/to/model path/to/image run_option, "
"e.g ./infer_model ./yolov7.onnx ./test.jpeg 0"
<< std::endl;
std::cout << "The data type of run_option is int, 0: run with cpu; 1: run "
"with gpu; 2: run with gpu and use tensorrt backend."
<< std::endl;
return -1;
}

if (std::atoi(argv[3]) == 0) {
CpuInfer();
} else if (std::atoi(argv[3]) == 1) {
GpuInfer();
} else if (std::atoi(argv[3]) == 2) {
TrtInfer();
}
return 0;
}
Loading