diff --git a/docs/api/vision_results/README.md b/docs/api/vision_results/README.md new file mode 100644 index 00000000000..0a05aeaf8cd --- /dev/null +++ b/docs/api/vision_results/README.md @@ -0,0 +1,8 @@ +# 视觉模型预测结果说明 + +FastDeploy根据视觉模型的任务类型，定义了不同的结构体(`csrcs/fastdeploy/vision/common/result.h`)来表达模型预测结果，具体如下表所示 + +| 结构体 | 文档 | 说明 | 相应模型 | +| :----- | :--- | :---- | :------- | +| ClassificationResult | [C++/Python文档](./classificiation_result.md) | 图像分类返回结果 | ResNet50、MobileNetV3等 | +| DetectionResult | [C++/Python文档](./detection_result.md) | 目标检测返回结果 | PPYOLOE、YOLOv7系列模型等 | diff --git a/docs/api/vision_results/classification_result.md b/docs/api/vision_results/classification_result.md new file mode 100644 index 00000000000..113db39608a --- /dev/null +++ b/docs/api/vision_results/classification_result.md @@ -0,0 +1,28 @@ +# ClassifyResult 图像分类结果 + +ClassifyResult代码定义在`csrcs/fastdeploy/vision/common/result.h`中，用于表明图像的分类结果和置信度。 + +## C++ 结构体 + +`fastdeploy::vision::ClassifyResult` + +``` +struct ClassifyResult { + std::vector label_ids; + std::vector scores; + void Clear(); + std::string Str(); +}; +``` + +- **label_ids**: 成员变量，表示单张图片的分类结果，其个数根据在使用分类模型时传入的topk决定，例如可以返回top 5的分类结果 +- **scores**: 成员变量，表示单张图片在相应分类结果上的置信度，其个数根据在使用分类模型时传入的topk决定，例如可以返回top 5的分类置信度 +- **Clear()**: 成员函数，用于清除结构体中存储的结果 +- **Str()**: 成员函数，将结构体中的信息以字符串形式输出（用于Debug） + +## Python结构体 + +`fastdeploy.vision.ClassifyResult` + +- **label_ids**(list of int): 成员变量，表示单张图片的分类结果，其个数根据在使用分类模型时传入的topk决定，例如可以返回top 5的分类结果 +- **scores**(list of float): 成员变量，表示单张图片在相应分类结果上的置信度，其个数根据在使用分类模型时传入的topk决定，例如可以返回top 5的分类置信度 diff --git a/docs/api/vision_results/detection_result.md b/docs/api/vision_results/detection_result.md new file mode 100644 index 00000000000..e44a27b34c3 --- /dev/null +++ b/docs/api/vision_results/detection_result.md @@ -0,0 +1,31 @@ +# DetectionResult 目标检测结果 + +DetectionResult代码定义在`csrcs/fastdeploy/vision/common/result.h`中，用于表明图像检测出来的目标框、目标类别和目标置信度。 + +## C++ 结构体 + +`fastdeploy::vision::DetectionResult` + +``` +struct DetectionResult { + std::vector> boxes; + std::vector scores; + std::vector label_ids; + void Clear(); + std::string Str(); +}; +``` + +- **boxes**: 成员变量，表示单张图片检测出来的所有目标框坐标，`boxes.size()`表示框的个数，每个框以4个float数值依次表示xmin, ymin, xmax, ymax， 即左上角和右下角坐标 +- **scores**: 成员变量，表示单张图片检测出来的所有目标置信度，其元素个数与`boxes.size()`一致 +- **label_ids**: 成员变量，表示单张图片检测出来的所有目标类别，其元素个数与`boxes.size()`一致 +- **Clear()**: 成员函数，用于清除结构体中存储的结果 +- **Str()**: 成员函数，将结构体中的信息以字符串形式输出（用于Debug） + +## Python结构体 + +`fastdeploy.vision.DetectionResult` + +- **boxes**(list of list(float)): 成员变量，表示单张图片检测出来的所有目标框坐标。boxes是一个list，其每个元素为一个长度为4的list， 表示为一个框，每个框以4个float数值依次表示xmin, ymin, xmax, ymax， 即左上角和右下角坐标 +- **scores**(list of float): 成员变量，表示单张图片检测出来的所有目标置信度 +- **label_ids(list of int): 成员变量，表示单张图片检测出来的所有目标类别 diff --git a/docs/quick_start/install.md b/docs/quick_start/install.md new file mode 100644 index 00000000000..65414606f85 --- /dev/null +++ b/docs/quick_start/install.md @@ -0,0 +1,21 @@ +# FastDeploy安装 + +## Python安装 + +首先安装FastDeploy的SDK管理工具 + +``` +pip install fastdeploy-python +``` + +然后通过命令行管理工具，根据自己的硬件安装相应SDK + +### 安装CPU Python SDK +``` +fastdeploy --install cpu +``` + +### 安装GPU Python SDK +``` +fastdeploy --install gpu +``` diff --git a/docs/quick_start/requirements.md b/docs/quick_start/requirements.md new file mode 100644 index 00000000000..462f9a5c96d --- /dev/null +++ b/docs/quick_start/requirements.md @@ -0,0 +1,21 @@ +# FastDeploy环境要求 + +## 系统平台 + +- Linux x64/aarch64 +- Windows 10 x64 +- Mac OSX (x86 10.0以上/ arm64 12.0以上) + +## 硬件 + +- Intel CPU +- Nvidia GPU + +## 软件 + +- cmake >= 3.12 +- gcc/g++ >= 8.2 +- python >= 3.6 +- Visual Studio 2019 (Windows平台) +- cuda >= 11.0 +- cudnn >= 8.0 diff --git a/new_examples/vision/detection/README.md b/new_examples/vision/detection/README.md new file mode 100644 index 00000000000..56235f101b4 --- /dev/null +++ b/new_examples/vision/detection/README.md @@ -0,0 +1,24 @@ +# 视觉模型部署 + +本目录下提供了各类视觉模型的部署，主要涵盖以下任务类型 + +| 任务类型 | 说明 | 预测结果结构体 | +| :------- | :----| :------------- | +| Detection | 目标检测，输入图像，检测图像中物体位置，并返回检测框坐标及类别和置信度 | DetectionResult | +| Segmentation | 语义分割，输入图像，给出图像中每个像素的分类及置信度 | SegmentationResult | +| Classification | 图像分类，输入图像，给出图像的分类结果和置信度 | ClassificationResult | + + +## FastDeploy API设计 + +视觉模型具有较有统一任务范式，在设计API时（包括C++/Python），FastDeploy将视觉模型的部署拆分为三个步骤 + +- 模型加载 +- 图像预处理 +- 模型推理 +- 推理结果后处理 + +FastDeploy针对飞桨的视觉套件，以及外部热门模型，提供端到端的部署服务，用户只需准备模型，按以下步骤即可完成整个模型的部署 + +- 加载模型 +- 调用`predict`接口 diff --git a/new_examples/vision/detection/yolov7/README.md b/new_examples/vision/detection/yolov7/README.md new file mode 100644 index 00000000000..e67a58f96ef --- /dev/null +++ b/new_examples/vision/detection/yolov7/README.md @@ -0,0 +1,37 @@ +# YOLOv7部署 + +## 版本依赖 + +- [YOLOv7 0.1](https://github.com/WongKinYiu/yolov7/releases/tag/v0.1) + + +## 导出ONNX模型 + +``` +#下载yolov7模型文件 +wget https://github.com/WongKinYiu/yolov7/releases/download/v0.1/yolov7.pt + +# 导出onnx格式文件 (Tips: 对应 YOLOv7 release v0.1 代码) +python models/export.py --grid --dynamic --weights PATH/TO/yolov7.pt + +# 如果您的代码版本中有支持NMS的ONNX文件导出，请使用如下命令导出ONNX文件(请暂时不要使用 "--end2end"，我们后续将支持带有NMS的ONNX模型的部署) +python export.py --grid --dynamic --weights PATH/TO/yolov7.pt + +# 移动onnx文件到demo目录 +cp PATH/TO/yolov7.onnx PATH/TO/model_zoo/vision/yolov7/ +``` + +## 预训练模型集合 + +为了方便开发者的测试，下面提供了YOLOv7导出的各系列模型，开发者可直接下载使用。 + +| 模型 | 大小 | 精度 | +| :--- | :--- | :--- | +| [YOLOv7](https://bj.bcebos.com/paddlehub/fastdeploy/yolov7.onnx) | 141MB | 51.4% | +| [YOLOv7-x] | 10MB | 51.4% | + + +## 模型部署 + +- [Python部署](python) +- [C++部署](cpp) diff --git a/new_examples/vision/detection/yolov7/cpp/CMakeLists.txt b/new_examples/vision/detection/yolov7/cpp/CMakeLists.txt new file mode 100644 index 00000000000..fea1a2888b9 --- /dev/null +++ b/new_examples/vision/detection/yolov7/cpp/CMakeLists.txt @@ -0,0 +1,14 @@ +PROJECT(infer_demo C CXX) +CMAKE_MINIMUM_REQUIRED (VERSION 3.12) + +# 指定下载解压后的fastdeploy库路径 +option(FASTDEPLOY_INSTALL_DIR "Path of downloaded fastdeploy sdk.") + +include(${FASTDEPLOY_INSTALL_DIR}/FastDeploy.cmake) + +# 添加FastDeploy依赖头文件 +include_directories(${FASTDEPLOY_INCS}) + +add_executable(infer_demo ${PROJECT_SOURCE_DIR}/infer.cc) +# 添加FastDeploy库依赖 +target_link_libraries(infer_demo ${FASTDEPLOY_LIBS}) diff --git a/new_examples/vision/detection/yolov7/cpp/README.md b/new_examples/vision/detection/yolov7/cpp/README.md new file mode 100644 index 00000000000..29eea1c1d2d --- /dev/null +++ b/new_examples/vision/detection/yolov7/cpp/README.md @@ -0,0 +1,65 @@ +# YOLOv7 C++部署示例 + +本目录下提供`infer.cc`快速完成YOLOv7在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。 + +在部署前，需确认以下两个步骤 + +- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../docs/quick_start/requirements.md) +- 2. 根据开发环境，下载预编译部署库，参考[FastDeploy预编译库](../../docs/compile/prebuild_libraries.md) + +以Linux上CPU推理为例，在本目录执行如下命令即可完成编译测试 + +``` +mkdir build +cd build +wget https://xxx.tgz +tar xvf fastdeploy-linux-x64-0.2.0.tgz +cmake .. -DFASTDEPLOY_INSTALL_DIR=${PWD}/fastdeploy-linux-x64-0.2.0 +make -j + +./infer_demo yolov7.onnx 000001.jpg +``` + +## YOLOv7 C++接口 + +### YOLOv7类 +``` +fastdeploy::vision::wongkinyiu::YOLOv7( + const string& model_file, + const string& params_file = "", + const RuntimeOption& runtime_option = RuntimeOption(), + const Frontend& model_format = Frontend::ONNX) +``` +YOLOv7模型加载和初始化，其中model_file为导出的ONNX模型格式。 + +**参数** + +> * **model_file**(str): 模型文件路径 +> * **params_file**(str): 参数文件路径，当模型格式为ONNX时，此参数传入空字符串即可 +> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置 +> * **model_format**(Frontend): 模型格式，默认为ONNX格式 + +#### Predict函数 +> ``` +> YOLOv7::Predict(cv::Mat* im, DetectionResult* result, +> float conf_threshold = 0.25, +> float nms_iou_threshold = 0.5) +> ``` +> 模型预测接口，输入图像直接输出检测结果。 +> +> **参数** +> +> > * **im**: 输入图像，注意需为HWC，BGR格式 +> > * **result**: 检测结果，包括检测框，各个框的置信度, DetectionResult说明参考[视觉模型预测结果](../../../../../docs/api/vision_results/) +> > * **conf_threshold**: 检测框置信度过滤阈值 +> > * **nms_iou_threshold**: NMS处理过程中iou阈值 + + +### 类成员变量 + +> > * **size**(vector): 通过此参数修改预处理过程中resize的大小，包含两个整型元素，表示[width, height], 默认值为[640, 640] + + +- [模型介绍](../../) +- [Python部署](../python) +- [视觉模型预测结果](../../../../../docs/api/vision_results/) diff --git a/new_examples/vision/detection/yolov7/cpp/infer.cc b/new_examples/vision/detection/yolov7/cpp/infer.cc new file mode 100644 index 00000000000..d5d4ad98123 --- /dev/null +++ b/new_examples/vision/detection/yolov7/cpp/infer.cc @@ -0,0 +1,106 @@ +// Copyright (c) 2022 PaddlePaddle Authors. All Rights Reserved. +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +#include "fastdeploy/vision.h" + +void CpuInfer(const std::string& model_file, const std::string& image_file) { + auto model = fastdeploy::vision::detection::YOLOv7(model_file); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + auto im_bak = im.clone(); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(&im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + + auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +void GpuInfer() { + auto option = fastdeploy::RuntimeOption(); + option.UseGpu(); + auto model = fastdeploy::vision::detection::YOLOv7(model_file, "", option); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + auto im_bak = im.clone(); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(&im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + + auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +void TrtInfer() { + auto option = fastdeploy::RuntimeOption(); + option.UseGpu(); + option.UseTrtBackend(); + option.SetTrtInputShape("image", {1, 3, 320, 320}, {1, 3, 640, 640}, + {1, 3, 1280, 1280}); + auto model = fastdeploy::vision::detection::YOLOv7(model_file, "", option); + if (!model.Initialized()) { + std::cerr << "Failed to initialize." << std::endl; + return; + } + + auto im = cv::imread(image_file); + auto im_bak = im.clone(); + + fastdeploy::vision::DetectionResult res; + if (!model.Predict(&im, &res)) { + std::cerr << "Failed to predict." << std::endl; + return; + } + + auto vis_im = fastdeploy::vision::Visualize::VisDetection(im_bak, res); + cv::imwrite("vis_result.jpg", vis_im); + std::cout << "Visualized result saved in ./vis_result.jpg" << std::endl; +} + +int main(int argc, char* argv[]) { + if (argc < 4) { + std::cout << "Usage: infer_demo path/to/model path/to/image run_option, " + "e.g ./infer_model ./yolov7.onnx ./test.jpeg 0" + << std::endl; + std::cout << "The data type of run_option is int, 0: run with cpu; 1: run " + "with gpu; 2: run with gpu and use tensorrt backend." + << std::endl; + return -1; + } + + if (std::atoi(argv[3]) == 0) { + CpuInfer(); + } else if (std::atoi(argv[3]) == 1) { + GpuInfer(); + } else if (std::atoi(argv[3]) == 2) { + TrtInfer(); + } + return 0; +} diff --git a/new_examples/vision/detection/yolov7/python/README.md b/new_examples/vision/detection/yolov7/python/README.md new file mode 100644 index 00000000000..6f9e4536642 --- /dev/null +++ b/new_examples/vision/detection/yolov7/python/README.md @@ -0,0 +1,62 @@ +# YOLOv7 Python部署示例 + +在部署前，需确认以下两个步骤 + +- 1. 软硬件环境满足要求，参考[FastDeploy环境要求](../../docs/quick_start/requirements.md) +- 2. FastDeploy Python安装，参考[FastDeploy Python安装](../../../../../docs/quick_start/install.md) + +本目录下提供`infer.py`快速完成YOLOv7在CPU/GPU，以及GPU上通过TensorRT加速部署的示例。执行如下脚本即可完成 + +``` +#下载yolov7模型文件和测试图片 +wget https://bj.bcebos.com/paddlehub/fastdeploy/yolov7.onnx +wget https://gitee.com/paddlepaddle/PaddleDetection/raw/release/2.4/demo/000000087038.jpg + +python infer.py --model yolov7.onnx --image 000000087038.jpg --device cpu +``` + +运行完成可视化结果如下图所示 + + +## YOLOv7 Python接口 + +``` +fastdeploy.vision.detection.YOLOv7(model_file, params_file=None, runtime_option=None, model_format=Frontend.ONNX) +``` + +YOLOv7模型加载和初始化，其中model_file为导出的ONNX模型格式 + +**参数** + +> * **model_file**(str): 模型文件路径 +> * **params_file**(str): 参数文件路径，当模型格式为ONNX格式时，此参数无需设定 +> * **runtime_option**(RuntimeOption): 后端推理配置，默认为None，即采用默认配置 +> * **model_format**(Frontend): 模型格式，默认为ONNX + +### predict函数 + +> ``` +> YOLOv7.predict(image_data, conf_threshold=0.25, nms_iou_threshold=0.5) +> ``` +> 模型预测结口，输入图像直接输出检测结果。 +> +> **参数** +> +> > * **image_data**(np.ndarray): 输入数据，注意需为HWC，BGR格式 +> > * **conf_threshold**(float): 检测框置信度过滤阈值 +> > * **nms_iou_threshold**(float): NMS处理过程中iou阈值 + +> **返回** +> +> > 返回`fastdeploy.vision.DetectionResult`结构体，结构体说明参考文档[视觉模型预测结果](../../../../../docs/api/vision_results/) + +### 类成员属性 + +> > * **size**(list | tuple): 通过此参数修改预处理过程中resize的大小，包含两个整型元素，表示[width, height], 默认值为[640, 640] + + +## 其它文档 + +- [YOLOv7 模型介绍](..) +- [YOLOv7 C++部署](../cpp) +- [模型预测结果说明](../../../../../docs/api/vision_results/) diff --git a/new_examples/vision/detection/yolov7/python/infer.py b/new_examples/vision/detection/yolov7/python/infer.py new file mode 100644 index 00000000000..acc8c3a5094 --- /dev/null +++ b/new_examples/vision/detection/yolov7/python/infer.py @@ -0,0 +1,52 @@ +import fastdeploy as fd +import cv2 + + +def parse_arguments(): + import argparse + import ast + parser = argparse.ArgumentParser() + parser.add_argument( + "--model", required=True, help="Path of yolov7 onnx model.") + parser.add_argument( + "--image", required=True, help="Path of test image file.") + parser.add_argument( + "--device", + type=str, + default='cpu', + help="Type of inference device, support 'cpu' or 'gpu'.") + parser.add_argument( + "--use_trt", + type=ast.literal_eval, + default=False, + help="Wether to use tensorrt.") + return parser.parse_args() + + +def build_option(args): + option = fd.RuntimeOption() + + if args.device.lower() == "gpu": + option.use_gpu() + + if args.use_trt: + option.use_trt_backend() + option.set_trt_input_shape("image", [1, 3, 320, 320], + [1, 3, 640, 640], [1, 3, 1280, 1280]) + return option + + +args = parse_arguments() + +# 配置runtime，加载模型 +runtime_option = build_option(args) +model = fd.vision.detection.YOLOv7(args.model, runtime_option=runtime_option) + +# 预测图片检测结果 +im = cv2.imread(args.image) +result = model.predict(im) + +# 预测结果可视化 +vis_im = fd.vision.vis_detection(im, result) +cv2.imwrite("visualized_result.jpg", vis_im) +print("Visualized result save in ./visualized_result.jpg")