测试了rvm和scrfd都比lite.ai.toolkit里的慢

[chfeizy]

rvm在这里用时大概 660ms到750ms
lite.ai.toolkit 大概 115ms到230ms

scrfd这里大概11ms到30ms
lite.ai.toolkit 大概 8ms到20ms

环境是MacBook Pro 2.6 GHz 六核Intel Core i7

[DefTruth]

@chfeizy 您好，请问可以提供一下您的测试代码吗？

[chfeizy]

long getCurrentTime()
{
    return std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
}
int main()
{
    std::string onnx_path = "/Users/also/www/FastDeploy/examples/vision/matting/rvm/cpp/rvm_mobilenetv3_fp32.onnx";

    auto option = fastdeploy::RuntimeOption();
  auto model = fastdeploy::vision::matting::RobustVideoMatting(onnx_path, "", option);
  if (!model.Initialized()) {
    std::cerr << "Failed to initialize." << std::endl;
    return 0;
  }
    fastdeploy::vision::MattingResult res;

    cv::VideoCapture cap;
    cv::Mat im;
    cap.open(0);
    if(! cap.isOpened())
    {
        std::cerr << "Cannot open the camera." << std::endl;
        return 0;
    }

    if( cap.isOpened()) {
        while (true) {
            cap >> im;
            int capture_width = cap.get(cv::CAP_PROP_FRAME_WIDTH);
            int capture_height = cap.get(cv::CAP_PROP_FRAME_HEIGHT);
            std::cout << "Image size: " << im.rows << "X" << im.cols << std::endl;
            long start= getCurrentTime();
            if (!model.Predict(&im, &res)) {
                std::cerr << "Failed to predict." << std::endl;
                return 0;
              }
            cv::Mat img_bgr = im.clone();

            cv::Mat vis_im = fastdeploy::vision::VisMatting(img_bgr, res);

            cv::imshow("result", vis_im);


            long end= getCurrentTime();

            std::cout << "time: " << end-start << std::endl;

            if((cv::waitKey(2)& 0xFF) == 'q')
                break;
        }
    }
  return 0;
}

rvm 的

[DefTruth]

@chfeizy lite.ai.toolkit的测试代码有吗？方便也提供一份不？

[chfeizy]

long getCurrentTime()
{
    return std::chrono::duration_cast<std::chrono::milliseconds>(std::chrono::system_clock::now().time_since_epoch()).count();
}

int main(__unused int argc, __unused char *argv[])
{
//  test_lite();
    std::string onnx_path = "/Users/also/www/lite.ai.toolkit/hub/onnx/cv/rvm_mobilenetv3_fp32_2.onnx";
//    std::string onnx_path = "/Users/also/www/lite.ai.toolkit/hub/onnx/cv/rvm_resnet50_fp32.onnx";
    std::string video_path = "../../../examples/lite/resources/test_lite_rvm_0.mp4";
    std::string background_path = "/Users/also/www/lite.ai.toolkit/examples/lite/resources/r.png";
    std::string output_path = "../../../logs/test_lite_rvm_0.mp4";

    cv::Mat background = cv::imread(background_path);
    auto *rvm = new lite::cv::matting::RobustVideoMatting(onnx_path, 4); // 16 threads
    lite::types::MattingContent content;

    cv::VideoCapture cap;
    cv::Mat im;
    cap.open(0);
    if(! cap.isOpened())
    {
        std::cerr << "Cannot open the camera." << std::endl;
        return 0;
    }

    if( cap.isOpened()) {
        while (true) {
            cap >> im;
            int capture_width = cap.get(cv::CAP_PROP_FRAME_WIDTH);
            int capture_height = cap.get(cv::CAP_PROP_FRAME_HEIGHT);
            //cout << "Image size: " << im.rows << "X" << im.cols << endl;
            long start= getCurrentTime();
            cv::Mat img_bgr = im.clone();
            cv::Mat det_input;
            rvm->detect(img_bgr, content, 0.4f, true, true, false);

            if (content.flag)
            {
                if (!content.merge_mat.empty()){
                    cv::imshow("result", content.merge_mat);
                }
                std::cout << "Default Version MobileHumanMatting Done!" << std::endl;
            }

            //delete fast_portrait_seg;

            long end= getCurrentTime();

            std::cout << "time: " << end-start << std::endl;

            if((cv::waitKey(2)& 0xFF) == 'q')
                break;
        }
    }
  return 0;
}

参数配置的0.4

[DefTruth]

感谢您的反馈哈~ 我们的同学正在排查具体的原因

[wjj19950828]

@chfeizy 这里有两点需要对齐下

• lite.ai.toolkit默认CPU后端为ONNXRuntime
• lite.ai.toolkit线程数使用为4个线程

所以先对齐一下，在FastDeploy 模型初始化阶段使用如下代码

auto option = fastdeploy::RuntimeOption();
option.UseOrtBackend();
option.SetCpuThreadNum(4)
auto model = fastdeploy::vision::matting::RobustVideoMatting(onnx_path, "", option);

[chfeizy]

(base) also@alsoMacBookPro build % ./infer_demo [INFO] fastdeploy/runtime.cc(496)::Init Runtime initialized with Backend::ORT in Device::CPU. Image size: 720X1280 time: 689 Image size: 720X1280 time: 757 Image size: 720X1280 time: 450 Image size: 720X1280 time: 463 Image size: 720X1280 time: 361 Image size: 720X1280 time: 385 Image size: 720X1280 time: 365 Image size: 720X1280 time: 384 Image size: 720X1280 time: 357 Image size: 720X1280 time: 389 Image size: 720X1280 time: 601 Image size: 720X1280 time: 448 Image size: 720X1280 time: 450 Image size: 720X1280 time: 364 Image size: 720X1280 time: 358 Image size: 720X1280 time: 437 Image size: 720X1280 time: 364 Image size: 720X1280 time: 472 Image size: 720X1280 time: 463 Image size: 720X1280 time: 430 Image size: 720X1280 time: 367 Image size: 720X1280 time: 313 Image size: 720X1280 time: 337 Image size: 720X1280 time: 282 Image size: 720X1280 time: 303 Image size: 720X1280 time: 354 Image size: 720X1280 time: 359 Image size: 720X1280

基本上还是有 280到360ms左右，还是慢的

[wjj19950828]

@chfeizy 目前RVM前后处理做了优化，可以参考这两个PR：
#658
#679

我这边自测了下，在我自己的Mac机器中：

CPU：Intel(R) Core(TM) i5-8257U CPU @ 1.40GHz
Python：3.7
Torch：1.12.1
共100次测速，前20次warmup

版本	Runtime(ms)	End2End(ms)
RVM TorchScript	2247.62	2277.16
FastDeploy 优化前	103.07	238.79
FastDeploy 优化后	102.79	113.78

可以看最后一行End2End，优化较明显

[ziqi-jin]

你好，我测试了SCRFD，结果如下
硬件环境：Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz
线程：4
Backend: OnnxRuntime
循环次数：100
FastDeploy: 20ms/pic
Lite: 24ms/pic
如果您希望进一步提高SCRFD的在FastDeploy的性能，可以参考RVM的优化方案。

[chfeizy]

rvm 测试还是200到240ms感觉还是差点

[chfeizy]

测试的 是 c++版本吗好像是 python 的