DeepStream-Yolo

NVIDIA DeepStream SDK 5.1 configuration for YOLO models

Improvements on this repository

• Darknet CFG params parser (not need to edit nvdsparsebbox_Yolo.cpp or another file for native models)
• Support to new_coords, beta_nms and scale_x_y params
• Support to new models not supported in official DeepStream SDK YOLO.
• Support to layers not supported in official DeepStream SDK YOLO.
• Support to activations not supported in official DeepStream SDK YOLO.
• Support to Convolutional groups

Tutorial

• Configuring to your custom model
• Multiple YOLO inferences

Benchmark

• mAP/FPS comparison between models

TensorRT conversion

• Native (tested models below)

  • YOLOv4x-Mish
  • YOLOv4-CSP
  • YOLOv4
  • YOLOv4-Tiny
  • YOLOv3-SSP
  • YOLOv3
  • YOLOv3-Tiny-PRN
  • YOLOv3-Tiny
  • YOLOv3-Lite
  • YOLOv3-Nano
  • YOLO-Fastest
  • YOLO-Fastest-XL
  • YOLOv2
  • YOLOv2-Tiny

• External

  • YOLOv5

Request

• Request native TensorRT conversion for your YOLO-based model

Requirements

• NVIDIA DeepStream SDK 5.1
• DeepStream-Yolo Native (for Darknet YOLO based models)
• DeepStream-Yolo External (for PyTorch YOLOv5 based model)

mAP/FPS comparison between models (OUTDATED)

DeepStream SDK YOLOv4: https://youtu.be/Qi_F_IYpuFQ

Darknet YOLOv4: https://youtu.be/AxJJ9fnJ7Xk

CUDA 10.2
Driver 440.33
TensorRT 7.2.1
cuDNN 8.0.5
OpenCV 3.2.0 (libopencv-dev)
OpenCV Python 4.4.0 (opencv-python)
PyTorch 1.7.0
Torchvision 0.8.1

JetPack 4.4.1
CUDA 10.2
TensorRT 7.1.3
cuDNN 8.0
OpenCV 4.1.1

DeepStream settings

• General

width = 1920
height = 1080
maintain-aspect-ratio = 0
batch-size = 1

• Evaluate mAP

valid = val2017 (COCO)
nms-iou-threshold = 0.6
pre-cluster-threshold = 0.001 (CONF_THRESH)

• Evaluate FPS and Demo

nms-iou-threshold = 0.45 (NMS; changed to beta_nms when available)
pre-cluster-threshold = 0.25 (CONF_THRESH)

Native TensorRT conversion

Run command

sudo chmod -R 777 /opt/nvidia/deepstream/deepstream-5.1/sources/

Download my native folder, rename to yolo and move to your deepstream/sources folder.

Download cfg and weights files from your model and move to deepstream/sources/yolo folder.

• YOLOv4x-Mish [cfg] [weights]
• YOLOv4-CSP [cfg] [weights]
• YOLOv4 [cfg] [weights]
• YOLOv4-Tiny [cfg] [weights]
• YOLOv3-SPP [cfg] [weights]
• YOLOv3 [cfg] [weights]
• YOLOv3-Tiny-PRN [cfg] [weights]
• YOLOv3-Tiny [cfg] [weights]
• YOLOv3-Lite [cfg] [weights]
• YOLOv3-Nano [cfg] [weights]
• YOLO-Fastest [cfg] [weights]
• YOLO-Fastest-XL [cfg] [weights]
• YOLOv2 [cfg] [weights]
• YOLOv2-Tiny [cfg] [weights]

Compile

• x86 platform

cd /opt/nvidia/deepstream/deepstream-5.1/sources/yolo
CUDA_VER=11.1 make -C nvdsinfer_custom_impl_Yolo

• Jetson platform

cd /opt/nvidia/deepstream/deepstream-5.1/sources/yolo
CUDA_VER=10.2 make -C nvdsinfer_custom_impl_Yolo

Edit config_infer_primary.txt for your model (example for YOLOv4)

[property]
...
# 0=RGB, 1=BGR, 2=GRAYSCALE
model-color-format=0
# CFG
custom-network-config=yolov4.cfg
# Weights
model-file=yolov4.weights
# Generated TensorRT model (will be created if it doesn't exist)
model-engine-file=model_b1_gpu0_fp32.engine
# Model labels file
labelfile-path=labels.txt
# Batch size
batch-size=1
# 0=FP32, 1=INT8, 2=FP16 mode
network-mode=0
# Number of classes in label file
num-detected-classes=80
...
[class-attrs-all]
# CONF_THRESH
pre-cluster-threshold=0.25

Run

deepstream-app -c deepstream_app_config.txt

If you want to use YOLOv2 or YOLOv2-Tiny models, change, before run, deepstream_app_config.txt

[primary-gie]
enable=1
gpu-id=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV2.txt

Note: config_infer_primary.txt uses cluster-mode=4 and NMS = 0.45 (via code) when beta_nms isn't available (when beta_nms is available, NMS = beta_nms), while config_infer_primary_yoloV2.txt uses cluster-mode=2 and nms-iou-threshold=0.45 to set NMS.

Request native TensorRT conversion for your YOLO-based model

To request moded files for native TensorRT conversion to use in DeepStream SDK, send me the model cfg and weights files via Issues tab.

Note: If your model are listed in native tab, you can use my native folder to run your model in DeepStream.

Extract metadata

You can get metadata from deepstream in Python and C++. For C++, you need edit deepstream-app or deepstream-test code. For Python your need install and edit deepstream_python_apps.

You need manipulate NvDsObjectMeta (Python/C++), NvDsFrameMeta (Python/C++) and NvOSD_RectParams (Python/C++) to get label, position, etc. of bboxs.

In C++ deepstream-app application, your code need be in analytics_done_buf_prob function. In C++/Python deepstream-test application, your code need be in osd_sink_pad_buffer_probe/tiler_src_pad_buffer_probe function.

Python is slightly slower than C (about 5-10%).

This code is open-source. You can use as you want. :)

If you want me to create commercial DeepStream SDK projects for you, contact me at email address available in GitHub.

My projects: https://www.youtube.com/MarcosLucianoTV