[model reasoning] deep stream 6.0 deployment yolov3 and yolov4 tutorial

Welcome to my official account, reply to 001 Google programming specification.

  O_o   >_<   o_O   O_o   ~_~   o_O

  Hello, I'm Jizhi horizon. This article introduces the method of deploying yolov3 and yolov4 using deep stream 6.0.

    yolo series are widely used target detection algorithms in engineering, especially from yolov3 to yolov4 and yolov5. Deepstream is a pipeline application proposed by NVIDIA to accelerate the landing of in-depth learning. When deepstream meets yolo, what sparks will it spark? Let's see.

    For the installation tutorial of deepstream, please refer to the previous articles I wrote:< [experience sharing] ubuntu installs deepstream6.0>,<[experience sharing] ubuntu installs deepstream5.1>.

    Let's take a look at the directory structure of deepstream6.0 source:

  • apps

    • apps-common
    • audio_apps
    • sample_apps: routines, such as deepstream app, deepstream test1
  • GST plugins: gstreamer plug-in
  • include: header
  • libs: Library
  • objectDetector_FasterRCNN: FasterRCNN example
  • objectDetector_SSD: SSD example
  • objectDetector_Yolo: YOLO example
  • tools: log related

1. Deep stream 6.0 deployment yolov3

  run yolov3 through the above objectDetector_Yolo project. In the objectDetector_Yolo project, we mainly focus on the following modules:

  • nvdsinfer_custom_impl_Yolo: yolov3 project implementation code;

    • nvdsinfer_yolo_engine.cpp: analytical model and generation engine
    • nvdsparsebbox_Yolo.cpp: analytic function of output layer, analytic target detection box
    • trt_utils.cpp and trt_utils.h: interface and implementation of tool class for constructing TensorRT network
    • yolo.cpp and yolo.h: interface and Implementation for generating yolo engine
    • yoloPlugins.cpp and yoloPlugins.h: interface and implementation of YoloLayerV3 and YoloLayerV3PluginCreator
    • kernels.cu: cuda core underlying implementation
  • config_infer_xxx_.txt: configuration of the model;
  • deepstream_app_config_xxx.txt: configuration file of Gstreamer nvinfer plug-in;
  • xxx.cfg, xxx.weights: model files;

    That's enough. Let's start.

1.1 download model file

  there is no yolov3 model file in the deepstream6.0 SDK. You need to download it yourself and send it.

   yolov3.cfg: https://github.com/pjreddie/d...；

  yolov3.weights: https://link.zhihu.com/?targe...；

    One more thing to say here is that if you have TensorRT yolov3.engine, you don't need the original model file. If you don't have. Engine, you will actually become. Engine according to the original file.

1.2 configure config_infer_primary_yolov3.txt

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
custom-network-config=yolov3.cfg
model-file=yolov3.weights
labelfile-path=labels.txt
int8-calib-file=yolov3-calibration.table.trt7.0
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=1
num-detected-classes=80
gie-unique-id=1
network-type=0
is-classifier=0
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV3
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so
engine-create-func-name=NvDsInferYoloCudaEngineGet

[class-attrs-all]
nms-iou-threshold=0.3
threshold=0.7

1.3 configure deepstream_app_config_yolov3.txt

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=3
uri=file://../../samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
#model-engine-file=model_b1_gpu0_int8.engine
labelfile-path=labels.txt
batch-size=1
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=2
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tracker]
enable=1
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so
ll-config-file=../../samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml
gpu-id=0
enable-batch-process=1
enable-past-frame=1
display-tracking-id=1

[tests]
file-loop=0

1.4 project compilation

    Go to / opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo:

cd /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo

    Execute the following two commands in sequence to compile and generate a. so file:

export CUDA_VER=11.4    # Set the same CUDA version as the device

    Or modify it in / opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/Makefile:

    Then perform compilation

make -C nvdsinfer_custom_impl_Yolo

     After compilation, the dynamic library file will be produced, and the libnvdsinfer_custom_impl_Yolo.so dynamic library file will be generated.

1.5 execution

deepstream-app -c deepstream_app_config_yoloV3.txt

     Here, the deployment of deep stream 6.0 yolov3 is completed.

2. Deep stream 6.0 deployment yolov4

  here we deploy yolov4 in different ways, that is, directly call TensorRT Engine instead of importing from the original model.

2.1 generate yolov4.engine using darknet2onnx2TRT

    Download yolov4 darknet original weight and give Baidu online disk transmission:

https://pan.baidu.com/s/1dAGEW8cm-dqK14TbhhVetA     Extraction code:dm5b

  clone model conversion project:

git clone https://github.com/Tianxiaomo/pytorch-YOLOv4.git Yolov42TRT

    Start model conversion:

cd Yolov42TRT

# darknet2onnx
python demo_darknet2onnx.py ./cfg/yolov4.cfg ./cfg/yolov4.weights ./data/dog.jpg 1

# onnx2trt
trtexec --onnx=./yolov4_1_3_608_608_static.onnx --fp16 --saveEngine=./yolov4.engine --device=0

     This will generate yolov4.engine.

2.2 deep stream yolov4 reasoning engineering configuration

    clone deepstream yolov4 reasoning project:

git clone https://github.com/NVIDIA-AI-IOT/yolov4_deepstream.git

cd yolov4_deepstream/deepstream_yolov4

    Configure config_infor_primary_yolov4.txt:

[property]
gpu-id=0
net-scale-factor=0.0039215697906911373
#0=RGB, 1=BGR
model-color-format=0
model-engine-file=yolov4.engine
labelfile-path=labels.txt
batch-size=1
## 0=FP32, 1=INT8, 2=FP16 mode
network-mode=2
num-detected-classes=80
gie-unique-id=1
network-type=0
is-classifier=0
## 0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering)
cluster-mode=2
maintain-aspect-ratio=1
parse-bbox-func-name=NvDsInferParseCustomYoloV4
custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so

[class-attrs-all]
nms-iou-threshold=0.6
pre-cluster-threshold=0.4

     Configure deepstream_app_config_yoloV4.txt:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=3
uri=file:/opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_1080p_h264.mp4
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=3
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0
container=1
codec=1
output-file=yolov4.mp4

[osd]
enable=1
gpu-id=0
border-width=1
text-size=12
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
live-source=0
batch-size=1
batched-push-timeout=40000
width=1280
height=720
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=1
gpu-id=0
model-engine-file=yolov4.engine
labelfile-path=labels.txt
batch-size=1

bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV4.txt

[tracker]
enable=0
tracker-width=512
tracker-height=320
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

[tests]
file-loop=0

    Copy yolov4.engine generated by 2.1 transformation to / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream.

2.3 project compilation

    Go to / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream:

cd /opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream

    Execute the following two commands in sequence to compile and generate a. so file:

export CUDA_VER=11.4    # Set the same CUDA version as the device

    Or modify it in / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream/nvdsinfer_custom_impl_Yolo/Makefile:

    Then perform compilation

make -C nvdsinfer_custom_impl_Yolo

     After compilation, the dynamic library file will be produced, and the libnvdsinfer_custom_impl_Yolo.so dynamic library file will be generated.

2.4 execution

deepstream-app -c deepstream_app_config_yoloV4.txt

    Here, the deployment of deep stream 6.0 yolov4 is completed.

    The above shared the methods of deploying yolov3 and yolov4 in deep stream 6.0. I hope my sharing will be helpful to your learning.

[official account transmission]
<[model reasoning] deep stream 6.0 deployment yolov3 and yolov4 tutorial>

Tags: Algorithm AI Deep Learning gpu nvidia

Posted on Fri, 26 Nov 2021 01:51:12 -0500 by reyjrar

Hot Tags

©2022 Programmer Help