Welcome to my official account, reply to 001 Google programming specification.
O_o >_< o_O O_o ~_~ o_O
Hello, I'm Jizhi horizon. This article introduces the method of deploying yolov3 and yolov4 using deep stream 6.0.
yolo series are widely used target detection algorithms in engineering, especially from yolov3 to yolov4 and yolov5. Deepstream is a pipeline application proposed by NVIDIA to accelerate the landing of in-depth learning. When deepstream meets yolo, what sparks will it spark? Let's see.
For the installation tutorial of deepstream, please refer to the previous articles I wrote:< [experience sharing] ubuntu installs deepstream6.0>,<[experience sharing] ubuntu installs deepstream5.1>.
Let's take a look at the directory structure of deepstream6.0 source:
apps
- apps-common
- audio_apps
- sample_apps: routines, such as deepstream app, deepstream test1
- GST plugins: gstreamer plug-in
- include: header
- libs: Library
- objectDetector_FasterRCNN: FasterRCNN example
- objectDetector_SSD: SSD example
- objectDetector_Yolo: YOLO example
- tools: log related
1. Deep stream 6.0 deployment yolov3
run yolov3 through the above objectDetector_Yolo project. In the objectDetector_Yolo project, we mainly focus on the following modules:
nvdsinfer_custom_impl_Yolo: yolov3 project implementation code;
- nvdsinfer_yolo_engine.cpp: analytical model and generation engine
- nvdsparsebbox_Yolo.cpp: analytic function of output layer, analytic target detection box
- trt_utils.cpp and trt_utils.h: interface and implementation of tool class for constructing TensorRT network
- yolo.cpp and yolo.h: interface and Implementation for generating yolo engine
- yoloPlugins.cpp and yoloPlugins.h: interface and implementation of YoloLayerV3 and YoloLayerV3PluginCreator
- kernels.cu: cuda core underlying implementation
- config_infer_xxx_.txt: configuration of the model;
- deepstream_app_config_xxx.txt: configuration file of Gstreamer nvinfer plug-in;
- xxx.cfg, xxx.weights: model files;
That's enough. Let's start.
1.1 download model file
there is no yolov3 model file in the deepstream6.0 SDK. You need to download it yourself and send it.
yolov3.cfg: https://github.com/pjreddie/d...;
yolov3.weights: https://link.zhihu.com/?targe...;
One more thing to say here is that if you have TensorRT yolov3.engine, you don't need the original model file. If you don't have. Engine, you will actually become. Engine according to the original file.
1.2 configure config_infer_primary_yolov3.txt
[property] gpu-id=0 net-scale-factor=0.0039215697906911373 #0=RGB, 1=BGR model-color-format=0 custom-network-config=yolov3.cfg model-file=yolov3.weights labelfile-path=labels.txt int8-calib-file=yolov3-calibration.table.trt7.0 ## 0=FP32, 1=INT8, 2=FP16 mode network-mode=1 num-detected-classes=80 gie-unique-id=1 network-type=0 is-classifier=0 cluster-mode=2 maintain-aspect-ratio=1 parse-bbox-func-name=NvDsInferParseCustomYoloV3 custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so engine-create-func-name=NvDsInferYoloCudaEngineGet [class-attrs-all] nms-iou-threshold=0.3 threshold=0.7
1.3 configure deepstream_app_config_yolov3.txt
[application] enable-perf-measurement=1 perf-measurement-interval-sec=5 [tiled-display] enable=1 rows=1 columns=1 width=1280 height=720 gpu-id=0 nvbuf-memory-type=0 [source0] enable=1 type=3 uri=file://../../samples/streams/sample_1080p_h264.mp4 num-sources=1 gpu-id=0 cudadec-memtype=0 [sink0] enable=1 #Type - 1=FakeSink 2=EglSink 3=File type=2 sync=0 source-id=0 gpu-id=0 nvbuf-memory-type=0 [osd] enable=1 gpu-id=0 border-width=1 text-size=15 text-color=1;1;1;1; text-bg-color=0.3;0.3;0.3;1 font=Serif show-clock=0 clock-x-offset=800 clock-y-offset=820 clock-text-size=12 clock-color=1;0;0;0 nvbuf-memory-type=0 [streammux] gpu-id=0 live-source=0 batch-size=1 batched-push-timeout=40000 width=1920 height=1080 enable-padding=0 nvbuf-memory-type=0 [primary-gie] enable=1 gpu-id=0 #model-engine-file=model_b1_gpu0_int8.engine labelfile-path=labels.txt batch-size=1 bbox-border-color0=1;0;0;1 bbox-border-color1=0;1;1;1 bbox-border-color2=0;0;1;1 bbox-border-color3=0;1;0;1 interval=2 gie-unique-id=1 nvbuf-memory-type=0 config-file=config_infer_primary_yoloV3.txt [tracker] enable=1 tracker-width=640 tracker-height=384 ll-lib-file=/opt/nvidia/deepstream/deepstream-6.0/lib/libnvds_nvmultiobjecttracker.so ll-config-file=../../samples/configs/deepstream-app/config_tracker_NvDCF_perf.yml gpu-id=0 enable-batch-process=1 enable-past-frame=1 display-tracking-id=1 [tests] file-loop=0
1.4 project compilation
Go to / opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo:
cd /opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo
Execute the following two commands in sequence to compile and generate a. so file:
export CUDA_VER=11.4 # Set the same CUDA version as the device
Or modify it in / opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/nvdsinfer_custom_impl_Yolo/Makefile:
Then perform compilation
make -C nvdsinfer_custom_impl_Yolo
After compilation, the dynamic library file will be produced, and the libnvdsinfer_custom_impl_Yolo.so dynamic library file will be generated.
1.5 execution
deepstream-app -c deepstream_app_config_yoloV3.txt
Here, the deployment of deep stream 6.0 yolov3 is completed.
2. Deep stream 6.0 deployment yolov4
here we deploy yolov4 in different ways, that is, directly call TensorRT Engine instead of importing from the original model.
2.1 generate yolov4.engine using darknet2onnx2TRT
Download yolov4 darknet original weight and give Baidu online disk transmission:
https://pan.baidu.com/s/1dAGEW8cm-dqK14TbhhVetA Extraction code:dm5b
clone model conversion project:
git clone https://github.com/Tianxiaomo/pytorch-YOLOv4.git Yolov42TRT
Start model conversion:
cd Yolov42TRT # darknet2onnx python demo_darknet2onnx.py ./cfg/yolov4.cfg ./cfg/yolov4.weights ./data/dog.jpg 1 # onnx2trt trtexec --onnx=./yolov4_1_3_608_608_static.onnx --fp16 --saveEngine=./yolov4.engine --device=0
This will generate yolov4.engine.
2.2 deep stream yolov4 reasoning engineering configuration
clone deepstream yolov4 reasoning project:
git clone https://github.com/NVIDIA-AI-IOT/yolov4_deepstream.git cd yolov4_deepstream/deepstream_yolov4
Configure config_infor_primary_yolov4.txt:
[property] gpu-id=0 net-scale-factor=0.0039215697906911373 #0=RGB, 1=BGR model-color-format=0 model-engine-file=yolov4.engine labelfile-path=labels.txt batch-size=1 ## 0=FP32, 1=INT8, 2=FP16 mode network-mode=2 num-detected-classes=80 gie-unique-id=1 network-type=0 is-classifier=0 ## 0=Group Rectangles, 1=DBSCAN, 2=NMS, 3= DBSCAN+NMS Hybrid, 4 = None(No clustering) cluster-mode=2 maintain-aspect-ratio=1 parse-bbox-func-name=NvDsInferParseCustomYoloV4 custom-lib-path=nvdsinfer_custom_impl_Yolo/libnvdsinfer_custom_impl_Yolo.so [class-attrs-all] nms-iou-threshold=0.6 pre-cluster-threshold=0.4
Configure deepstream_app_config_yoloV4.txt:
[application] enable-perf-measurement=1 perf-measurement-interval-sec=5 [tiled-display] enable=0 rows=1 columns=1 width=1280 height=720 gpu-id=0 nvbuf-memory-type=0 [source0] enable=1 type=3 uri=file:/opt/nvidia/deepstream/deepstream-6.0/samples/streams/sample_1080p_h264.mp4 num-sources=1 gpu-id=0 cudadec-memtype=0 [sink0] enable=1 #Type - 1=FakeSink 2=EglSink 3=File type=3 sync=0 source-id=0 gpu-id=0 nvbuf-memory-type=0 container=1 codec=1 output-file=yolov4.mp4 [osd] enable=1 gpu-id=0 border-width=1 text-size=12 text-color=1;1;1;1; text-bg-color=0.3;0.3;0.3;1 font=Serif show-clock=0 clock-x-offset=800 clock-y-offset=820 clock-text-size=12 clock-color=1;0;0;0 nvbuf-memory-type=0 [streammux] gpu-id=0 live-source=0 batch-size=1 batched-push-timeout=40000 width=1280 height=720 enable-padding=0 nvbuf-memory-type=0 [primary-gie] enable=1 gpu-id=0 model-engine-file=yolov4.engine labelfile-path=labels.txt batch-size=1 bbox-border-color0=1;0;0;1 bbox-border-color1=0;1;1;1 bbox-border-color2=0;0;1;1 bbox-border-color3=0;1;0;1 interval=0 gie-unique-id=1 nvbuf-memory-type=0 config-file=config_infer_primary_yoloV4.txt [tracker] enable=0 tracker-width=512 tracker-height=320 ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so [tests] file-loop=0
Copy yolov4.engine generated by 2.1 transformation to / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream.
2.3 project compilation
Go to / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream:
cd /opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream
Execute the following two commands in sequence to compile and generate a. so file:
export CUDA_VER=11.4 # Set the same CUDA version as the device
Or modify it in / opt/nvidia/deepstream/deepstream-6.0/sources/yolov4_deepstream/nvdsinfer_custom_impl_Yolo/Makefile:
Then perform compilation
make -C nvdsinfer_custom_impl_Yolo
After compilation, the dynamic library file will be produced, and the libnvdsinfer_custom_impl_Yolo.so dynamic library file will be generated.
2.4 execution
deepstream-app -c deepstream_app_config_yoloV4.txt
Here, the deployment of deep stream 6.0 yolov4 is completed.
The above shared the methods of deploying yolov3 and yolov4 in deep stream 6.0. I hope my sharing will be helpful to your learning.
[official account transmission]
<[model reasoning] deep stream 6.0 deployment yolov3 and yolov4 tutorial>