Day 19 - installing AlexeyAB/darknet ON Amazon Linux 2
Today's task plan installs the AlexeyAB/darknet version of YOLO, which is the main author of yoov4. This version can produce more measurement indicators. Of course, you can also use yoov4 for image recognition, but it is difficult to install.
The following AWS EC2 is required for the following installation settings:
- EC2 Instance type: g4dn.2xlarge
- AMI ID: ami-0cccf4ac9f2e9bd92
- AMI Name: Deep Learning AMI (Amazon Linux 2) Version 49.0
Figure 1. EC2 configuration information
After EC2 is started, log in to the EC2 host through ssh protocol and update all packages first.
sudo yum update -y
According to the instructions on AlexeyAB/darknet github website, the following components need to be installed first:
- CMake >= 3.18.
- Powershell : it is related to the Windows environment and does not need to be installed.
- CUDA >= 10.2.
- OpenCV >= 2.4.
- cuDNN >= 8.0.2.
- GPU with Compute Capability (CC) >= 3.0:
Check cmake version > = 3.18
Check whether the version meets the system requirements. Check cmake. The version is 3.20.4, which meets the requirements greater than 3.18. The check result using rpm instruction is self installed, not yum.
cmake --version which cmake rpm -qf /usr/local/bin/cmake
Figure 2. Checking cmake version
The following is to update the original cmake to version 3.21, download the original file from the official cmake website, install the required development kit, recompile and install. In principle, our installation is in the user directory, so we will go to the user directory before downloading.
sudo yum remove cmake cmake3 sudo yum install mesa-libGL mesa-libGL-devel -y cd ~ mkdir cmake && cd cmake wget https://github.com/Kitware/CMake/releases/download/v3.21.0/cmake-3.21.0.tar.gz tar zxfv cmake-3.21.0.tar.gz cd cmake-3.21.0/ ./bootstrap # It takes about 20 minutes to find the configuration of the current system and generate a Makefile make # It takes about 20 minutes to compile sudo make install # Installation requires administrator privileges because it will be installed in the / usr/local/bin directory cd ~
Check CUDA version > = 10.2
Check the version of CUDA. It is found that multiple versions of CUDA are installed in this AMI. Use NVIDIA SMI to watch the operation of GPU. It is found that GPU uses 11.0, but the currently available version is 10.0. Five versions such as 10.0, 10.1, 10.2, 11.1 and 11.1 are installed in the whole system.
nvidia-smi nvcc --version ls -ld /usr/local/cuda*
Figure 3. Check CUDA version
Delete the original link and specify it again, so that it can correspond to CUDA version 11.0.
sudo rm /usr/local/cuda sudo ln -s /usr/local/cuda-11.0 /usr/local/cuda export CUDA_BIN_PATH="/usr/local/cuda-11.0/bin" export CUDA_HOME="/usr/local/cuda-11.0/" export CUDA_PATH="/usr/local/cuda-11.0/"
Figure 4. Setting CUDA version to 11.0
Check cuDNN version > = 8.0.2
The version check of cuDNN is the following instruction, and the version number of cuDNN will be displayed. The version installed in AMI is less than 8.0.2, which can be referred to Installing cuDNN On Linux To install this, you have to register with nVidia before you can download it, so you don't care about it. (a self willed author who hates to register new accounts all the time)
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
Check OpenCV version > = 2.4
Use the following three methods to find:
- Through suite Manager: rpm
- Suite configuration in linux: PKG config
- File of the whole disk: locate / find
The first two were not found, and the third one was found, but it was incomplete, so I had to install it myself.
rpm -qa | grep opencv pkg-config --cflags --libs opencv4 sudo updatdb locate opencv4
The instructions for installing opencv 4.2 are as follows. The installation includes opencv and its extension module contrib:
sudo yum install git cmake gcc-c++ mkdir opencv && cd opencv # Download original file wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.2.0.zip wget -O opencv.zip https://github.com/opencv/opencv/archive/4.2.0.zip unzip opencv.zip unzip opencv_contrib.zip mv opencv-4.2.0/ opencv mv opencv_contrib-4.2.0/ opencv_contrib mkdir -p build && cd build # ../opencv_contrib/modules / is an additional module, - DOPENCV_GENERATE_PKGCONFIG=ON is the installation configuration file of the Creation Kit,.. / opencv is the directory where the opencv original file is located, cmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules -DOPENCV_GENERATE_PKGCONFIG=ON ../opencv make # It takes about an hour to compile sudo make install # Install OpenCV, this is very fast # Confirm that opencv's share library can be accessed by other programs more /etc/ld.so.conf.d/opencv.conf sudo ldconfig -v
Figure 5. Sharing library with OpenCV enabled
Set the installation environment of OpenCV and add these two lines in. bashrc under the user directory, as shown in the following figure.
PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib64/pkgconfig/ export PKG_CONFIG_PATH
Figure 6. Setting user startup file
Finally, check the opencv installation. The results are shown in the figure below.
pkg-config --cflags --libs opencv4
Figure 7. Setting user startup file
Check GPU with compute capability (CC) > = 3.0
Connect directly GPU with Compute Capability (CC) On the website, you can directly find the GPU model of Tesla T4, as shown in the figure below. The version number is 7.5.
Figure 8. GPU computing architecture model
After checking the relevant software, start to install AlexeyAB/darknet, enter the following commands to set the directory, download the original file, establish the compilation environment, compile and install. Then download the weight file of the trained yolov4 and identify it. The execution screen is shown in the figure below. It can be found that compared with yolov3, one more potted plant is identified.
cd mkdir AlexeyAB && cd AlexeyAB git clone https://github.com/AlexeyAB/darknet . mkdir build_release&& cd build_release cmake .. cmake --build . --target install --parallel 8 cd .. wget https://github.com/AlexeyAB/darknet/releases/download/darknet_yolo_v3_optimal/yolov4.weights ./darknet detector test ./cfg/coco.data ./cfg/yolov4.cfg ./yolov4.weights data/dog.jpg
Figure 9. Execute yolov4 for identification
Fig. 10 yolov4 identification results
- Install Cmake 3 on AWS Linux, https://www.matbra.com/2017/12/07/install-cmake-on-aws-linux.html
- Yolo v4, v3 and v2 for Windows and Linux, https://github.com/AlexeyAB/darknet
- How to install OpenCV on Amazon Linux?, https://stackoverflow.com/questions/34244606/how-to-install-opencv-on-amazon-linux/34245634
- Installation in Linux, https://docs.opencv.org/master/d7/d9f/tutorial_linux_install.html
- Package opencv was not found in the pkg-config search path, https://stackoverflow.com/questions/15320267/package-opencv-was-not-found-in-the-pkg-config-search-path
- cmake-3.18.1, https://cmake.org/files/v3.18/cmake-3.18.1.tar.gz
- Installing cuDNN On Linux, https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installlinux-tar
- Library for Windows and Linux, Ubuntu(x86_64, armsbsa, PPC architecture), https://developer.nvidia.com/rdp/cudnn-archive