1. Uninstall the original environment
Original environment: graphics card driver -- 470.57.02 cuda -- 10.0 cudnn -- 7.6 TensorRT 7.0.0.11
Uninstall cuda10.0 and the corresponding cudnn:
cd /usr/local/cuda/bin sudo ./uninstall_cuda_10.0.pl cd .. sudo rm -rf cuda-10.0
2. Install cuda 11.3 and corresponding cudnn
cuda installation
cuda Each version is available in https://developer.nvidia.com/cuda-toolkit-archive Download
The options for downloading cuda 11.3 are as follows:
sudo wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600 sudo wget https://developer.download.nvidia.com/compute/cuda/11.3.0/local_installers/cuda-repo-ubuntu1804-11-3-local_11.3.0-465.19.01-1_amd64.deb sudo dpkg -i cuda-repo-ubuntu1804-11-3-local_11.3.0-465.19.01-1_amd64.deb sudo apt-key add /var/cuda-repo-ubuntu1804-11-3-local/7fa2af80.pub sudo apt-get update sudo apt-get -y install cuda
A pit is encountered here. An error is reported in the last sentence. The error message is:
cuda : rely on: cuda-11-3 (>= 11.3.0) But it will not be installed
Solution:
1. install aptitude-> sudo apt-get install aptitude 2.take apt-get change into aptitude -> sudo aptitude install cuda 3.There are many conflicts displayed, including the prompt that the 470 drive needs to be replaced with the 465 drive. choice y Install (here I choose) y Not installed, And there was no response. Instead, I chose first n,After that, I chose y Before the installation is successful, which can be used as a reference) 4. restart -> nvidia-smi 5. nvcc --version Display version number, successful
The final NVIDIA SMI results are as follows:
nvcc --version results are as follows:
Configure environment variables
sudo gedit ~/.bashrc Add at the end of the file export PATH=/usr/local/cuda-11.3/bin${PATH:+:${PATH}} export LD_LIBRARY_PATH=/usr/local/cuda/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} export CUDA_HOME=/usr/local/cuda Save and enter source ~/.bashrc take effect
Is the test effective
cd /usr/local/cuda-11.3/samples/1_Utilities/deviceQuery sudo make ./deviceQuery
If the result is as follows, the configuration is successful
cudnn installation
The download address is: https://developer.nvidia.com/rdp/cudnn-archive
What I choose here is
After downloading the compressed package, unzip the compressed package and cd to the unzipped file path / open the terminal under this path
1.Input separately sudo cp cuda/include/cudnn.h /usr/local/cuda-11.3/include/ sudo cp cuda/include/cudnn_version.h /usr/local/cuda-11.3/include/ sudo cp cuda/lib64/libcudnn* /usr/local/cuda-11.3/lib64/ sudo chmod a+r /usr/local/cuda-11.3/include/cudnn.h 2.Enter the following code to view the version cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2
If the results are as follows, the installation is successful
3. Install TensorRT 8.0
On the web site https://developer.nvidia.com/nvidia-tensorrt-8x-download download
The corresponding version is the third one in the figure below
Official website Guide: NVIDIA Deep Learning TensorRT Documentation
The downloaded package is: nv-tensorrt-repo-ubuntu 1804-cuda11.3-trt8.0.1.6-ga-20210626_ 1-1_ amd64.deb
Switch to the directory of the download package and execute:
sudo dpkg -i nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626_1-1_amd64.deb sudo apt-key add /var/nv-tensorrt-repo-ubuntu1804-cuda11.3-trt8.0.1.6-ga-20210626/7fa2af80.pub sudo apt-get update sudo apt-get install tensorrt
If you use version 3.X of python, run
sudo apt-get install python3-libnvinfer-dev
Verify installation
dpkg -l | grep TensorRT
My results are as follows:
4. Running TensorRT with Python and c + +
python
In fact, I can't import tensorrt successfully after completing the above installation. I wonder if it is related to my use of anaconda environment. So I decided to save the country by downloading the TensorRT version of tar, and build the python environment by using the whl extracted from the tar file. The specific steps are as follows:
(1) Enter Official website Download the tar format file, which is the first one in the figure below.
(2) After decompression, enter the extracted TensorRT-8.0.1.6 folder, enter the python folder, and open the terminal for operation
pip install tensorrt-8.0.1.6-cp36-none-linux_x86_64.whl // Remember to correspond to your own python version number cd ../uff pip install uff-0.6.9-py2.py3-none-any.whl cd ../graphsurgeon pip install graphsurgeon-0.4.5-py2.py3-none-any.whl cd ../onnx_graphsurgeon pip install onnx_graphsurgeon-0.3.10-py2.py3-none-any.whl // Run python python // input import tensorrt tensorr.__version__ // If you output 8.0.1.6, the installation is successful
(3) The code for creating engine has changed slightly (or my original code is not standard enough, just for reference)
Original code
with trt.Builder(TRT_LOGGER) as builder, builder.create_network(EXPLICIT_BATCH) as network, trt.OnnxParser( network, TRT_LOGGER) as parser: builder.max_batch_size = 10 builder.max_workspace_size = 1 << 5 with open(engine_path, 'rb') as model: if not parser.parse(model.read()): raise TypeError("Parser parse failed.") engine = builder.build_cuda_engine(network)
Modern code
with trt.Builder(TRT_LOGGER) as builder, builder.create_network(EXPLICIT_BATCH) as network, \ builder.create_builder_config() as config, trt.OnnxParser(network, TRT_LOGGER) as parser, \ trt.Runtime(TRT_LOGGER) as runtime: config.max_workspace_size = 1 << 28 with open(engine_path, 'rb') as model: if not parser.parse(model.read()): raise TypeError("Parser parse failed.") plan = builder.build_serialized_network(network, config) engine = runtime.deserialize_cuda_engine(plan)
C++
The C++ide I used is codeblocks, and codeblocks of ubuntu18.04 is the chip, and make complaints about it.
(1) Set
Place the mouse over the root directory of your document tree, that is, your project, and right-click->build options...->The following interface appears, and enter the information shown in the figure Relevant information.!!! Pay attention to changing to your own path.
(2) The code for building engine has not changed much. It should be noted that if the serialization result file saved with TensorRT 7 cannot be directly deserialized, loaded and run with TensorRT 8, it is necessary to serialize with TensorRT 8 again and save the engine. Secondly, the construction code of gLogger has changed slightly, as follows:
class Logger:public ILogger { void log(Severity severity, const char* msg) noexcept // Just change the original override to noexcept, // I don't know why -- { // suppress info-level messages if(severity != Severity::kINFO) cout << msg << endl; } }gLogger;