The compilation and installation of Caffe GPU under linux

Server environment

Ubuntu18.04,

python3.6

CUDA 10.0,

cuDNN 7.6.3

opencv3.4.6

View cuda and cudnn versions

nvcc -V
cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2

Basically, they are installed in the directory of common users (caffe). CUDA10 and CUDNN in the server have been installed (in the normal location). python has also been installed in the root directory. opencv3.4.6 is installed directly in my own user directory (caffe). See my blog for details

No permission to install opencv3.4.6 under linux

1. Source download

Download Caffe directly from Github

https://github.com/BVLC/caffe

unzip caffe-master.zip
mv caffe-master caffe

Required dependent environment installation

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
sudo apt-get install --no-install-recommends libboost-all-dev
sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

2. Profile modification

In caffe directory

cd caffe

Modification of Makefile.config file

cp Makefile.config.example Makefile.config 
vim Makefile.config

Make the following changes to Makefile.config

Will:
#USE_CUDNN := 1 
Changed to:
USE_CUDNN := 1 
Will:
# USE_OPENCV := 0
 Changed to:
USE_OPENCV := 1
Will:
#OPENCV_VERSION := 3  
Changed to:
OPENCV_VERSION := 3
Will:
#WITH_PYTHON_LAYER := 1   
//Changed to:
WITH_PYTHON_LAYER := 1  

In the first two lines of CUDA arch: = there are two lines in front of * * 20 and * * 21, but I deleted them directly:

# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := #-gencode arch=compute_20,code=sm_20 \
		#-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

For the python path, you need to modify it (I use opencv3.6 under the root directory), and the numpy path under the Python interpreter also needs to be added.

take python2.7 Note out
#PYTHON_INCLUDE := /usr/include/python2.7 \
                /usr/lib/python2.7/dist-packages/numpy/core/include

//Add the path of Python 3.6 below
PYTHON_INCLUDE := /usr/include/python3.6 \
                /usr/local/lib/python3.6/dist-packages/numpy/core/include

Comment out the following,
#INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
#LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

//Then add
INCLUDE_DIRS := /home/caffe/opencv34/include/ $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := /home/caffe/opencv34/lib/ $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial/

Be careful

I put the installation path of OpenCV in include and lib at the front, so that the compiler can find this path first. I tried and failed after that. Because the compiler is stupid, multiple opencv versions are prone to conflict. Always report the following errors

.build_release/lib/libcaffe.so: undefined reference to `cv::imread(cv::String const&, int)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::allocate(unsigned long)'
.build_release/lib/libcaffe.so: undefined reference to `cv::String::deallocate()'
.build_release/lib/libcaffe.so: undefined reference to `cv::imencode(cv::String const&, cv::_InputArray const&, std::vector<unsigned char, std::allocator<unsigned char> >&, std::vector<int, std::allocator<int> > const&)'

 

Modify the Makefile file

vim Makefile

As shown in the figure below ()

 

Note the following
#NVCCFLAGS +=-ccbin=$(CXX) -Xcompiler-fPIC $(COMMON_FLAGS)
//Add to
NVCCFLAGS += -D_FORCE_INLINES -ccbin=$(CXX) -Xcompiler -fPIC $(COMMON_FLAGS)

 

​Note the following
#LIBRARIES += glog gflags protobuf boost_system boost_filesystem m
//Add to
LIBRARIES += glog gflags protobuf leveldb snappy \
             lmdb boost_system boost_filesystem hdf5_hl hdf5 m \
             opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs

3. Compile and install

In the caffe directory, make clean is mainly used to clean up the last compilation (the first compilation may not be executed).

 

make clean
make all -j16

Succeed

 

Testing

make test -j16

 

make runtest -j16

Error occurred

The main reason is that / usr/local/lib does not have this libcudart.so.10.0. Now you only need to copy libcudart.so.10.0 under the file / usr/local/cuda/lib64 to the folder / usr/local/lib. Just execute the following command.

sudo cp /usr/local/cuda/lib64/libcudart.so.10.0 /usr/local/lib/ && sudo ldconfig

If the / usr/local/lib file is missing libcudart.so.10.0, it is estimated that libcublas.so.10.0, libcurand.so.10.0 and libcudnn.so.7 are also missing.

sudo cp /usr/local/cuda/lib64/libcublas.so.10.0 /usr/local/lib/ && sudo ldconfig
sudo cp /usr/local/cuda/lib64/libcurand.so.10.0 /usr/local/lib/ && sudo ldconfig
sudo cp /usr/local/cuda/lib64/libcudnn.so.7 /usr/local/lib/ && sudo ldconfig

If all kinds of environments are ok, in caffe directory

make runtest -j16

 

Install pycaffe with the following configuration

make pycaffe -j16

 

Basically, caffe compilation and installation are basically completed

Attention

Output python in caffe directory, unable to use pycaffe

But it can be used in caffe user directory

4. MNIST model training and testing

train

In the caffe directory, download the mnist dataset

./data/mnist/get_mnist.sh

Convert data format

./examples/mnist/create_mnist.sh

Training

./examples/mnist/train_lenet.sh

 

test

In caffe directory

 

touch test_lenet.sh
vim test_lenet.sh

Add to file

 

./build/tools/caffe.bin test -model examples/mnist/lenet_train_test.prototxt -weights examples/mnist/lenet_iter_10000.caffemodel -iterations 100
chmod +x test_lenet.sh
./test_lenet.sh 

The successful test shows that the whole environment of caffe is ok.

 

101 original articles published, 117 praised, 320000 visitors+
Private letter follow

Tags: sudo Makefile Python vim

Posted on Tue, 10 Mar 2020 03:20:51 -0400 by verN