C++ Implements Neural Network from Zero

Note: All source addresses are attached at the end of the article

Suggestion: Find the right time to read after collecting.

Since it is implemented in C++, it is natural to think of designing a neural network class to represent the neural network, which I call the Net class here.Since this class name is so common that it is likely to conflict with programs written by others, it is not difficult to think of my last name, Liu, because all my programs are included in the namespace liu.In the previous blog reverse propagation algorithm Resource collation, I listed a few more good resources.Students who are not familiar with theory and have a learning spirit can go out and turn left to see the resources for this article.It is assumed that the reader has a certain understanding of the basic theory of neural networks.

Before coding really begins, it is still necessary to give an account of the neural network basis, which is actually the idea of designing classes and writing programs.In short, a neural network consists of several elements:

• Neuronal Nodes

• layer

• weights

• bias item

The two main calculation processes of a neural network are forward propagation and reverse propagation.Each layer's forward propagation contains a weighted sum (convolution?)Linear operation and non-linear operation of activation function.Reverse propagation mainly uses BP algorithm to update weights.Although there are many details, the above is sufficient for this first article.

Calculations in neural networks can almost be expressed in the form of matrix calculations, which is one of the reasons why I use the Mat class of OpenCV, which provides a very complete and fully optimized array operation method. Another reason is that the library I am most familiar with is OpenCV...There are many good libraries and frameworks that represent different parts of a neural network with many classes.Blob classes represent data, Layer classes represent layers, and Optimizer classes represent optimization algorithms.However, it is not so complex here, but mainly limited in ability. Only one Net class is used to represent the neural network.

Or let the program talk directly. The Net class is included in Net.h, roughly as follows.

#ifndef NET_H #define NET_H #endif // NET_H #pragma once #include <iostream> #include<opencv2corecore.hpp> #include<opencv2highguihighgui.hpp> //#include<iomanip> #include"Function.h" namespace liu { class Net { public: std::vector<int> layer_neuron_num; std::vector<cv::Mat> layer; std::vector<cv::Mat> weights; std::vector<cv::Mat> bias; public: Net() {}; ~Net() {}; //Initialize net:genetate weights matrices,layer matrices and bias matrices // bias default all zero void initNet(std::vector<int> layer_neuron_num_); //Initialise the weights matrices. void initWeights(int type = 0, double a = 0., double b = 0.1); //Initialise the bias matrices. void initBias(cv::Scalar& bias); //Forward void forward(); //Forward void backward(); protected: //initialise the weight matrix.if type =0,Gaussian.else uniform. void initWeight(cv::Mat &dst, int type, double a, double b); //Activation function cv::Mat activationFunction(cv::Mat &x, std::string func_type); //Compute delta error void deltaError(); //Update weights void updateWeights(); }; }

The above is not the complete form of the Net class, but a simplified version corresponding to the content of this article, which will appear clearer after simplification.

Now the Net class has only four member variables:

• Number of neurons per layer (layer_neuron_num)

• layer

• Weight Matrix (weights)

• bias item

Needless to say, for ease of calculation, each layer and offset is also represented by Mat, and each layer and offset is represented by a single column matrix.

In addition to the default constructors and destructors, member functions of the Net class also include:

• initNet(): Used to initialize neural networks

• InitWeights(): Initialize the weight matrix and call the initWeights() function

• initBias(): Initialize offset

• forward(): Performs forward operations, including linear operations and nonlinear activation, while calculating errors

• backward(): Perform reverse propagation and call the updateWeights() function to update the weight.

These functions are already the core of the neural network program.The rest of the content is to slowly achieve, what to add when it is achieved, open the way on mountains, and build bridges on rivers.

Let's start with the initNet() function, which takes only one parameter, the number of neurons per layer, and then initializes the neural network.The meaning of initializing a neural network here is to generate a matrix for each layer, a matrix for each weight, and a matrix for each offset.It sounds simple, but it's also simple.

The implementation code is in Net.cpp.

There is no difficulty in generating all kinds of matrices here. The only thing to be concerned about is the determination of the number of rows and columns of the weight matrix.It is worth noting that the weight is set to 0 by default.

//Initialize net void Net::initNet(std::vector<int> layer_neuron_num_) { layer_neuron_num = layer_neuron_num_; //Generate every layer. layer.resize(layer_neuron_num.size()); for (int i = 0; i < layer.size(); i++) { layer[i].create(layer_neuron_num[i], 1, CV_32FC1); } std::cout << "Generate layers, successfully!" << std::endl; //Generate every weights matrix and bias weights.resize(layer.size() - 1); bias.resize(layer.size() - 1); for (int i = 0; i < (layer.size() - 1); ++i) { weights[i].create(layer[i + 1].rows, layer[i].rows, CV_32FC1); //bias[i].create(layer[i + 1].rows, 1, CV_32FC1); bias[i] = cv::Mat::zeros(layer[i + 1].rows, 1, CV_32FC1); } std::cout << "Generate weights matrices and bias, successfully!" << std::endl; std::cout << "Initialise Net, done!" << std::endl; }

The weight initialization function initWeights() calls the initWeight() function, which is essentially to initialize one or more differences.

Offset initialization assigns the same value to all offsets.The Scalar object is used here to assign values to the matrix.

//initialise the weights matrix.if type =0,Gaussian.else uniform. void Net::initWeight(cv::Mat &dst, int type, double a, double b) { if (type == 0) { randn(dst, a, b); } else { randu(dst, a, b); } } //initialise the weights matrix. void Net::initWeights(int type, double a, double b) { //Initialise weights cv::Matrices and bias for (int i = 0; i < weights.size(); ++i) { initWeight(weights[i], 0, 0., 0.1); } }

Offset initialization assigns the same value to all offsets.The Scalar object is used here to assign values to the matrix.

//Initialise the bias matrices. void Net::initBias(cv::Scalar& bias_) { for (int i = 0; i < bias.size(); i++) { bias[i] = bias_; } }

So far, all the parts of the neural network that need to be initialized have been initialized.

We can initialize a neural network with the following code. Although it has no function, at least we can test if the current code has a BUG:

#include"../include/Net.h" //<opencv2opencv.hpp> using namespace std; using namespace cv; using namespace liu; int main(int argc, char *argv[]) { //Set neuron number of every layer vector<int> layer_neuron_num = { 784,100,10 }; // Initialise Net and weights Net net; net.initNet(layer_neuron_num); net.initWeights(0, 0., 0.01); net.initBias(Scalar(0.05)); getchar(); return 0; }

There's no problem with hands-on testing.

All the code is hosted on Github and you can download it if you are interested.The source link address is

https://github.com/LiuXiaolong19920720/simple_net

• Give the thumbs-up 1
• Collection
• share
• • Article Report

3 February 2020, 20:40 | Views: 4869