catalogue
2. Set optimizer, loss function
Achievement display
Training once:
30 training sessions:
Basic theory
Here, the output layer uses the softmax activation function to convert the output data into probability:
Neural network prototype:
28 * 28 = 784 input pixels correspond to 784 input neurons; Ten output neurons correspond to ten numbers of 0 ~ 9 respectively.
(in order to improve the accuracy of training, I added some hidden layers)
1, Data preparation
# Data preparation def Data_Preparation(): global train_data, train_label, test_data, test_label, mnist_train, \ mnist_test, images, labels
1. Load dataset
# 1. Load dataset mnist = tf.keras.datasets.mnist (train_data, train_label), (test_data, test_label) = mnist.load_data() # Training set data train_ The data shape of data is (60000, 28, 28) # Training set label train_ The data shape of label is (60000) # Test set data test_ The data shape of data is (10000, 28, 28) # Test set label test_ The data shape of label is (10000) images, labels = test_data, test_label
2. Data processing
2-1. Normalization
# 2-1. Data set normalization train_data = train_data/255 test_data = test_data/255
2-2. Single heat coding
0 ~ 9 represent their values with the position of 1 in 10 binary numbers respectively.
# 2-2. Label heat code train_label = tf.keras.utils.to_categorical(train_label, num_classes=10) test_label = tf.keras.utils.to_categorical(test_label, num_classes=10) # Length of heat code converted to heat code data to be transferred
2, Neural network fitting
1. Build neural network
Input layer (data flattening: (28,28) - > (784)), hidden layer and output layer (10 neurons correspond to 10 numbers).
# 1. Creating neural networks model = tf.keras.models.Sequential([ tf.keras.layers.Flatten(input_shape=(28, 28)), # Input data compression, 3D to 2D # (60000,28,28) -> (60000, 784) tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(10, activation='softmax') # Full connection layer # Output neuron number activation function (softmax) ])
The number of hidden layers and neurons are not fixed, and they are added after their own test.
Before adding hidden layers:
After adding hidden layer;
2. Set optimizer, loss function
# 2. Set optimizer, loss function model.compile(optimizer=SGD(0.3), loss='mse', metrics=['accuracy']) # Optimizer learning rate 0.3 loss function (mean square error) retention label (accuracy)
The best learning rate tested at present: 0.3 (this model)
3. Training
# 3. Training model.fit(train_data, train_label, epochs=20, batch_size=32, validation_data=(test_data, test_label)) # The training set traverses 20 times, a group of 32 test sets
Training for 30 times:
D:\Software\Python\Python\python.exe D:/Study/AI/OpenCV/draft.py/main.py
Epoch 1/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0393 - accuracy: 0.7064 - val_loss: 0.0114 - val_accuracy: 0.9270
Epoch 2/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0105 - accuracy: 0.9310 - val_loss: 0.0075 - val_accuracy: 0.9498
Epoch 3/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0072 - accuracy: 0.9538 - val_loss: 0.0062 - val_accuracy: 0.9603
Epoch 4/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0057 - accuracy: 0.9635 - val_loss: 0.0055 - val_accuracy: 0.9658
Epoch 5/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0047 - accuracy: 0.9703 - val_loss: 0.0048 - val_accuracy: 0.9691
Epoch 6/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0040 - accuracy: 0.9746 - val_loss: 0.0048 - val_accuracy: 0.9694
Epoch 7/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0034 - accuracy: 0.9787 - val_loss: 0.0049 - val_accuracy: 0.9669
Epoch 8/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0030 - accuracy: 0.9816 - val_loss: 0.0043 - val_accuracy: 0.9713
Epoch 9/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0026 - accuracy: 0.9844 - val_loss: 0.0038 - val_accuracy: 0.9760
Epoch 10/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0023 - accuracy: 0.9864 - val_loss: 0.0050 - val_accuracy: 0.9677
Epoch 11/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0021 - accuracy: 0.9873 - val_loss: 0.0037 - val_accuracy: 0.9764
Epoch 12/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0018 - accuracy: 0.9894 - val_loss: 0.0037 - val_accuracy: 0.9758
Epoch 13/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0017 - accuracy: 0.9901 - val_loss: 0.0041 - val_accuracy: 0.9734
Epoch 14/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0015 - accuracy: 0.9911 - val_loss: 0.0045 - val_accuracy: 0.9708
Epoch 15/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0014 - accuracy: 0.9921 - val_loss: 0.0038 - val_accuracy: 0.9760
Epoch 16/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0013 - accuracy: 0.9922 - val_loss: 0.0036 - val_accuracy: 0.9764
Epoch 17/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0012 - accuracy: 0.9932 - val_loss: 0.0035 - val_accuracy: 0.9771
Epoch 18/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0011 - accuracy: 0.9942 - val_loss: 0.0034 - val_accuracy: 0.9783
Epoch 19/30
1875/1875 [==============================] - 7s 4ms/step - loss: 9.5588e-04 - accuracy: 0.9947 - val_loss: 0.0034 - val_accuracy: 0.9788
Epoch 20/30
1875/1875 [==============================] - 6s 3ms/step - loss: 9.9405e-04 - accuracy: 0.9942 - val_loss: 0.0037 - val_accuracy: 0.9767
Epoch 21/30
1875/1875 [==============================] - 7s 4ms/step - loss: 8.7466e-04 - accuracy: 0.9952 - val_loss: 0.0037 - val_accuracy: 0.9757
Epoch 22/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.5603e-04 - accuracy: 0.9959 - val_loss: 0.0037 - val_accuracy: 0.9773
Epoch 23/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.8775e-04 - accuracy: 0.9955 - val_loss: 0.0035 - val_accuracy: 0.9783
Epoch 24/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.2748e-04 - accuracy: 0.9961 - val_loss: 0.0034 - val_accuracy: 0.9779
Epoch 25/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.2780e-04 - accuracy: 0.9959 - val_loss: 0.0036 - val_accuracy: 0.9777
Epoch 26/30
1875/1875 [==============================] - 6s 3ms/step - loss: 5.9373e-04 - accuracy: 0.9969 - val_loss: 0.0032 - val_accuracy: 0.9792
Epoch 27/30
1875/1875 [==============================] - 6s 3ms/step - loss: 5.6153e-04 - accuracy: 0.9970 - val_loss: 0.0034 - val_accuracy: 0.9786
Epoch 28/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.7011e-04 - accuracy: 0.9969 - val_loss: 0.0033 - val_accuracy: 0.9792
Epoch 29/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.0371e-04 - accuracy: 0.9973 - val_loss: 0.0037 - val_accuracy: 0.9767
Epoch 30/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.2224e-04 - accuracy: 0.9972 - val_loss: 0.0033 - val_accuracy: 0.9795
It can be found that if the training times are too high, there will be little change in the future.
3, Forecast
1. Backup image dataset
Here, two image data sets are prepared, one for later prediction classification and one for displaying images.
Why not use one? Because the two dimensions are different, the predicted data needs a higher dimension.
# Image add dimension Images = images[:, np.newaxis] # Images are displayed normally, and images are used for prediction
2. Forecast classification
# Forecast classification classification = model.predict(Images[i], batch_size=10) # Get results result.append(np.argmax(classification[0]))
3. Display results (plt)
# Display results x = int(i/3) y = i%3 ax[x][y].set_title(f'label:{labels[i]}-predict:{result[i]}') # Set title ax[x][y].imshow(images[i], 'gray') # Display image ax[x][y].axis('off') # Hide axes
Total code
# keras handwritten numeral recognition import os os.environ['TF_CPP_MIN_LOG_LEVEL']='2' import tensorflow as tf from tensorflow.keras.optimizers import SGD import numpy as np import matplotlib.pyplot as plt # Data preparation def Data_Preparation(): global train_data, train_label, test_data, test_label, mnist_train, \ mnist_test, images, labels # 1. Load dataset mnist = tf.keras.datasets.mnist (train_data, train_label), (test_data, test_label) = mnist.load_data() # Training set data train_ The data shape of data is (60000, 28, 28) # Training set label train_ The data shape of label is (60000) # Test set data test_ The data shape of data is (10000, 28, 28) # Test set label test_ The data shape of label is (10000) images, labels = test_data, test_label # 2. Data processing (normalization, single heat coding) # 2-1. Data set normalization train_data = train_data/255 test_data = test_data/255 # 2-2. Label heat code train_label = tf.keras.utils.to_categorical(train_label, num_classes=10) test_label = tf.keras.utils.to_categorical(test_label, num_classes=10) # Length of heat code converted to heat code data to be transferred # Neural network fitting def Neural_Network(): global model # 1. Creating neural networks model = tf.keras.models.Sequential([ # tf.keras.layers.Flatten(input_shape=(28, 28)), # Input data compression, 3D to 2D # (60000,28,28) -> (60000, 784) tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu), tf.keras.layers.Dense(10, activation='softmax') # Full connection layer # Output neuron number activation function (softmax) ]) # 2. Set optimizer, loss function model.compile(optimizer=SGD(0.3), loss='mse', metrics=['accuracy']) # Optimizer learning rate 0.3 loss function (mean square error) retention label (accuracy) # 3. Training model.fit(train_data, train_label, epochs=1, batch_size=32, validation_data=(test_data, test_label)) # The training set traverses 20 times, a group of 32 test sets # Prediction of handwritten digits (prediction of visual part test set) def Predict(): global images # Prediction results result = [] # plt chart f, ax = plt.subplots(3, 3, figsize=(8, 6)) # Image add dimension Images = images[:, np.newaxis] # Images are displayed normally, and images are used for prediction # Select part of the data in the test set for visual classification for i in range(9): # Forecast classification classification = model.predict(Images[i], batch_size=10) # Get results result.append(np.argmax(classification[0])) # Display results x = int(i/3) y = i%3 ax[x][y].set_title(f'label:{labels[i]}-predict:{result[i]}') # Set title ax[x][y].imshow(images[i], 'gray') # Display image ax[x][y].axis('off') # Hide axes plt.show() # Data preparation Data_Preparation() # Neural network construction Neural_Network() # Handwritten numeral prediction Predict()