catalogue

Achievement display

Basic theory

  1, Data preparation

1. Load dataset

2. Data processing

2-1. Normalization

2-2. Single heat coding

2, Neural network fitting

1. Build neural network

2. Set optimizer, loss function

3. Training

3, Forecast

1. Backup image dataset

2. Forecast classification

3. Display results (plt)

Total code

Achievement display

Training once:

  30 training sessions:

Basic theory

 

Here, the output layer uses the softmax activation function to convert the output data into probability:

Neural network prototype:

28 * 28 = 784 input pixels correspond to 784 input neurons; Ten output neurons correspond to ten numbers of 0 ~ 9 respectively.

(in order to improve the accuracy of training, I added some hidden layers)

 

 

  1, Data preparation

# Data preparation
def Data_Preparation():
    global train_data, train_label, test_data, test_label, mnist_train, \
        mnist_test, images, labels

1. Load dataset

# 1. Load dataset
    mnist = tf.keras.datasets.mnist
    (train_data, train_label), (test_data, test_label) = mnist.load_data()
    # Training set data train_ The data shape of data is (60000, 28, 28)
    # Training set label train_ The data shape of label is (60000)
    # Test set data test_ The data shape of data is (10000, 28, 28)
    # Test set label test_ The data shape of label is (10000)
    images, labels = test_data, test_label

2. Data processing

2-1. Normalization

# 2-1. Data set normalization
    train_data = train_data/255
    test_data = test_data/255

2-2. Single heat coding

0 ~ 9 represent their values with the position of 1 in 10 binary numbers respectively.

# 2-2. Label heat code
    train_label = tf.keras.utils.to_categorical(train_label, num_classes=10)
    test_label = tf.keras.utils.to_categorical(test_label, num_classes=10)
    #                           Length of heat code converted to heat code data to be transferred

2, Neural network fitting

1. Build neural network

Input layer (data flattening: (28,28) - > (784)), hidden layer and output layer (10 neurons correspond to 10 numbers).

# 1. Creating neural networks
    model = tf.keras.models.Sequential([
        tf.keras.layers.Flatten(input_shape=(28, 28)),   # Input data compression, 3D to 2D
        # (60000,28,28) -> (60000, 784)
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(10, activation='softmax')  # Full connection layer
    #                       Output neuron number activation function (softmax)
    ])

The number of hidden layers and neurons are not fixed, and they are added after their own test.

Before adding hidden layers:  

  After adding hidden layer;

 

2. Set optimizer, loss function

# 2. Set optimizer, loss function
    model.compile(optimizer=SGD(0.3), loss='mse', metrics=['accuracy'])
    #             Optimizer learning rate 0.3 loss function (mean square error) retention label (accuracy)

The best learning rate tested at present: 0.3 (this model)

3. Training

# 3. Training
    model.fit(train_data, train_label, epochs=20, batch_size=32, validation_data=(test_data, test_label))
#             The training set traverses 20 times, a group of 32 test sets

Training for 30 times:  

 D:\Software\Python\Python\python.exe D:/Study/AI/OpenCV/draft.py/main.py
Epoch 1/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0393 - accuracy: 0.7064 - val_loss: 0.0114 - val_accuracy: 0.9270
Epoch 2/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0105 - accuracy: 0.9310 - val_loss: 0.0075 - val_accuracy: 0.9498
Epoch 3/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0072 - accuracy: 0.9538 - val_loss: 0.0062 - val_accuracy: 0.9603
Epoch 4/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0057 - accuracy: 0.9635 - val_loss: 0.0055 - val_accuracy: 0.9658
Epoch 5/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0047 - accuracy: 0.9703 - val_loss: 0.0048 - val_accuracy: 0.9691
Epoch 6/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0040 - accuracy: 0.9746 - val_loss: 0.0048 - val_accuracy: 0.9694
Epoch 7/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0034 - accuracy: 0.9787 - val_loss: 0.0049 - val_accuracy: 0.9669
Epoch 8/30
1875/1875 [==============================] - 5s 3ms/step - loss: 0.0030 - accuracy: 0.9816 - val_loss: 0.0043 - val_accuracy: 0.9713
Epoch 9/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0026 - accuracy: 0.9844 - val_loss: 0.0038 - val_accuracy: 0.9760
Epoch 10/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0023 - accuracy: 0.9864 - val_loss: 0.0050 - val_accuracy: 0.9677
Epoch 11/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0021 - accuracy: 0.9873 - val_loss: 0.0037 - val_accuracy: 0.9764
Epoch 12/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0018 - accuracy: 0.9894 - val_loss: 0.0037 - val_accuracy: 0.9758
Epoch 13/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0017 - accuracy: 0.9901 - val_loss: 0.0041 - val_accuracy: 0.9734
Epoch 14/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0015 - accuracy: 0.9911 - val_loss: 0.0045 - val_accuracy: 0.9708
Epoch 15/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0014 - accuracy: 0.9921 - val_loss: 0.0038 - val_accuracy: 0.9760
Epoch 16/30
1875/1875 [==============================] - 7s 4ms/step - loss: 0.0013 - accuracy: 0.9922 - val_loss: 0.0036 - val_accuracy: 0.9764
Epoch 17/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0012 - accuracy: 0.9932 - val_loss: 0.0035 - val_accuracy: 0.9771
Epoch 18/30
1875/1875 [==============================] - 6s 3ms/step - loss: 0.0011 - accuracy: 0.9942 - val_loss: 0.0034 - val_accuracy: 0.9783
Epoch 19/30
1875/1875 [==============================] - 7s 4ms/step - loss: 9.5588e-04 - accuracy: 0.9947 - val_loss: 0.0034 - val_accuracy: 0.9788
Epoch 20/30
1875/1875 [==============================] - 6s 3ms/step - loss: 9.9405e-04 - accuracy: 0.9942 - val_loss: 0.0037 - val_accuracy: 0.9767
Epoch 21/30
1875/1875 [==============================] - 7s 4ms/step - loss: 8.7466e-04 - accuracy: 0.9952 - val_loss: 0.0037 - val_accuracy: 0.9757
Epoch 22/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.5603e-04 - accuracy: 0.9959 - val_loss: 0.0037 - val_accuracy: 0.9773
Epoch 23/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.8775e-04 - accuracy: 0.9955 - val_loss: 0.0035 - val_accuracy: 0.9783
Epoch 24/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.2748e-04 - accuracy: 0.9961 - val_loss: 0.0034 - val_accuracy: 0.9779
Epoch 25/30
1875/1875 [==============================] - 7s 4ms/step - loss: 7.2780e-04 - accuracy: 0.9959 - val_loss: 0.0036 - val_accuracy: 0.9777
Epoch 26/30
1875/1875 [==============================] - 6s 3ms/step - loss: 5.9373e-04 - accuracy: 0.9969 - val_loss: 0.0032 - val_accuracy: 0.9792
Epoch 27/30
1875/1875 [==============================] - 6s 3ms/step - loss: 5.6153e-04 - accuracy: 0.9970 - val_loss: 0.0034 - val_accuracy: 0.9786
Epoch 28/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.7011e-04 - accuracy: 0.9969 - val_loss: 0.0033 - val_accuracy: 0.9792
Epoch 29/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.0371e-04 - accuracy: 0.9973 - val_loss: 0.0037 - val_accuracy: 0.9767
Epoch 30/30
1875/1875 [==============================] - 7s 4ms/step - loss: 5.2224e-04 - accuracy: 0.9972 - val_loss: 0.0033 - val_accuracy: 0.9795

It can be found that if the training times are too high, there will be little change in the future.  

3, Forecast

1. Backup image dataset

Here, two image data sets are prepared, one for later prediction classification and one for displaying images.

Why not use one? Because the two dimensions are different, the predicted data needs a higher dimension.  

# Image add dimension
    Images = images[:, np.newaxis]
    # Images are displayed normally, and images are used for prediction

2. Forecast classification

# Forecast classification
        classification = model.predict(Images[i], batch_size=10)
        # Get results
        result.append(np.argmax(classification[0]))

3. Display results (plt)

# Display results
        x = int(i/3)
        y = i%3

        ax[x][y].set_title(f'label:{labels[i]}-predict:{result[i]}')    # Set title
        ax[x][y].imshow(images[i], 'gray')                              # Display image
        ax[x][y].axis('off')                                            # Hide axes

Total code

# keras handwritten numeral recognition
import os
os.environ['TF_CPP_MIN_LOG_LEVEL']='2'

import tensorflow as tf
from tensorflow.keras.optimizers import SGD
import numpy as np
import matplotlib.pyplot as plt

# Data preparation
def Data_Preparation():
    global train_data, train_label, test_data, test_label, mnist_train, \
        mnist_test, images, labels
    # 1. Load dataset
    mnist = tf.keras.datasets.mnist
    (train_data, train_label), (test_data, test_label) = mnist.load_data()
    # Training set data train_ The data shape of data is (60000, 28, 28)
    # Training set label train_ The data shape of label is (60000)
    # Test set data test_ The data shape of data is (10000, 28, 28)
    # Test set label test_ The data shape of label is (10000)
    images, labels = test_data, test_label

    # 2. Data processing (normalization, single heat coding)
    # 2-1. Data set normalization
    train_data = train_data/255
    test_data = test_data/255

    # 2-2. Label heat code
    train_label = tf.keras.utils.to_categorical(train_label, num_classes=10)
    test_label = tf.keras.utils.to_categorical(test_label, num_classes=10)
    #                           Length of heat code converted to heat code data to be transferred


# Neural network fitting
def Neural_Network():
    global model
    # 1. Creating neural networks
    model = tf.keras.models.Sequential([
        #
        tf.keras.layers.Flatten(input_shape=(28, 28)),   # Input data compression, 3D to 2D
        # (60000,28,28) -> (60000, 784)
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(200 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(100 + 1, activation=tf.nn.relu),
        tf.keras.layers.Dense(10, activation='softmax')  # Full connection layer
    #                       Output neuron number activation function (softmax)
    ])

    # 2. Set optimizer, loss function
    model.compile(optimizer=SGD(0.3), loss='mse', metrics=['accuracy'])
    #             Optimizer learning rate 0.3 loss function (mean square error) retention label (accuracy)

    # 3. Training
    model.fit(train_data, train_label, epochs=1, batch_size=32, validation_data=(test_data, test_label))
#             The training set traverses 20 times, a group of 32 test sets


# Prediction of handwritten digits (prediction of visual part test set)
def Predict():
    global images
    # Prediction results
    result = []
    # plt chart
    f, ax = plt.subplots(3, 3, figsize=(8, 6))

    # Image add dimension
    Images = images[:, np.newaxis]
    # Images are displayed normally, and images are used for prediction

    # Select part of the data in the test set for visual classification
    for i in range(9):
        # Forecast classification
        classification = model.predict(Images[i], batch_size=10)
        # Get results
        result.append(np.argmax(classification[0]))

        # Display results
        x = int(i/3)
        y = i%3

        ax[x][y].set_title(f'label:{labels[i]}-predict:{result[i]}')    # Set title
        ax[x][y].imshow(images[i], 'gray')                              # Display image
        ax[x][y].axis('off')                                            # Hide axes

    plt.show()


# Data preparation
Data_Preparation()
# Neural network construction
Neural_Network()
# Handwritten numeral prediction
Predict()