TensorFlow Hub shares many pre trained model components.

The main content of this case:

1. adopt tf.keras Using TensorFlow Hub
2. Using TensorFlow Hub to complete image classification task
3. Complete simple transfer learning

1. Import the required Library

import tensorflow as tf
import tensorflow_hub as hub
import matplotlib.pylab as plt
import numpy as np
from PIL import Image

for i in [tf,hub,np]:
    print(i.__name__,": ",i.__version__,sep="")

Output:

tensorflow: 2.2.0
tensorflow_hub: 0.8.0
numpy: 1.17.4

2. ImageNet classifier

2.1 download ImageNet classifier from TensorFlow Hub

classifier_url = "https://hub.tensorflow.google.cn/google/tf2-preview/mobilenet_v2/classification/2"

image_shape = (224,224)
classifier = tf.keras.Sequential([
    hub.KerasLayer(classifier_url,input_shape=image_shape+(3,))  # Tuple addition is used: (224224) + (3,) = (224224,3)
])

No output. No news, the best news!

2.2 download forecast data

grace_hopper = tf.keras.utils.get_file("image.jpg","https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg",cache_dir="./")
grace_hopper = Image.open(grace_hopper).resize(image_shape)
grace_hopper

Output:

Downloading data from https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg
65536/61306 [================================] - 1s 23us/step

# Normalize the value to a number between 0 and 1
grace_hopper = np.array(grace_hopper)/255.0
grace_hopper.shape

Output:

(224, 224, 3)

result = classifier.predict(grace_hopper[np.newaxis,...])
result.shape

Output:

(1, 1001)

predicted_class = np.argmax(result[0],axis=-1)
predicted_class

Output:

653

2.3 decoding prediction results

Decodes the predicted result based on the label of the ImageNet dataset.

# Download ImageNet dataset Tags
labels_path = tf.keras.utils.get_file("ImageNetLabels.txt","https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt",cache_dir="./")
imagenet_labels = np.array(open(labels_path).read().splitlines())

Output:

Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt
16384/10484 [==============================================] - 0s 27us/step

# Label name of the output forecast
plt.imshow(grace_hopper)
plt.axis("off")
predicted_class_name = imagenet_labels[predicted_class]
_ = plt.title("Prediction: "+predicted_class_name.title())

Output:

The prediction result is: military uniform, and the prediction is correct.

3. Predict the results directly with the pre training model

3.1 download data set

data_root = tf.keras.utils.get_file("flower_photos","https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz",untar=True)
data_root

Output:

'C:\\Users\\my-pc\\.keras\\datasets\\flower_photos'

use tf.keras.preprocessing.image.ImageDataGenerator Load the data and use the reset parameter to convert the data to a floating-point number between 0 and 1.

image_generator = tf.keras.preprocessing.image.ImageDataGenerator(rescale=1./255)
image_data = image_generator.flow_from_directory(data_root,target_size=image_shape)

Output:

Found 3670 images belonging to 5 classes.

flow_ from_ The default batch size of the directory function is 32. The detailed default parameters are as follows:

flow_from_directory(
    directory, target_size=(256, 256), color_mode='rgb', classes=None,
    class_mode='categorical', batch_size=32, shuffle=True, seed=None,
    save_to_dir=None, save_prefix='', save_format='png', follow_links=False,
    subset=None, interpolation='nearest' )

for image_batch, label_batch in image_data:
    print("Image batch shape: ",image_batch.shape)
    print("Label batch shape: ",label_batch.shape)
    break

Output:

Image batch shape:  (32, 224, 224, 3)
Label batch shape:  (32, 5)

3.2 batch prediction image data

result_batch = classifier.predict(image_batch)
result_batch.shape

Output:

(32, 1001)

3.3 visual prediction results

predicted_class_names = imagenet_labels[np.argmax(result_batch,axis=-1)]
predicted_class_names

Output:

array(['bee', 'sea urchin', 'daisy', 'daisy', 'daisy', 'daisy',
       'sea urchin', 'pot', 'daisy', 'pot', 'sea urchin', 'orange', 'ear',
       'rapeseed', 'daisy', 'orange', 'daisy', 'daisy', 'rapeseed',
       'cardoon', 'fly', 'picket fence', 'ant', 'sulphur butterfly',
       'white stork', 'rapeseed', 'earthstar', 'buckeye', 'daisy',
       'totem pole', 'quill', 'daisy'], dtype='<U30')

plt.figure(figsize=(10,10))
plt.subplots_adjust(hspace=0.5)
for n in range(30):
    plt.subplot(6,5,n+1)
    plt.imshow(image_batch[n])
    plt.title(predicted_class_names[n])
    plt.axis("off")
    plt.suptitle("ImageNet Predictions")

Output:

As the output above shows, almost all of the predictions are wrong. At this point, you may wonder if the code is wrong or something else is wrong.

In fact, the ImageNet training data set we downloaded does not have samples of these flowers, or similar samples, which makes the model unable to correctly identify these flowers.

4. Transfer learning

The above results show that the error rate is very high when using the ImageNet training model to predict directly, so migration learning is needed to play a role.

4.1 download headless model

headless model: there is no final classification layer, only the convolution layer of the previous extracted features.

feature_extractor_url = "https://hub.tensorflow.google.cn/google/tf2-preview/mobilenet_v2/feature_vector/2"
feature_extractor_layer = hub.KerasLayer(feature_extractor_url,input_shape=(224,224,3))

feature_batch = feature_extractor_layer(image_batch)
feature_batch.shape

Output:

TensorShape([32, 1280])

As shown in the above output, the predicted output of a single image download model is 1280 vectors, that is to say, there should be 1280 neurons in this layer when adding neural network.

4.2 building a new model

When the full connection layer training is added later, the parameters in the downloaded headless model need to be set as untrainable.

feature_extractor_layer.trainable = False

model = tf.keras.Sequential([
    feature_extractor_layer,
    tf.keras.layers.Dense(image_data.num_classes)
])

model.compile(optimizer="adam",
             loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True),
             metrics=["accuracy"])

model.summary()

Output:

Model: "sequential_2"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
keras_layer_1 (KerasLayer)   (None, 1280)              2257984   
_________________________________________________________________
dense_1 (Dense)              (None, 5)                 6405      
=================================================================
Total params: 2,264,389
Trainable params: 6,405
Non-trainable params: 2,257,984
_________________________________________________________________

4.3 training model

class CollectBatchStats(tf.keras.callbacks.Callback):
    def __init__(self):
        self.batch_losses = []
        self.batch_acc = []
    def on_train_batch_end(self,batch,logs=None):
        self.batch_losses.append(logs["loss"])
        self.batch_acc.append(logs["accuracy"])
        self.model.reset_metrics()

steps_per_epoch = np.ceil(image_data.samples/image_data.batch_size)
batch_stats_callback = CollectBatchStats()

history = model.fit_generator(image_data,epochs=2,
                             steps_per_epoch=steps_per_epoch,
                             callbacks=[batch_stats_callback])

Output:

Epoch 1/2
115/115 [==============================] - 12s 104ms/step - loss: 0.3387 - accuracy: 0.8750
Epoch 2/2
115/115 [==============================] - 8s 69ms/step - loss: 0.2902 - accuracy: 0.9062

plt.figure()
plt.ylabel("Loss")
plt.xlabel("Training Steps")
plt.ylim([0,2])
plt.plot(batch_stats_callback.batch_losses)

Output:

plt.figure()
plt.ylabel("Accuracy")
plt.xlabel("Training Steps")
plt.ylim([0,1])
plt.plot(batch_stats_callback.batch_acc)

Output:

From the above results, it can be found that after only two rounds of training, the accuracy rate has achieved good results.

5. Forecast with new model

class_names = sorted(image_data.class_indices.items(),key=lambda pair:pair[1])
class_names = np.array([key.title() for key, value in class_names])
class_names

Output:

array(['Daisy', 'Dandelion', 'Roses', 'Sunflowers', 'Tulips'],
      dtype='<U10')

predicted_batch = model.predict(image_batch)
predicted_id = np.argmax(predicted_batch,axis=-1)
predicted_label_batch = class_names[predicted_id]

# visualization
label_id = np.argmax(label_batch, axis=-1)

plt.figure(figsize=(10,10))
plt.subplots_adjust(hspace=0.5)
for n in range(30):
    plt.subplot(6,5,n+1)
    plt.imshow(image_batch[n])
    color = "green" if predicted_id[n] == label_id[n] else "red"
    plt.title(predicted_label_batch[n].title(),color=color)
    plt.axis("off")
    plt.suptitle("Model Predictions(green: correct, red:incorrect)")

Output:

Using the new model to predict, only one of the 30 results is wrong. This is a good result.

6. Export model

6.1 save model to local

import time

t = time.time()

export_path = "./TransferLearning_SavedModel/{}".format(int(t))
model.save(export_path, save_format="tf")

Output:

INFO:tensorflow:Assets written to: ./TransferLearning_SavedModel/1592880647\assets
INFO:tensorflow:Assets written to: ./TransferLearning_SavedModel/1592880647\assets

The model has been saved to transferlearning_ Saved model directory

6.2 loading saved local models

reloaded_model = tf.keras.models.load_model(export_path)   # Load saved local model

result_batch = model.predict(image_batch)  # Model prediction results before saving
reloaded_result_batch = reloaded_model.predict(image_batch) # Reload model predictions

abs(reloaded_result_batch - result_batch).max()  # Compare the prediction results of the two

Output:

0.0

As like as two peas, the model is exactly the same as the model before it was saved.