3. Neural network and deep learning

3.1 origin of fashion MNIST dataset

1. Computer programs generally consist of two main parts: code and data
2. For deep learning, software is the network itself, especially the weight generated by training in the training process
3. The job of neural network programmers is to supervise and guide the learning process through training (which can be seen as an indirect way to write software or code)

3.1.1 fashion MNIST dataset

MNIST is a very famous handwritten numeral data set (M:Modify; NIST: National Institute of Standard and Technology)
MNIST has 70000 images in total: 60000 for training; 10000 for testing; Ten categories 0-9 in total
The fashion MNIST data set is from Zalando website: 10 categories correspond to 10 kinds of clothing; 70000 28x28 grayscale images
The purpose of fashion MNIST is to replace MNIST data set as a benchmark to test machine learning algorithm
Similarities and differences between fashion MNIST and MNIST dataset: (1) differences: the images in MNIST dataset are handwritten images, while the images in fashion MNIST are real images; (2) Same: the two data sets have the same data scale, image size, data format, and segmentation method of training set and test set
Why MNIST is so popular: 1. The size of the data set allows deep learning researchers to quickly check and reproduce their algorithms; 2. The data set can be used in all deep learning frameworks
The torch vision package in Pytorch can load the fashion MNIST dataset

3.2 importing and loading datasets using torchvision

3.2.1 process of creating deep learning project:

Prepare dataset
Create network model
Training network model
Analysis results

3.2.2 data preparation follows ETL process:

Extract, transform, load
The package included in pytorch can simplify the ETL process

3.2.3 data preparation:

1. Extraction: obtain fashion MNIST image data from source data
2. Conversion: convert data into tensor form
3. Load: encapsulate data into objects to make it easier to access
The biggest difference between fashion MNIST and MNIST datasets in calling is the difference in URL
torch.utils.data.Dataset: an abstract class used to represent a dataset
torch.utils.data.DataLoader: wraps the dataset and provides access to the underlying

import torch
import torchvision
import torchvision.transforms as transforms # It can help transform data

train_set = torchvision.datasets.FashionMNIST(
    root = './data/FashionMNIST',   # The local storage location of the dataset
    train = True,                   # Data sets for training
    download = True,                # If there is no local data, it will be downloaded automatically
    transform = transforms.Compose([
        transforms.ToTensor()         
    ])                              # Convert image to tensor
)

train_loader = torch.utils.data.DataLoader(train_set)
# The training set is packaged or loaded into the data loader, and the basic data can be accessed in the desired format;
# The data loader enables us to access data and provide query functions

report errors:

D:\Anaconda3_install\envs\pytorch_1.9\lib\site-packages\torchvision\datasets\mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ..\torch\csrc\utils\tensor_numpy.cpp:180.)
  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)

Solution: modify the mnist.py file
The given numpy array is not writeable, and Pytorch does not support non writeable tensor
This link resolves my error report

My solution:
Click error report to directly jump to minst.py, and directly jump to the position of copy=False, and then delete copy=False.

Bach_size adjustment
Explained Bach_ The relevance of size.
Batch in neural network_ Meaning and setting method of size parameter
Explained some setting skills, advantages and disadvantages