Paddle regular season: positioning of macular fovea in film fundus color photos - the first scheme in September

Paddle regular season: positioning of macular fovea in film fundus color photos - the first scheme in September

(1) Competition introduction

Introduction to competition questions

Top personal homepage, stamp here see

The focus of the regular season of page macular localization is to research and develop algorithms related to the localization of macular structure in patients' fundus photos. The objective of the regular season is to evaluate and compare automatic algorithms for locating macula on a common retinal fundus image data set. The specific purpose is to predict the coordinate value of macular fovea in the image.

Fovea is the area with the sharpest color and resolution in the retina. For example, at about 3.5mm of the temporal side of the optic disc, there is a yellow cell called macula, and the central depression is the central depression. Accurate location of the fovea can assist doctors in the diagnosis of diabetic retinopathy and macular degeneration.

Schedule competition system

(1) The regular sculling competition is open to the whole society until the competition questions are offline;

(2) There are no preliminary and semi-final competitions in the regular sculling competition, and the ranking is based on the best results submitted by each contestant in the current month. The monthly competition cycle is from the first day of this month to the last day of this month;

(3) During the competition, players can submit their works up to 5 times a day (prediction results + original code), and the system automatically selects the highest score as the list record;

(4) The general list of the previous month is published on the 1st-5th of each month. The top 10 in the month and the re examined by code can get the honorary certificate issued by Baidu PaddlePaddle. For the first time contestants on the list, they can also get an additional special gift package (1 oar peripheral prize + 100 hour GPU power card). The staff will notify the top 10 contestants of last month by email to submit materials for code review. Please pay attention to the email notice. Special reminder: contestants who have obtained a special gift package can only obtain another special gift package if they write a new studio project based on this competition topic and are rated as selected;

(5) The first player with a score of more than 0.04 can win additional awards: Xiaodu is at home;

(6) Players are encouraged to sign up for regular sculling competitions with multiple themes to promote learning and improve the in-depth learning ability of developers in all aspects.

(2) Data introduction

The Sun Yat Sen eye center of Sun Yat sen University provides 800 fundus color photos with macular foveal coordinates for players' training model and 400 labeled data for the platform to test the model.

Data description

The gold standard provided in this regular season was manually marked by 7 ophthalmologists from Zhongshan eye center of Sun Yat sen University, and then fused into the final marking result by another senior expert. The fovea coordinate information corresponding to the data set provided in this competition is stored in the xlsx file named "Fovea_Location_train". The first column corresponds to the file name of the fundus image (including the extension ". jpg"), the second column contains the x coordinate and the third column contains the y coordinate.
chart

Training data set

File name: Train
There is a folder fundus in the Train folder_ Images and an xlsx file.

fundus_ The images folder contains 800 fundus color photos with a resolution of 1444 × 1444, or 2124 × 2056. Name shapes such as H0001.jpg, P0001.jpg, N0001.jpg and V0001.jpg.
The xlsx file contains the x and y coordinate information corresponding to 800 fundus color photos.

Test data set

File name: the page testing 400 images folder contains 400 fundus color photos, such as T0001.jpg.

(3) Personal ideas + highlights of personal solutions

Custom dataset reading pictures and labels

class dataset(paddle.io.Dataset):
    def __init__(self,img_list,label_listx,label_listy,transform=None,transform2=None,mode='train'):

        self.image=img_list
        self.labelx=label_listx
        self.labely=label_listy
        self.mode=mode
        self.transform=transform
        self.transform2=transform2
    def load_img(self, image_path):

        img=cv2.imread(image_path,1)
        img=cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
        return img

    def __getitem__(self,index):
        img = self.load_img(self.image[index])
        labelx = self.labelx[index]
        labely = self.labely[index]
        img_size=img.shape
        label=np.array([labelx,labely])
        img=np.array(img,dtype='float32')   
        label=np.array(label,dtype='float32')    
        
        if self.transform:
            if self.mode=='train':
                img, label = self.transform([img,label])
            else:
                img, label = self.transform2([img, label])


        img=np.array(img,dtype='float32')
        label=np.array(label,dtype='float32')
        return img,label

    def __len__(self):
        return len(self.image)

Augment data with data enhancement methods

class Resize(object):
    # Adjusts the input image to the specified size

    def __init__(self, output_size):
        assert isinstance(output_size, (int, tuple))
        self.output_size = output_size

    def __call__(self, data):

        image = data[0]    # Get picture
        key_pts = data[1]  # Get label

        image_copy = np.copy(image)      
        key_pts_copy = np.copy(key_pts)

        h, w = image_copy.shape[:2]

        new_h, new_w = self.output_size,self.output_size

        new_h, new_w = int(new_h), int(new_w)

        img = F.resize(image_copy, (new_h, new_w))
        
        # scale the pts, too
        #key_pts_copy[::2] = key_pts_copy[::2] * new_w / w
        #key_pts_copy[1::2] = key_pts_copy[1::2] * new_h / h

        return img, key_pts_copy




class GrayNormalize(object):
    # Change the picture to grayscale and zoom its value to [0, 1]
    # Shrink the label between [- 1, 1]

    def __call__(self, data):
        image = data[0]   # Get picture
        key_pts = data[1] # Get label
        
        image_copy = np.copy(image)
        key_pts_copy = np.copy(key_pts)

        # Grayscale picture
        gray_scale = paddle.vision.transforms.Grayscale(num_output_channels=3)
        image_copy = gray_scale(image_copy)
        
        # Zoom the picture value to [0, 1]
        image_copy = (image_copy-127.5) / 127.5
        
        # Zoom coordinate points to [- 1, 1]
        #mean = data_mean # Get tag mean
        #std = data_std   # Get label standard deviation

        #key_pts_copy = (key_pts_copy - mean)/std

        return image_copy, key_pts_copy

class ToCHW(object):
    # Change the format of the image from HWC to CHW
    def __call__(self, data):

        image = data[0]
        key_pts = data[1]

        transpose = T.Transpose((2, 0, 1)) # Change to CHW
        image = transpose(image)
        
        return image, key_pts

Load data asynchronously

train_loader = paddle.io.DataLoader(train_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=True, num_workers=0)
val_loader = paddle.io.DataLoader(val_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=True, num_workers=0)
test_loader=paddle.io.DataLoader(test_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=False, num_workers=0)

Use the pre training model for training

For example: resnet50, resnet101, resnet152, mobilenet_v1,mobilenet_v2 et al

self.net=paddle.vision.resnet152(pretrained=True)

Use cosine announcing strategy to dynamically adjust the learning rate

lr = paddle.optimizer.lr.CosineAnnealingDecay(learning_rate=1e-4,T_max=64)

Tried a new network structure:

class MyNet(paddle.nn.Layer):
    def __init__(self):
        super(MyNet, self).__init__()
        self.resnet = paddle.vision.resnet50(pretrained=True, num_classes=0) # The remove final fc output is [?, 2048, 1, 1]
        self.flatten = paddle.nn.Flatten()
        self.linear_1 = paddle.nn.Linear(2048, 512)
        self.linear_2 = paddle.nn.Linear(512, 256)
        self.linear_3 = paddle.nn.Linear(256, 2)
        self.relu = paddle.nn.ReLU()
        self.dropout = paddle.nn.Dropout(0.2)
    
    def forward(self, inputs):
        # print('input', inputs)
        y = self.resnet(inputs)
        y = self.flatten(y)
        y = self.linear_1(y)
        y = self.linear_2(y)
        y = self.relu(y)
        y = self.dropout(y)
        y = self.linear_3(y)
        y = paddle.nn.functional.sigmoid(y)

        return y

Tried a custom loss function:

def cal_coordinate_Loss(logit, label, alpha = 0.5):
    """
    logit: shape [batch, ndim]
    label: shape [batch, ndim]
    ndim = 2 represents coordinate_x and coordinaate_y
    alpha: weight for MSELoss and 1-alpha for ED loss
    return: combine MSELoss and ED Loss for x and y, shape [batch, 1]
    """
    alpha = alpha
    mse_loss = nn.MSELoss(reduction='mean')

    mse_x = mse_loss(logit[:,0],label[:,0])
    mse_y = mse_loss(logit[:,1],label[:,1])
    mse_l = 0.5*(mse_x + mse_y)
    # print('mse_l', mse_l)

    ed_loss = []
    # print(logit.shape[0])
    for i in range(logit.shape[0]):
        logit_tmp = logit[i,:].numpy()
        label_tmp = label[i,:].numpy()
        # print('cal_coordinate_loss_ed', logit_tmp, label_tmp)        
        ed_tmp = euclidean_distances([logit_tmp], [label_tmp])
        # print('ed_tmp:', ed_tmp[0][0])
        ed_loss.append(ed_tmp)
    
    ed_l = sum(ed_loss)/len(ed_loss)
    # print('ed_l', ed_l)
    # print('alpha', alpha)
    loss = alpha * mse_l + (1-alpha) * ed_l
    # print('loss in function', loss)
    return loss
class SelfDefineLoss(paddle.nn.Layer):
   """
   1. inherit paddle.nn.Layer
   """
   def __init__(self):
       """
       2. The constructor can define parameters according to its actual algorithm requirements and use requirements
       """
       super(SelfDefineLoss, self).__init__()

   def forward(self, input, label):
       """
       3. realization forward Function, forward When called, two parameters are passed: input and label
           - input: Single or batch training data are output through forward calculation of the model
           - label: Label data corresponding to single or batch training data
           The interface return value is a Tensor,The loss after adding or calculating the mean value according to the user-defined logic
       """
       # Using PaddlePaddle related API to customize computing logic
       output = cal_coordinate_Loss(input,label)
       return output

(4) Specific scheme sharing

Code reference: Face key point detection of "deep learning 7-day clock in camp"

Decompress dataset

!unzip -oq /home/aistudio/data/data116960/Regular season: PALM Localization of macular fovea in fundus color photography.zip
!mv │г╣ц╚№г║PALM╤█╡╫▓╩╒╒╓╨╗╞░▀╓╨╤ы░╝╢и╬╗ Regular season: PALM Localization of macular fovea in fundus color photography
!rm -rf __MACOSX

!mv /home/aistudio/PALM Localization of macular fovea in fundus color photography/* /home/aistudio/work/Regular season: PALM Localization of macular fovea in fundus color photography/

View data labels

import blackhole.dataframe as pd
df=pd.read_excel('Regular season: PALM Localization of macular fovea in fundus color photography/Train/Fovea_Location_train.xlsx')

df.head()
imgNameFovea_XFovea_Y
0H0001.jpg743.96790.54
1H0002.jpg1394.82725.54
2H0003.jpg1361.74870.72
3H0004.jpg703.15742.44
4H0005.jpg1070.951037.54
# Calculate the mean and standard deviation of the label for normalization of the label
key_pts_values = df.values[:,1:] # Take out label information
data_mean = key_pts_values.mean() # Calculated mean
data_std = key_pts_values.std() # Calculate standard deviation


print('The average value of the label is:', data_mean)
print('The standard deviation of the label is:', data_std)
The average value of the label is: 1085.6073687500023
 The standard deviation of the label is: 183.5345073716085

Data enhancement

import paddle.vision.transforms.functional as F
class Resize(object):
    # Adjusts the input image to the specified size

    def __init__(self, output_size):
        assert isinstance(output_size, (int, tuple))
        self.output_size = output_size

    def __call__(self, data):

        image = data[0]    # Get picture
        key_pts = data[1]  # Get label

        image_copy = np.copy(image)      
        key_pts_copy = np.copy(key_pts)

        h, w = image_copy.shape[:2]

        new_h, new_w = self.output_size,self.output_size

        new_h, new_w = int(new_h), int(new_w)

        img = F.resize(image_copy, (new_h, new_w))
        
        # scale the pts, too
        #key_pts_copy[::2] = key_pts_copy[::2] * new_w / w
        #key_pts_copy[1::2] = key_pts_copy[1::2] * new_h / h

        return img, key_pts_copy




class GrayNormalize(object):
    # Change the picture to grayscale and zoom its value to [0, 1]
    # Shrink the label between [- 1, 1]

    def __call__(self, data):
        image = data[0]   # Get picture
        key_pts = data[1] # Get label
        
        image_copy = np.copy(image)
        key_pts_copy = np.copy(key_pts)

        # Grayscale picture
        gray_scale = paddle.vision.transforms.Grayscale(num_output_channels=3)
        image_copy = gray_scale(image_copy)
        
        # Zoom the picture value to [0, 1]
        image_copy = (image_copy-127.5) / 127.5
        
        # Zoom coordinate points to [- 1, 1]
        #mean = data_mean # Get tag mean
        #std = data_std   # Get label standard deviation

        #key_pts_copy = (key_pts_copy - mean)/std

        return image_copy, key_pts_copy

class ToCHW(object):
    # Change the format of the image from HWC to CHW
    def __call__(self, data):

        image = data[0]
        key_pts = data[1]

        transpose = T.Transpose((2, 0, 1)) # Change to CHW
        image = transpose(image)
        
        return image, key_pts
import paddle.vision.transforms as T
data_transform = T.Compose([
                        Resize(224),
                        
                        GrayNormalize(),
                        ToCHW(),
                         ])
data_transform2 = T.Compose([
                        Resize(224),
                        GrayNormalize(),
                        ToCHW(),
                        ])

Custom dataset

path='Regular season: PALM Localization of macular fovea in fundus color photography/Train/fundus_image/'
df=df.sample(frac=1)
image_list=[]
label_listx=[]
label_listy=[]

for i in range(len(df)):
        image_list.append(path+df['imgName'][i])
        label_listx.append(df['Fovea_X'][i])
        label_listy.append(df['Fovea_Y'][i])
import os
test_path='Regular season: PALM Localization of macular fovea in fundus color photography/PALM-Testing400-Images'
test_list=[]
test_labelx=[]
test_labely=[]

list = os.listdir(test_path)  # List all directories and files under the folder
for i in range(0, len(list)):
    path = os.path.join(test_path, list[i])
    test_list.append(path)
    test_labelx.append(0)
    test_labely.append(0)

import paddle
import cv2
import numpy as np
class dataset(paddle.io.Dataset):
    def __init__(self,img_list,label_listx,label_listy,transform=None,transform2=None,mode='train'):

        self.image=img_list
        self.labelx=label_listx
        self.labely=label_listy
        self.mode=mode
        self.transform=transform
        self.transform2=transform2
    def load_img(self, image_path):

        img=cv2.imread(image_path,1)
        img=cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
        h,w,c=img.shape
        return img,h,w


    def __getitem__(self,index):
        img,h,w = self.load_img(self.image[index])
        labelx = self.labelx[index]/w
        labely = self.labely[index]/h
        img_size=img.shape

        if self.transform:
            if self.mode=='train':
                img, label = self.transform([img, [labelx,labely]])
            else:
                img, label = self.transform2([img, [labelx,labely]])
        
        
        label=np.array(label,dtype='float32')
        img=np.array(img,dtype='float32')
        return img,label

    def __len__(self):
        return len(self.image)

Training set, verification set, test set

radio=0.8
train_list=image_list[:int(len(image_list)*radio)]
train_labelx=label_listx[:int(len(label_listx)*radio)]
train_labely=label_listy[:int(len(label_listy)*radio)]


val_list=image_list[int(len(image_list)*radio):]
val_labelx=label_listx[int(len(label_listx)*radio):]
val_labely=label_listy[int(len(label_listy)*radio):]


train_ds=dataset(train_list,train_labelx,train_labely,data_transform,data_transform2,'train')
val_ds=dataset(val_list,val_labelx,val_labely,data_transform,data_transform2,'valid')
test_ds=dataset(test_list,test_labelx,test_labely,data_transform,data_transform2,'test')

view picture

import matplotlib.pyplot as plt
for i,data in enumerate(train_ds):
    
    img,label=data

    img=img.transpose([1,2,0])
    print(img.shape)
    
    plt.title(label)
    plt.imshow(img)
    plt.show()

    if i==0:
        break
Clipping input data to the valid range for imshow with RGB data ([0..1] for floats or [0..255] for integers).


(224, 224, 3)

Model networking

Choose one of the following two network structures

class MyNet(paddle.nn.Layer):
    def __init__(self,num_classes=2):
        super(MyNet,self).__init__()
        self.net=paddle.vision.resnet152(pretrained=True)

        self.fc1=paddle.nn.Linear(1000,512)
        self.relu=paddle.nn.ReLU()
        self.fc2=paddle.nn.Linear(512,num_classes)

    def forward(self,inputs):
        out=self.net(inputs)

        out=self.fc1(out)
        out=self.relu(out)
        out=self.fc2(out)

        return out
class MyNet(paddle.nn.Layer):
    def __init__(self):
        super(MyNet, self).__init__()
        self.resnet = paddle.vision.resnet50(pretrained=True, num_classes=0) # The remove final fc output is [?, 2048, 1, 1]
        self.flatten = paddle.nn.Flatten()
        self.linear_1 = paddle.nn.Linear(2048, 512)
        self.linear_2 = paddle.nn.Linear(512, 256)
        self.linear_3 = paddle.nn.Linear(256, 2)
        self.relu = paddle.nn.ReLU()
        self.dropout = paddle.nn.Dropout(0.2)
    
    def forward(self, inputs):

        y = self.resnet(inputs)
        y = self.flatten(y)
        y = self.linear_1(y)
        y = self.linear_2(y)
        y = self.relu(y)
        y = self.dropout(y)
        y = self.linear_3(y)
        y = paddle.nn.functional.sigmoid(y)

        return y

Load data asynchronously

train_loader = paddle.io.DataLoader(train_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=True, num_workers=0)
val_loader = paddle.io.DataLoader(val_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=False, num_workers=0)
test_loader=paddle.io.DataLoader(test_ds, places=paddle.CPUPlace(), batch_size=32, shuffle=False, num_workers=0)

Custom loss function

from sklearn.metrics.pairwise import euclidean_distances 
import paddle.nn as nn
# loss function 
def cal_coordinate_Loss(logit, label, alpha = 0.5):
    """
    logit: shape [batch, ndim]
    label: shape [batch, ndim]
    ndim = 2 represents coordinate_x and coordinaate_y
    alpha: weight for MSELoss and 1-alpha for ED loss
    return: combine MSELoss and ED Loss for x and y, shape [batch, 1]
    """
    alpha = alpha
    mse_loss = nn.MSELoss(reduction='mean')

    mse_x = mse_loss(logit[:,0],label[:,0])
    mse_y = mse_loss(logit[:,1],label[:,1])
    mse_l = 0.5*(mse_x + mse_y)
    # print('mse_l', mse_l)

    ed_loss = []
    # print(logit.shape[0])
    for i in range(logit.shape[0]):
        logit_tmp = logit[i,:].numpy()
        label_tmp = label[i,:].numpy()
        # print('cal_coordinate_loss_ed', logit_tmp, label_tmp)        
        ed_tmp = euclidean_distances([logit_tmp], [label_tmp])
        # print('ed_tmp:', ed_tmp[0][0])
        ed_loss.append(ed_tmp)
    
    ed_l = sum(ed_loss)/len(ed_loss)
    # print('ed_l', ed_l)
    # print('alpha', alpha)
    loss = alpha * mse_l + (1-alpha) * ed_l
    # print('loss in function', loss)
    return loss
class SelfDefineLoss(paddle.nn.Layer):
   """
   1. inherit paddle.nn.Layer
   """
   def __init__(self):
       """
       2. The constructor can define parameters according to its actual algorithm requirements and use requirements
       """
       super(SelfDefineLoss, self).__init__()

   def forward(self, input, label):
       """
       3. realization forward Function, forward When called, two parameters are passed: input and label
           - input: Single or batch training data are output through forward calculation of the model
           - label: Label data corresponding to single or batch training data
           The interface return value is a Tensor,The loss after adding or calculating the mean value according to the user-defined logic
       """
       # Using PaddlePaddle related API to customize computing logic
       output = cal_coordinate_Loss(input,label)
       return output

Model training and visualization

If the picture size is large, the batch should be appropriately reduced_ Size to prevent explosion of video memory.

from utils import NME
visualdl=paddle.callbacks.VisualDL(log_dir='visual_log')
#Define input

Batch_size=32
EPOCHS=20
step_each_epoch = len(train_ds)//Batch_size


# Encapsulate the model using the pad. Model
model = paddle.Model(MyNet())

#Model loading
model.load('/home/aistudio/work/lup/final')


lr = paddle.optimizer.lr.CosineAnnealingDecay(learning_rate=1e-5,
                                                T_max=step_each_epoch * EPOCHS)




# Define Adam optimizer
optimizer = paddle.optimizer.Adam(learning_rate=lr,
                                weight_decay=1e-5,
                                parameters=model.parameters())
# Define SmoothL1Loss
loss =paddle.nn.SmoothL1Loss()
#loss =SelfDefineLoss()

# Using custom metrics
metric = NME()

model.prepare(optimizer=optimizer, loss=loss, metrics=metric)




# Callback function of training visual DL tool

# Start the whole process training of the model
model.fit(train_loader,  # Training data set
          val_loader,   # Evaluation data set
          epochs=EPOCHS,       # Total rounds of training
          batch_size=Batch_size,  # Batch size for training
          save_dir="/home/aistudio/work/lup", #Save the model parameters and optimizer parameters to the customized folder
          save_freq=1,                    #Set how many epoch s to save model parameters and optimizer parameters
          verbose=1 ,      # Log display form
          callbacks=[visualdl]
          )  # Set up visualization
The loss value printed in the log is the current step, and the metric is the average value of previous steps.
Epoch 1/20


/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/paddle/nn/layer/norm.py:641: UserWarning: When training, we now always track global mean and variance.
  "When training, we now always track global mean and variance.")


step 20/20 [==============================] - loss: 2.4430e-04 - nme: 0.0307 - 2s/step          
save checkpoint at /home/aistudio/work/lup/0
Eval begin...
step 5/5 [==============================] - loss: 2.0670e-04 - nme: 0.0364 - 2s/step
Eval samples: 160
Epoch 2/20
step 20/20 [==============================] - loss: 2.6849e-04 - nme: 0.0280 - 2s/step          
save checkpoint at /home/aistudio/work/lup/1
Eval begin...
step 5/5 [==============================] - loss: 1.7744e-04 - nme: 0.0347 - 2s/step
Eval samples: 160
Epoch 3/20
step 20/20 [==============================] - loss: 1.6573e-04 - nme: 0.0250 - 2s/step          
save checkpoint at /home/aistudio/work/lup/2
Eval begin...
step 5/5 [==============================] - loss: 2.2369e-04 - nme: 0.0359 - 2s/step
Eval samples: 160
Epoch 4/20
step 20/20 [==============================] - loss: 2.4454e-04 - nme: 0.0250 - 2s/step          
save checkpoint at /home/aistudio/work/lup/3
Eval begin...
step 5/5 [==============================] - loss: 2.0749e-04 - nme: 0.0345 - 2s/step
Eval samples: 160
Epoch 5/20
step 20/20 [==============================] - loss: 2.6878e-04 - nme: 0.0244 - 2s/step          
save checkpoint at /home/aistudio/work/lup/4
Eval begin...
step 5/5 [==============================] - loss: 2.2863e-04 - nme: 0.0352 - 2s/step
Eval samples: 160
Epoch 6/20
step 20/20 [==============================] - loss: 2.1071e-04 - nme: 0.0224 - 2s/step          
save checkpoint at /home/aistudio/work/lup/5
Eval begin...
step 5/5 [==============================] - loss: 1.7807e-04 - nme: 0.0345 - 2s/step
Eval samples: 160
Epoch 7/20
step 20/20 [==============================] - loss: 3.4471e-04 - nme: 0.0229 - 2s/step          
save checkpoint at /home/aistudio/work/lup/6
Eval begin...
step 5/5 [==============================] - loss: 1.7128e-04 - nme: 0.0337 - 2s/step
Eval samples: 160
Epoch 8/20
step 20/20 [==============================] - loss: 1.9624e-04 - nme: 0.0224 - 2s/step          
save checkpoint at /home/aistudio/work/lup/7
Eval begin...
step 5/5 [==============================] - loss: 1.7516e-04 - nme: 0.0339 - 2s/step
Eval samples: 160
Epoch 9/20
step 20/20 [==============================] - loss: 1.2843e-04 - nme: 0.0213 - 2s/step          
save checkpoint at /home/aistudio/work/lup/8
Eval begin...
step 5/5 [==============================] - loss: 2.3785e-04 - nme: 0.0355 - 2s/step
Eval samples: 160
Epoch 10/20
step 20/20 [==============================] - loss: 2.0355e-04 - nme: 0.0207 - 2s/step          
save checkpoint at /home/aistudio/work/lup/9
Eval begin...
step 5/5 [==============================] - loss: 1.8947e-04 - nme: 0.0341 - 2s/step
Eval samples: 160
Epoch 11/20
step 20/20 [==============================] - loss: 2.8434e-04 - nme: 0.0210 - 2s/step          
save checkpoint at /home/aistudio/work/lup/10
Eval begin...
step 5/5 [==============================] - loss: 2.1608e-04 - nme: 0.0342 - 2s/step
Eval samples: 160
Epoch 12/20
step 20/20 [==============================] - loss: 2.5887e-04 - nme: 0.0205 - 2s/step          
save checkpoint at /home/aistudio/work/lup/11
Eval begin...
step 5/5 [==============================] - loss: 1.9807e-04 - nme: 0.0343 - 2s/step
Eval samples: 160
Epoch 13/20
step 20/20 [==============================] - loss: 1.1868e-04 - nme: 0.0193 - 2s/step          
save checkpoint at /home/aistudio/work/lup/12
Eval begin...
step 5/5 [==============================] - loss: 1.8711e-04 - nme: 0.0338 - 2s/step
Eval samples: 160
Epoch 14/20
step 20/20 [==============================] - loss: 1.3952e-04 - nme: 0.0196 - 2s/step          
save checkpoint at /home/aistudio/work/lup/13
Eval begin...
step 5/5 [==============================] - loss: 1.8402e-04 - nme: 0.0335 - 2s/step
Eval samples: 160
Epoch 15/20
step 20/20 [==============================] - loss: 1.2068e-04 - nme: 0.0200 - 2s/step          
save checkpoint at /home/aistudio/work/lup/14
Eval begin...
step 5/5 [==============================] - loss: 1.9125e-04 - nme: 0.0338 - 2s/step
Eval samples: 160
Epoch 16/20
step 20/20 [==============================] - loss: 1.9597e-04 - nme: 0.0199 - 2s/step          
save checkpoint at /home/aistudio/work/lup/15
Eval begin...
step 5/5 [==============================] - loss: 1.8797e-04 - nme: 0.0336 - 2s/step
Eval samples: 160
Epoch 17/20
step 20/20 [==============================] - loss: 9.2524e-05 - nme: 0.0186 - 2s/step          
save checkpoint at /home/aistudio/work/lup/16
Eval begin...
step 5/5 [==============================] - loss: 1.9081e-04 - nme: 0.0339 - 2s/step
Eval samples: 160
Epoch 18/20
step 20/20 [==============================] - loss: 1.3978e-04 - nme: 0.0190 - 2s/step          
save checkpoint at /home/aistudio/work/lup/17
Eval begin...
step 5/5 [==============================] - loss: 1.9073e-04 - nme: 0.0338 - 2s/step
Eval samples: 160
Epoch 19/20
step 20/20 [==============================] - loss: 1.6930e-04 - nme: 0.0185 - 2s/step          
save checkpoint at /home/aistudio/work/lup/18
Eval begin...
step 5/5 [==============================] - loss: 1.9041e-04 - nme: 0.0336 - 2s/step
Eval samples: 160
Epoch 20/20
step 20/20 [==============================] - loss: 5.3823e-05 - nme: 0.0185 - 2s/step          
save checkpoint at /home/aistudio/work/lup/19
Eval begin...
step 5/5 [==============================] - loss: 1.9054e-04 - nme: 0.0337 - 2s/step
Eval samples: 160
save checkpoint at /home/aistudio/work/lup/final

Model evaluation

# Model evaluation
model.load('/home/aistudio/work/lup/13')
result = model.evaluate(val_loader, verbose=1)
print(result)
Eval begin...
step 5/5 [==============================] - loss: 1.8402e-04 - nme: 0.0335 - 2s/step
Eval samples: 160
{'loss': [0.00018401867], 'nme': 0.033542318427836}

Perform prediction operation

# Perform prediction operation
result = model.predict(test_loader)
Predict begin...
step 13/13 [==============================] - 2s/step         
Predict samples: 400
# Get test picture size and picture name
test_path='Regular season: PALM Localization of macular fovea in fundus color photography/PALM-Testing400-Images'
test_size=[]
FileName=[]
for i in range(len(list)):
    path = os.path.join(test_path, list[i])
    img=cv2.imread(path,1)
    test_size.append(img.shape)
    FileName.append(list[i])
test_size=np.array(test_size)
result=np.array(result)
pred=[]
for i in range(len(result[0])):
    pred.extend(result[0][i])
pred=np.array(pred) 

pred = paddle.to_tensor(pred)
out=np.array(pred).reshape(-1,2)

#Fovea_X=out[:,0]*data_std+data_mean
#Fovea_Y=out[:,1]*data_std+data_mean
Fovea_X=out[:,0]
Fovea_Y=out[:,1]

Fovea_X=Fovea_X*test_size[:,1]
Fovea_Y=Fovea_Y*test_size[:,0]

submission = pd.DataFrame(data={
                            "FileName": FileName,
                            "Fovea_X": Fovea_X,
                            "Fovea_Y": Fovea_Y
                        })
submission=submission.sort_values(by='FileName')
submission.to_csv("result.csv", index=False)
/opt/conda/envs/python35-paddle120-env/lib/python3.7/site-packages/ipykernel_launcher.py:1: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray.
  """Entry point for launching an IPython kernel.

Result file view

submission.head()
FileNameFovea_XFovea_Y
358T0001.jpg1285.678868988.328547
227T0002.jpg1073.5387281069.430128
116T0003.jpg1067.2829021032.210720
51T0004.jpg1178.123714984.575601
315T0005.jpg1268.955749740.036470

Result voting integration

Simple voting integrated learning can improve the effect and try to choose the one with high score to vote.

In statistics and machine learning, ensemble learning methods use multiple learning algorithms to obtain better prediction performance than any single learning algorithm alone.

Different prediction results can be obtained by running multiple models with different super parameters, different characteristics and different structures. Here I use the simple voting method, taking the average value as the final prediction result. After multiple results are predicted, vote. The code is as follows:

import numpy as np
import blackhole.dataframe as pd
df1=pd.read_csv('result41.958.csv')
df2=pd.read_csv('result41.958.csv')
df3=pd.read_csv('result49.65447.csv')
df4=pd.read_csv('result49.75246.csv')


dfs=[df1,df2,df3,df4]

File_Name=[]
Fovea_X=[]
Fovea_Y=[]
for i in range(len(df1)):
    File_Name.append(dfs[0]['FileName'][i])
    avgx=(sum(np.array(dfs[x]['Fovea_X'][i]) for x in range(len(dfs))))/len(dfs)
    avgy=(sum(np.array(dfs[x]['Fovea_Y'][i]) for x in range(len(dfs))))/len(dfs)
    
    Fovea_X.append(avgx)
    Fovea_Y.append(avgy)
submission = pd.DataFrame(data={
                            "FileName": File_Name,
                            "Fovea_X": Fovea_X,
                            "Fovea_Y":Fovea_Y
                        })
submission=submission.sort_values(by='FileName')
p.array(dfs[x]['Fovea_Y'][i]) for x in range(len(dfs))))/len(dfs)
    
    Fovea_X.append(avgx)
    Fovea_Y.append(avgy)
submission = pd.DataFrame(data={
                            "FileName": File_Name,
                            "Fovea_X": Fovea_X,
                            "Fovea_Y":Fovea_Y
                        })
submission=submission.sort_values(by='FileName')
submission.to_csv("result.csv", index=False)

(5) Summary and improvement direction

1. Try a few more pre training models.

2. Choose the right learning rate.

3. Replace another optimizer.

4. Voting can improve results, but there is a ceiling.

5. "UserWarning: When training, we now always track global mean and variance" often appears when using the custom loss function, which has not been solved.

6. The training neural network was specially used to classify the visible and invisible situations of the fovea. As a result, the test set was not divided into the invisible category of the fovea, but it was always felt that the positioning of the test set should be (0, 0), which was not very sure.

(6) Experience of flying oars + suggestions for other players to learn flying oars

I suggest you take more Baidu AI Studio courses and read more AI Studio projects written by others. You may have inspiration and get better results in the competition.

(7)One More Thing

Localization of macular fovea in fundus color photography

1.Pathological myopia classification with simultaneous lesion segmentation using deep learning

2.Detection of Pathological Myopia and Optic Disc Segmentation with Deep Convolutional Neural Networks

I got the gold level in AI Studio and lit 9 badges to turn on each other~

Tags: Computer Vision paddlepaddle

Posted on Fri, 26 Nov 2021 20:41:06 -0500 by Singularity