Establishment of face recognition data set (dlib+opencv) and face recognition

1.1 library file preparation

• First, we need to prepare two library files
• First: dlib_face_recognition_resnet_model_v1.dat
  Baidu online disk download link:
  Link: https://pan.baidu.com/s/1tPt6JaAoksUKBm0MVO248A
  Extraction code: 0000
• Second: shape_predictor_68_face_landmarks.dat
  Baidu online disk download link:
  Link: https://pan.baidu.com/s/1_I0Z5WLpKb-w6IWjH-RIgQ
  Extraction code: 0001
• After downloading, put it in a folder. This file path will be used in the code later.

1.2 entering face data

1.2.1 code

• Open the Jupiter notebook of Acaconda3
• Enter code:

• Be sure to pay attention to the file path and change it to your own path.

import cv2 import dlib import os import sys import random # Storage location output_dir = 'D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/1' size = 64 if not os.path.exists(output_dir): os.makedirs(output_dir) # Change the brightness and contrast of the picture def relight(img, light=1, bias=0): w = img.shape[1] h = img.shape[0] #image = [] for i in range(0,w): for j in range(0,h): for c in range(3): tmp = int(img[j,i,c]*light + bias) if tmp > 255: tmp = 255 elif tmp < 0: tmp = 0 img[j,i,c] = tmp return img #Use the front_face_detector of dlib as our feature extractor detector = dlib.get_frontal_face_detector() # Open the camera parameter as the input stream, which can be a camera or video file camera = cv2.VideoCapture(0) #camera = cv2.VideoCapture('C:/Users/CUNGU/Videos/Captures/wang.mp4') index = 1 while True: if (index <= 20):#Store 20 face feature images print('Being processed picture %s' % index) # Read photos from camera success, img = camera.read() # Convert to grayscale image gray_img = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Face detection using detector dets = detector(gray_img, 1) for i, d in enumerate(dets): x1 = d.top() if d.top() > 0 else 0 y1 = d.bottom() if d.bottom() > 0 else 0 x2 = d.left() if d.left() > 0 else 0 y2 = d.right() if d.right() > 0 else 0 face = img[x1:y1,x2:y2] # Adjust the contrast and brightness of the picture. Both contrast and brightness values take random numbers, which can increase the diversity of samples face = relight(face, random.uniform(0.5, 1.5), random.randint(-50, 50)) face = cv2.resize(face, (size,size)) cv2.imshow('image', face) cv2.imwrite(output_dir+'/'+str(index)+'.jpg', face) index += 1 key = cv2.waitKey(30) & 0xff if key == 27: break else: print('Finished!') # release camera camera.release() # delete all the windows cv2.destroyAllWindows() break

1.2.2 operation results

• Find it under the corresponding file and you can see the 20 intercepted faces:

Collect 68 feature point arrays corresponding to 20 pictures and save them in the same directory as face_features.txt (i is a number from 01 to 20). Through 20 features, calculate the average feature array face_feature_mean.csv.

2.1 dlib face feature detection principle

1. Extract feature points:
   Example:
   

2. Save feature values to a CSV file

3. Calculate the Euclidean distance of the characteristic data set for comparison, and judge the same person when the error is less than a certain threshold.

2.2 collect the feature point array and calculate the average feature array

2.2.1 code

# Extract face features from face image files and store them in CSV # Features extraction from images and save into features_all.csv # return_128d_features() gets the 128D features of an image # compute_the_mean() calculates the 128D characteristic mean from cv2 import cv2 as cv2 import os import dlib from skimage import io import csv import numpy as np # The path to read the face image file path_images_from_camera = "D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/" # Dlib forward face detector detector = dlib.get_frontal_face_detector() # Dlib face predictor predictor = dlib.shape_predictor("D:/ProgramData/wenjian/shape_predictor_68_face_landmarks.dat") # Dlib face recognition model # Face recognition model, the object maps human faces into 128D vectors face_rec = dlib.face_recognition_model_v1("D:/ProgramData/wenjian/dlib_face_recognition_resnet_model_v1.dat") # Returns the 128D feature of a single image def return_128d_features(path_img): img_rd = io.imread(path_img) img_gray = cv2.cvtColor(img_rd, cv2.COLOR_BGR2RGB) faces = detector(img_gray, 1) print("%-40s %-20s" % ("Face image detected / image with faces detected:", path_img), '\n') # Because it is possible to detect the cut face, the face can not be detected # So make sure that the face image that detects the face is used to calculate the feature if len(faces) != 0: shape = predictor(img_gray, faces[0]) face_descriptor = face_rec.compute_face_descriptor(img_gray, shape) else: face_descriptor = 0 print("no face") return face_descriptor # Extract the photo features in the folder and write them to CSV def return_features_mean_personX(path_faces_personX): features_list_personX = [] photos_list = os.listdir(path_faces_personX) if photos_list: for i in range(len(photos_list)): # Call return_128d_features() to get 128d features print("%-40s %-20s" % ("Face image being read / image to read:", path_faces_personX + "/" + photos_list[i])) features_128d = return_128d_features(path_faces_personX + "/" + photos_list[i]) # print(features_128d) # If you encounter a picture with no face detected, skip if features_128d == 0: i += 1 else: features_list_personX.append(features_128d) i1=str(i+1) add="D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/csv/"+"face_features"+i1+".csv" print(add) with open(add, "w", newline="") as csvfile: writer1 = csv.writer(csvfile) writer1.writerow(features_128d) else: print("The image file in the folder is empty / Warning: No images in " + path_faces_personX + '/', '\n') # Calculate the mean of 128D features # N x 128D -> 1 x 128D if features_list_personX: features_mean_personX = np.array(features_list_personX).mean(axis=0) else: features_mean_personX = '0' return features_mean_personX # Read the data of all someone's face images people = os.listdir(path_images_from_camera) people.sort() with open("D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/face_feature_mean.csv", "w", newline="") as csvfile: writer = csv.writer(csvfile) for person in people: print("##### " + person + " #####") # Get the mean/average features of face/personX, it will be a list with a length of 128D features_mean_personX = return_features_mean_personX(path_images_from_camera + person) writer.writerow(features_mean_personX) print("Characteristic mean / The mean of features:", list(features_mean_personX)) print('\n') print("All entered face data are stored in / Save all the features of faces registered into: D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/face_feature_mean.csv")

2.2.2 operation results

• Find it under the corresponding file, and you can see the. csv file of collecting feature point array and average feature array

Using dlib and opencv programming, open the camera, extract the features of the captured face, and calculate the error (Euclidean distance) with the average features. When the error is less than a certain threshold, it is judged as the same person, otherwise it is judged as unknown.

3.1 Code:

# Camera real-time face recognition import os import winsound # system sound from playsound import playsound # Audio playback import dlib # Face processing library Dlib import csv # Deposit form import time import sys import numpy as np # Data processing library numpy from cv2 import cv2 as cv2 # Image processing library OpenCv import pandas as pd # Data processing library Pandas # Face recognition model, extract 128D feature vector # face recognition model, the object maps human faces into 128D vectors # Refer this tutorial: http://dlib.net/python/index.html#dlib.face_recognition_model_v1 facerec = dlib.face_recognition_model_v1("D:/ProgramData/wenjian/dlib_face_recognition_resnet_model_v1.dat") # Calculate the Euclidean distance between two 128D vectors # compute the e-distance between two 128D features def return_euclidean_distance(feature_1, feature_2): feature_1 = np.array(feature_1) feature_2 = np.array(feature_2) dist = np.sqrt(np.sum(np.square(feature_1 - feature_2))) return dist # Process the csv that stores all face features path_features_known_csv = "D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/face_feature_mean.csv" csv_rd = pd.read_csv(path_features_known_csv, header=None) # An array used to store all input face features # the array to save the features of faces in the database features_known_arr = [] # Read known face data # print known faces for i in range(csv_rd.shape[0]): features_someone_arr = [] for j in range(0, len(csv_rd.iloc[i, :])): features_someone_arr.append(csv_rd.iloc[i, :][j]) features_known_arr.append(features_someone_arr) print("Faces in Database: ", len(features_known_arr)) # Dlib detector and predictor # The detector and predictor will be used detector = dlib.get_frontal_face_detector() predictor = dlib.shape_predictor('D:/ProgramData/wenjian/shape_predictor_68_face_landmarks.dat') # Create cv2 camera object # cv2.VideoCapture(0) to use the default camera of PC, # and you can use local video name by use cv2.VideoCapture(filename) cap = cv2.VideoCapture(0) # cap.set(propId, value) # Set video parameters, video parameters set by propId, and parameter values set by value cap.set(3, 480) # Cap. Isepened() returns true/false to check whether the initialization is successful # when the camera is open while cap.isOpened(): flag, img_rd = cap.read() kk = cv2.waitKey(1) # Take gray img_gray = cv2.cvtColor(img_rd, cv2.COLOR_RGB2GRAY) # Number of faces faces = detector(img_gray, 0) # font to write later font = cv2.FONT_HERSHEY_COMPLEX # Store the coordinates / names of all faces captured in the current camera # the list to save the positions and names of current faces captured pos_namelist = [] name_namelist = [] # Press the q key to exit # press 'q' to exit if kk == ord('q'): break else: # when face detected if len(faces) != 0: # Obtain the features of all faces of the currently captured image and store them in features_cap_arr # get the features captured and save into features_cap_arr features_cap_arr = [] for i in range(len(faces)): shape = predictor(img_rd, faces[i]) features_cap_arr.append(facerec.compute_face_descriptor(img_rd, shape)) # Traverse all faces in the captured image # traversal all the faces in the database for k in range(len(faces)): print("##### camera person", k+1, "#####") # Let the person's name follow below the rectangle # Determine the location coordinates of the person's name # Let's default that everyone doesn't know. It's unknown # set the default names of faces with "unknown" name_namelist.append("unknown") # the positions of faces captured pos_namelist.append(tuple([faces[k].left(), int(faces[k].bottom() + (faces[k].bottom() - faces[k].top())/4)])) # For a face, traverse all stored face features # for every faces detected, compare the faces in the database e_distance_list = [] for i in range(len(features_known_arr)): # If person_X data is not empty if str(features_known_arr[i][0]) != '0.0': print("with person", str(i + 1), "the e distance: ", end='') e_distance_tmp = return_euclidean_distance(features_cap_arr[k], features_known_arr[i]) print(e_distance_tmp) e_distance_list.append(e_distance_tmp) else: # Empty data person_X e_distance_list.append(999999999) # Find out the closest face data # Find the one with minimum e distance similar_person_num = e_distance_list.index(min(e_distance_list)) print("Minimum e distance with person", int(similar_person_num)+1) # Calculate the Euclidean distance between face recognition features and data set features # If the distance is less than 0.4, it is marked as an identifiable person if min(e_distance_list) < 0.4: # Here you can modify the names of people marked in the camera # Here you can modify the names shown on the camera # 1. Traverse folder directory folder_name = 'D:/myworkspace/JupyterNotebook/People/person/person1/631907060228/' # Closest face sum=similar_person_num+1 key_id=1 # Compare from the first face data folder # Get the file name in the folder: 1wang, 2zhou, 3 file_names = os.listdir(folder_name) for name in file_names: # print(name+'->'+str(key_id)) if sum ==key_id: #winsound.Beep(300,500)# Ring: 300 frequency, 500 duration name_namelist[k] = name[0:]#Delete the first number of person name (for video output identification) key_id += 1 # Play welcome sound #playsound('D:/myworkspace/JupyterNotebook/People/music/welcome.wav') # print("May be person "+str(int(similar_person_num)+1)) # -----------Filter out faces and save them to the visitor folder------------ for i, d in enumerate(faces): x1 = d.top() if d.top() > 0 else 0 y1 = d.bottom() if d.bottom() > 0 else 0 x2 = d.left() if d.left() > 0 else 0 y2 = d.right() if d.right() > 0 else 0 face = img_rd[x1:y1,x2:y2] size = 64 face = cv2.resize(face, (size,size)) # The path to store the visitor face image file path_visitors_save_dir = "D:/myworkspace/JupyterNotebook/People/person/person1/visitor/known" # Storage format: 2019-06-24-14-33-40wang.jpg now_time = time.strftime("%Y-%m-%d-%H-%M-%S", time.localtime()) save_name = str(now_time)+str(name_namelist[k])+'.jpg' # print(save_name) # The complete url of this picture save save_path = path_visitors_save_dir+'/'+ save_name # Traverse all file names in the visitor folder visitor_names = os.listdir(path_visitors_save_dir) visitor_name='' for name in visitor_names: # Name slice to minutes: 2019-06-26-11-33-00wangyu.jpg visitor_name=(name[0:16]+'-00'+name[19:]) # print(visitor_name) visitor_save=(save_name[0:16]+'-00'+save_name[19:]) # print(visitor_save) # Duplicate names within one minute will not be saved if visitor_save!=visitor_name: cv2.imwrite(save_path, face) print('New storage:'+path_visitors_save_dir+'/'+str(now_time)+str(name_namelist[k])+'.jpg') else: print('Duplicate, unsaved!') else: # Play unrecognized sound #playsound('D:/myworkspace/JupyterNotebook/People/music/sorry.wav') print("Unknown person") # -----Save picture------- # -----------Filter out faces and save them to the visitor folder------------ for i, d in enumerate(faces): x1 = d.top() if d.top() > 0 else 0 y1 = d.bottom() if d.bottom() > 0 else 0 x2 = d.left() if d.left() > 0 else 0 y2 = d.right() if d.right() > 0 else 0 face = img_rd[x1:y1,x2:y2] size = 64 face = cv2.resize(face, (size,size)) # The path to store the visitor - unknown face image file path_visitors_save_dir = "D:/myworkspace/JupyterNotebook/People/person/person1/visitor/unknown" # Storage format: 2019-06-24-14-33-40 unknown.jpg now_time = time.strftime("%Y-%m-%d-%H-%M-%S", time.localtime()) # print(save_name) # The complete url of this picture save save_path = path_visitors_save_dir+'/'+ str(now_time)+'unknown.jpg' cv2.imwrite(save_path, face) print('New storage:'+path_visitors_save_dir+'/'+str(now_time)+'unknown.jpg') # Rectangular box # draw rectangle for kk, d in enumerate(faces): # Draw a rectangular box cv2.rectangle(img_rd, tuple([d.left(), d.top()]), tuple([d.right(), d.bottom()]), (0, 255, 255), 2) print('\n') # Write the face name under the face box # write names under rectangle for i in range(len(faces)): cv2.putText(img_rd, name_namelist[i], pos_namelist[i], font, 0.8, (0, 255, 255), 1, cv2.LINE_AA) print("Faces in camera now:", name_namelist, "\n") #cv2.putText(img_rd, "Press 'q': Quit", (20, 450), font, 0.8, (84, 255, 159), 1, cv2.LINE_AA) cv2.putText(img_rd, "Face Recognition", (20, 40), font, 1, (0, 0, 255), 1, cv2.LINE_AA) cv2.putText(img_rd, "Visitors: " + str(len(faces)), (20, 100), font, 1, (0, 0, 255), 1, cv2.LINE_AA) # The window displays show with opencv cv2.imshow("camera", img_rd) # release camera cap.release() # delete all the windows cv2.destroyAllWindows()

3.2 operation results

• Display 1 after recognition:

• Unrecognized display unknown:

It is recommended that the three parts of face input, face dataset construction and face recognition run with the new Python 3 of different Jupiter notebook, which will not be easy to make mistakes, and it will be easier to troubleshoot.

This experiment is also very interesting. Face recognition can be achieved by inputting faces, analyzing pictures and obtaining data. It seems simple, but it is not. Face recognition, we should first recognize the face, and then recognize whether it is you or not.

When running the face detection and recognition code, you may not know how to exit. Method: press q/Q to exit.

1. Establishment and application of face recognition data set
2. Dlib model face feature detection principle and demo
3. Face feature extraction based on dlib database [build your own face recognition data set]

Add new comment

0 comments