Design and implementation of catering pest rat identification and control system based on MobileNet-v3 and YOLOv5

In the context of the new era, "refreshing and clean" and "health" have become two important factors for people's yearning for a better life, which is also the constant pursuit and goal of current catering industry managers. With the development and progress of science and technology, artificial intelligence has become an important product of civilized society and is gradually applied to all aspects of daily life. Based on this, it is very necessary to design and develop a system that can effectively prevent pests and rats to improve management efficiency and health service quality

This paper mainly includes:

  1. be based on   Design and implementation of insect and mouse recognition function of MobileNet-v3
  2. Design and implementation of mouse detection function based on YOLOv5
  3. Design and implementation of function packaging and human-computer interaction interface based on PyQt5  

catalogue

1. Finished product effect demonstration

two   be based on   Design and implementation of insect and mouse recognition function of MobileNet-v3

three   Design and implementation of mouse detection function based on YOLOv5

4 design and implementation of function packaging and human-computer interaction interface based on PyQt5

5 main references

1. Finished product effect demonstration

1.1 picture finished product effect demonstration

1.2 video finished product effect demonstration

two   be based on   Design and implementation of insect and mouse recognition function of MobileNet-v3

2.1 acquisition and sorting of data sets

2.1.1 data acquisition

As we all know, data acquisition is an essential skill in the field of deep learning. There are various data acquisition methods, specifically: ① find the public data sets related to the task (such as COCO data sets and ImageNet data sets used for image recognition); ② Use the outsourcing platform to obtain data (such as Alibaba crowdsourcing, baidu data crowdsourcing, JD micro industry, etc.) ③ of course, you can also collect data with your own camera as needed; ④ The last way is to use web crawler technology.

Here, the fourth data acquisition method - web crawler is used. Used is a Baidu image crawler APP written by myself before.

For readers interested in the baidu image crawler APP, please move to:
First blog post: When the crawler encounters PyQt5: GUI interface, baidu image crawling is realized
Get GitHub source code: PyQt5/reptile at main · zhao302014/PyQt5 · GitHub

Using the crawler tool written by myself, a total of   There are 50 data sets of Five insect and mouse categories, including ant, cockroach, fly, mouse and pillworm, with a total of 250.

2.1.2 data sorting

Data collation can mainly consider three levels: data inspection and normalization, data De duplication and data set division.

Obviously, the data set obtained in the previous step is too small in both component and total, which is easy to cause over fitting due to too small data set or too large model. Therefore, the simplest and rough method to prevent over fitting is adopted here: implicit regularization method - data enhancement. Specifically, data enhancement methods including Scale, Horizontal, Rotate, Darker, brightener, Translation and AddNoise are used (implemented based on relevant methods of OpenCV)

# Scaling operation of data enhancement
def Scale(image, scale):
    return cv2.resize(image, None, fx=scale, fy=scale, interpolation=cv2.INTER_LINEAR)

# Horizontal flip of data enhancement
def Flip_Horizontal(image):
    return cv2.flip(image, 1, dst=None)  # Horizontal mirror

# Vertical flip of data enhancement
def Flip_Vertical(image):
    return cv2.flip(image, 0, dst=None)  # mirror vertically 

# Data enhanced rotation
def Rotate(image, angle=15, scale=0.9):
    w = image.shape[1]
    h = image.shape[0]
    M = cv2.getRotationMatrix2D((w / 2, h / 2), angle, scale)  # Rotation matrix
    image = cv2.warpAffine(image, M, (w, h))   # rotate
    return image

# Darkening of data enhancement
def Darker(image, percetage=0.9):
    image_copy = image.copy()
    w = image.shape[1]
    h = image.shape[0]
    for xi in range(0, w):
        for xj in range(0, h):
            image_copy[xj, xi, 0] = int(image[xj, xi, 0] * percetage)
            image_copy[xj, xi, 1] = int(image[xj, xi, 1] * percetage)
            image_copy[xj, xi, 2] = int(image[xj, xi, 2] * percetage)
    return image_copy

# Brightening of data enhancement
def Brighter(image, percetage=1.1):
    image_copy = image.copy()
    w = image.shape[1]
    h = image.shape[0]
    for xi in range(0, w):
        for xj in range(0, h):
            image_copy[xj, xi, 0] = np.clip(int(image[xj, xi, 0] * percetage), a_max=255, a_min=0)
            image_copy[xj, xi, 1] = np.clip(int(image[xj, xi, 1] * percetage), a_max=255, a_min=0)
            image_copy[xj, xi, 2] = np.clip(int(image[xj, xi, 2] * percetage), a_max=255, a_min=0)
    return image_copy

# Data enhanced translation
def Translation(img, x, y):
    img_info = img.shape
    height = img_info[0]
    width = img_info[1]
    mat_translation = np.float32([[1, 0, x], [0, 1, y]])  # Transformation matrix: set the calculation matrix required for translation transformation: 2 rows and 3 columns (translation transformation: where x represents the translation distance in the horizontal direction and y represents the translation distance in the vertical direction)
    dst = cv2.warpAffine(img, mat_translation, (width, height))  # Transformation function
    return dst

# Increased salt and pepper noise for data enhancement
def SaltAndPepper(src, percetage):
    SP_NoiseImg = src.copy()
    SP_NoiseNum = int(percetage * src.shape[0] * src.shape[1])
    for i in range(SP_NoiseNum):
        randR = np.random.randint(0, src.shape[0] - 1)
        randG = np.random.randint(0, src.shape[1] - 1)
        randB = np.random.randint(0, 3)
        if np.random.randint(0, 1) == 0:
            SP_NoiseImg[randR, randG, randB] = 0
        else:
            SP_NoiseImg[randR, randG, randB] = 255
    return SP_NoiseImg

# Increased Gaussian noise for data enhancement
def GaussianNoise(image, percetage):
    G_Noiseimg = image.copy()
    w = image.shape[1]
    h = image.shape[0]
    G_NoiseNum = int(percetage * image.shape[0] * image.shape[1])
    for i in range(G_NoiseNum):
        temp_x = np.random.randint(0, h)
        temp_y = np.random.randint(0, w)
        G_Noiseimg[temp_x][temp_y][np.random.randint(3)] = np.random.randn(1)[0]
    return G_Noiseimg

# Increased Gaussian filtering for data enhancement
def Blur(img):
    blur = cv2.GaussianBlur(img, (7, 7), 1.5)   # CV2. Gaussian blur (image, convolution kernel, standard deviation)
    return blur

2.2 why MobileNet-v3

2.2.1 comparison with other models under the same data set  

2.2.2 introduction to mobilenet-v3

MobileNet-v3 integrates The idea of this model:

2.3 type prompt and elimination strategy implementation

These two functions are actually realized in two ways: the "type prompt" function uses crawler technology, and the data source is part of the overview of Baidu Encyclopedia entries; the "elimination strategy" function uses the traditional string return method.

Core code based on crawler technology:

def class_value(content):
    # Request address
    url = 'https://baike.baidu.com/item/' + urllib.parse.quote(content)
    # Request header
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36'
    }
    # Using request address and request header to construct request object
    req = urllib.request.Request(url=url, headers=headers, method='GET')
    # Send request and get response
    response = urllib.request.urlopen(req)
    # Read the response and get the text
    text = response.read().decode('utf-8')
    # Construct _elementobject
    html = etree.HTML(text)
    # Use xpath to match the data to get a list of matching strings
    sen_list = html.xpath('//div[contains(@class,"lemma-summary") or contains(@class,"lemmaWgt-lemmaSummary")]//text()')
    # Filter the data and remove the blank
    sen_list_after_filter = [item.strip('\n') for item in sen_list]
    # Concatenates a list of strings into strings and returns
    return ''.join(sen_list_after_filter)

2.4 demo implementation effect

performance index
Model sizespeedAccuracy raterecall F1-score
5.96 MB169ms/img91.20%83.71%0.8729

2.5 future outlook

Of course, this is only an experimental one Of course, if you want to actually land, it is very different. The biggest reason is that there are too few types of data sets, which is also a point that needs to be improved in the future

three   Design and implementation of mouse detection function based on YOLOv5

3.1 collation and acquisition of data sets

Similarly, the dataset obtains 500 common mouse pictures in daily life. The crawler tool used is the baidu picture crawler APP written by yourself mentioned above. Because the essence of this function is target detection, the Make Sense online annotation tool is used.

Make Sense online labeling tool: Make Sense

3.2 introduction to yolov5

3.3 design and implementation of rodent detection

The project is based on the design and implementation of YOLOv5   The open source project source code of Yolov 5 v5.0 on GitHub.

A comprehensive series of explanations from installation to example application   For readers interested in GitHub YOLOv5 open source code column, please move to: https://blog.csdn.net/it_charge/category_11244228.html

Final effect demonstration:

performance index
Model sizespeedAccuracy raterecall mAP_0.5
54.4 MB

69.7s/100img

(NVIDIA GeForce

GTX 1050 Ti)

93.17%91.65%0.9240

3.4 future outlook

  • Improvement in remote detection
  • Improvement in individual testing

4 design and implementation of function packaging and human-computer interaction interface based on PyQt5

4.1 PyQt5 environment installation

Install first   PyQt:

pip install pyQt5 -i https://pypi.tuna.tsinghua.edu.cn/simple
 
or
 
pip install -i https://mirror.baidu.com/pypi/simple pyQt5
 or
pip install pyQt5 -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com
 or
 Mixed use

Next install   QtDesigner:

pip install pyqt5-tools  -i https://pypi.tuna.tsinghua.edu.cn/simple
 
or
 
pip install -i https://mirror.baidu.com/pypi/simple pyqt5-tools
 or
pip install pyqt5-tools-i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com
 or
 Mixed use

Note: for readers interested in Python GUI installation rules, please move to: Python GUI quick start RongZi's blog - CSDN blog 

4.2 excerpts of key sentences

De border

self.setWindowFlags(Qt.FramelessWindowHint)

Set background image

window_pale = QtGui.QPalette()
window_pale.setBrush(self.backgroundRole(), QtGui.QBrush(QtGui.QPixmap("background.jpg"))) 
self.setPalette(window_pale)

  Set pop-up window

QMessageBox.information(self, "i 'm sorry", "This function is under emergency repair...")

Override mouse events

# Override mouse movement events
def mouseMoveEvent(self, e: QMouseEvent):
    if self._tracking:
        self._endPos = e.pos() - self._startPos
        self.move(self.pos() + self._endPos)
        self.setCursor(QCursor(Qt.SizeAllCursor))
# Override mouse press event
def mousePressEvent(self, e: QMouseEvent):
    if e.button() == Qt.LeftButton:
        self._startPos = QPoint(e.x(), e.y())
        self._tracking = True
# Override mouse lift event
def mouseReleaseEvent(self, e: QMouseEvent):
    if e.button() == Qt.LeftButton:
        self._tracking = False
        self._startPos = None
        self._endPos = None
        self.setCursor(QCursor(Qt.ArrowCursor))

display picture

self.classlabel = QLabel(self)
self.classlabel.setFixedSize(100, 100)
self.classlabel.move(100, 100)
self.classlabel.setPixmap(QPixmap("image.png"))
self.classlabel.setScaledContents(True)

display text

self.classlabel = QLabel(self)
self.classlabel.setText("Text content")
self.classlabel.move(100, 100)
self.classlabel.setStyleSheet(
        "QLabel{color:black;font-size:15px;font-weight:bold;font-family:KaiTi;}"
 )

simulation   The system overrides the minimize, shrink interface, and close buttons

# Red button: overrides the close event
self.button_red = QPushButton(self)
self.button_red.move(900, 11)
self.button_red.setFixedSize(18, 18)
self.button_red.setStyleSheet("QPushButton{background:#CE0000;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}"
                              "QPushButton:hover{background:red;}"
                              "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}")
self.button_red.clicked.connect(self.quit_button)
# Yellow button: overrides the zoom out event
self.button_orange = QPushButton(self)
self.button_orange.move(865, 11)
self.button_orange.setFixedSize(18, 18)
self.button_orange.setStyleSheet("QPushButton{background:orange;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}"
                                 "QPushButton:hover{background:#FFD306}"
                                 "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}")
# Green button: override minimize event
self.button_green = QPushButton(self)
self.button_green.move(830, 11)
self.button_green.setFixedSize(18, 18)
self.button_green.setStyleSheet("QPushButton{background:green;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}"
                                "QPushButton:hover{background:#08BF14}"
                                "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}")

def quit_button(self):
        quit()

Realize picture rotation

    self.classlabel = QLabel(self)
    self.classlabel.setFixedSize(777, 333)
    self.classlabel.move(100, 222)
    self.classlabel.setPixmap(QPixmap("image.jpg"))
    self.classlabel.setScaledContents(True)
    global lu
    self.n = 1
    self.lu = "./images/" + str(self.n) + ".jpg"
    self.pm = QPixmap(self.lu)
    self.lbpic = myLabel(self)
    self.lbpic.setPixmap(self.pm)
    self.lbpic.resize(777, 333)
    self.lbpic.move(100, 222)
    self.lbpic.setScaledContents(True)
    self.lbpic._signal.connect(self.callbacklog)  # joining signal
    self.timer1 = QTimer(self)
    self.timer1.timeout.connect(self.timer_TimeOut)
    self.timer1.start(2000)
    self.show()

def timer_TimeOut(self):
    self.n += 1
    if self.n > 3:
        self.n = 1
    self.lu = "./images/" + str(self.n) + ".jpg"
    self.pm = QPixmap(self.lu)
    self.lbpic.setPixmap(self.pm)

def callbacklog(self, msg):
    from PIL import Image
    import matplotlib.pyplot as plt
    img = Image.open(self.lu)
    plt.figure("image")
    plt.imshow(img)
    plt.show()

  Implement button selection

self.radioButton_1 = QtWidgets.QRadioButton(self)
self.radioButton_1.setGeometry(QtCore.QRect(100, 100, 100, 100))
self.radioButton_1.setStyleSheet("color:black;font-size:18px;font-weight:bold;font-family:KaiTi;")
self.radioButton_1.setObjectName("radioButton_1")
self.radioButton_2 = QtWidgets.QRadioButton(self)
self.radioButton_2.setGeometry(QtCore.QRect(100, 100, 100, 100))
self.radioButton_2.setStyleSheet("color:black;font-size:18px;font-weight:bold;font-family:KaiTi;")
self.radioButton_2.setObjectName("radioButton_2")
translate = QtCore.QCoreApplication.translate
self.radioButton_1.setText(translate("Form", "Option one"))
self.radioButton_1.setChecked(True)
self.radioButton_2.setText(translate("Form", "Option 2"))

Implement drop-down list selection

self.cb1 = QComboBox(self)
self.cb1.move(503, 181)
self.cb1.addItems(['Option 1'])
self.cb2 = QComboBox(self)
self.cb2.move(606, 181)
self.cb2.addItems(['Option 2.1', 'Option 2.2', 'Option 2.3'])
self.cb3 = QComboBox(self)
self.cb3.move(693, 181)
self.cb3.addItems(['Option 3.1', 'Option 3.2', 'Option 3.3', 'Option 3.4', 'Option 3.5'])

  Select a local folder to export pictures

def openimage(self):
    self.imgName, self.imgType = QFileDialog.getOpenFileName(self, 'Select Picture', '.', 'image file(*.jpg)')
    jpg_img = QtGui.QPixmap(self.imgName).scaled(self.imglabel.width(), self.imglabel.height())
    self.imglabel.setPixmap(jpg_img)

  Select a local folder to export video

def button_video_open(self):
    video_name, _ = QtWidgets.QFileDialog.getOpenFileName(self, "Open video", "", "*.mp4;;*.avi;;All Files(*)")
    flag = self.cap.open(video_name)
    if flag == False:
        QtWidgets.QMessageBox.warning(self, u"Warning", u"Failed to open video", buttons=QtWidgets.QMessageBox.Ok,
                                      defaultButton=QtWidgets.QMessageBox.Ok)
    else:
        self.out = cv2.VideoWriter('prediction.avi', cv2.VideoWriter_fourcc(*'MJPG'), 20,
                                   (int(self.cap.get(3)), int(self.cap.get(4))))
        self.timer_video.start(30)
        self.pushButton_video.setDisabled(True)
        self.pushButton_img.setDisabled(True)
        self.pushButton_camera.setDisabled(True)

  Achieve camera display effect

def button_camera_open(self):
    if not self.timer_video.isActive():
        # The first local camera is used by default
        flag = self.cap.open(0)
        if flag == False:
            QtWidgets.QMessageBox.warning(self, u"Warning", u"Failed to open camera", buttons=QtWidgets.QMessageBox.Ok,
                                          defaultButton=QtWidgets.QMessageBox.Ok)
        else:
            self.out = cv2.VideoWriter('prediction.avi', cv2.VideoWriter_fourcc(*'MJPG'), 20,
                                       (int(self.cap.get(3)), int(self.cap.get(4))))
            self.timer_video.start(30)
            self.pushButton_video.setDisabled(True)
            self.pushButton_img.setDisabled(True)
            self.pushButton_camera.setText(u"Turn off the camera")
    else:
        self.timer_video.stop()
        self.cap.release()
        self.out.release()
        self.label.clear()
        self.init_logo()
        self.pushButton_video.setDisabled(False)
        self.pushButton_img.setDisabled(False)
        self.pushButton_camera.setText(u"Camera detection")

Insert web pages into the implementation interface

# Create a new QWebEngineView() object
self.qwebengine = QWebEngineView(self)
# Set the position and size of the web page displayed in the window
self.qwebengine.setGeometry(100, 100, 100, 100)
# Load web address in QWebEngineView
self.qwebengine.load(QUrl("http://www.baidu.com"))

4.3 future outlook

  •   Connect with the database to realize the user registration function, making the project closer to the actual landing expectation.

5 main references

[1] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. arXiv preprint arXiv:1506.02640, 2015.

[2] Joseph Redmon, Ali Farhadi. YOLO9000:Better, Faster, Stronger. arXiv preprint arXiv:1612.08242v1, 2015.

[3] J. Redmon and A. Farhadi. Yolov3: An incremental improve_x0002_ment. arXiv, 2018.

[4] Alexey Bochkovskiy, Chien-Yao Wang and Hong-Yuan Mark Liao. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv, 2020.

[5] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, Hartwig Adam.Searching for MobileNetV3. arXiv, 2019.

Copyright notice: This article is CSDN Blogger「RongZi! The most beautiful baby!」Original articles, follow CC 4.0 BY-SA Copyright agreement.
         For reprint, please attach the original source link and this statement in an eye-catching position.

Links to other original columns of CSDN blogger "RongZi! The most beautiful!"
Learning notes column of machine learning: https://blog.csdn.net/it_charge/category_9920949.html 
Learning notes column of digital image processing: https://blog.csdn.net/it_charge/category_9899861.html 
Python Network Data crawling and analysis column: https://blog.csdn.net/it_charge/category_10097766.html
GitHub YOLOv5 open source code project series explanation column: https://blog.csdn.net/it_charge/category_11244228.html
Column of "in depth learning over and over" required articles: https://blog.csdn.net/it_charge/category_11251478.html
Application of 23 design patterns in the glory of the king column: https://blog.csdn.net/it_charge/category_9842143.html

Address of blogger station B: Personal space of RongZi python small class_ Beep beep beep_ Bilibili

Thanks for reading!   Thank you for your support!   Thanks for your attention!

Welcome to exchange comments and learn together!

  year    month    Chinese mainland Chongqing




END

Tags: Python AI Computer Vision

Posted on Sat, 23 Oct 2021 20:51:27 -0400 by rushenas