In the context of the new era, "refreshing and clean" and "health" have become two important factors for people's yearning for a better life, which is also the constant pursuit and goal of current catering industry managers. With the development and progress of science and technology, artificial intelligence has become an important product of civilized society and is gradually applied to all aspects of daily life. Based on this, it is very necessary to design and develop a system that can effectively prevent pests and rats to improve management efficiency and health service quality
This paper mainly includes:
- be based on Design and implementation of insect and mouse recognition function of MobileNet-v3
- Design and implementation of mouse detection function based on YOLOv5
- Design and implementation of function packaging and human-computer interaction interface based on PyQt5
catalogue
1. Finished product effect demonstration
two be based on Design and implementation of insect and mouse recognition function of MobileNet-v3
three Design and implementation of mouse detection function based on YOLOv5
1. Finished product effect demonstration
1.1 picture finished product effect demonstration
1.2 video finished product effect demonstration
two be based on Design and implementation of insect and mouse recognition function of MobileNet-v3
2.1 acquisition and sorting of data sets
2.1.1 data acquisition
As we all know, data acquisition is an essential skill in the field of deep learning. There are various data acquisition methods, specifically: ① find the public data sets related to the task (such as COCO data sets and ImageNet data sets used for image recognition); ② Use the outsourcing platform to obtain data (such as Alibaba crowdsourcing, baidu data crowdsourcing, JD micro industry, etc.) ③ of course, you can also collect data with your own camera as needed; ④ The last way is to use web crawler technology.
Here, the fourth data acquisition method - web crawler is used. Used is a Baidu image crawler APP written by myself before.
For readers interested in the baidu image crawler APP, please move to:
First blog post: When the crawler encounters PyQt5: GUI interface, baidu image crawling is realized
Get GitHub source code: PyQt5/reptile at main · zhao302014/PyQt5 · GitHub
Using the crawler tool written by myself, a total of There are 50 data sets of Five insect and mouse categories, including ant, cockroach, fly, mouse and pillworm, with a total of 250.
2.1.2 data sorting
Data collation can mainly consider three levels: data inspection and normalization, data De duplication and data set division.
Obviously, the data set obtained in the previous step is too small in both component and total, which is easy to cause over fitting due to too small data set or too large model. Therefore, the simplest and rough method to prevent over fitting is adopted here: implicit regularization method - data enhancement. Specifically, data enhancement methods including Scale, Horizontal, Rotate, Darker, brightener, Translation and AddNoise are used (implemented based on relevant methods of OpenCV)
# Scaling operation of data enhancement def Scale(image, scale): return cv2.resize(image, None, fx=scale, fy=scale, interpolation=cv2.INTER_LINEAR) # Horizontal flip of data enhancement def Flip_Horizontal(image): return cv2.flip(image, 1, dst=None) # Horizontal mirror # Vertical flip of data enhancement def Flip_Vertical(image): return cv2.flip(image, 0, dst=None) # mirror vertically # Data enhanced rotation def Rotate(image, angle=15, scale=0.9): w = image.shape[1] h = image.shape[0] M = cv2.getRotationMatrix2D((w / 2, h / 2), angle, scale) # Rotation matrix image = cv2.warpAffine(image, M, (w, h)) # rotate return image # Darkening of data enhancement def Darker(image, percetage=0.9): image_copy = image.copy() w = image.shape[1] h = image.shape[0] for xi in range(0, w): for xj in range(0, h): image_copy[xj, xi, 0] = int(image[xj, xi, 0] * percetage) image_copy[xj, xi, 1] = int(image[xj, xi, 1] * percetage) image_copy[xj, xi, 2] = int(image[xj, xi, 2] * percetage) return image_copy # Brightening of data enhancement def Brighter(image, percetage=1.1): image_copy = image.copy() w = image.shape[1] h = image.shape[0] for xi in range(0, w): for xj in range(0, h): image_copy[xj, xi, 0] = np.clip(int(image[xj, xi, 0] * percetage), a_max=255, a_min=0) image_copy[xj, xi, 1] = np.clip(int(image[xj, xi, 1] * percetage), a_max=255, a_min=0) image_copy[xj, xi, 2] = np.clip(int(image[xj, xi, 2] * percetage), a_max=255, a_min=0) return image_copy # Data enhanced translation def Translation(img, x, y): img_info = img.shape height = img_info[0] width = img_info[1] mat_translation = np.float32([[1, 0, x], [0, 1, y]]) # Transformation matrix: set the calculation matrix required for translation transformation: 2 rows and 3 columns (translation transformation: where x represents the translation distance in the horizontal direction and y represents the translation distance in the vertical direction) dst = cv2.warpAffine(img, mat_translation, (width, height)) # Transformation function return dst # Increased salt and pepper noise for data enhancement def SaltAndPepper(src, percetage): SP_NoiseImg = src.copy() SP_NoiseNum = int(percetage * src.shape[0] * src.shape[1]) for i in range(SP_NoiseNum): randR = np.random.randint(0, src.shape[0] - 1) randG = np.random.randint(0, src.shape[1] - 1) randB = np.random.randint(0, 3) if np.random.randint(0, 1) == 0: SP_NoiseImg[randR, randG, randB] = 0 else: SP_NoiseImg[randR, randG, randB] = 255 return SP_NoiseImg # Increased Gaussian noise for data enhancement def GaussianNoise(image, percetage): G_Noiseimg = image.copy() w = image.shape[1] h = image.shape[0] G_NoiseNum = int(percetage * image.shape[0] * image.shape[1]) for i in range(G_NoiseNum): temp_x = np.random.randint(0, h) temp_y = np.random.randint(0, w) G_Noiseimg[temp_x][temp_y][np.random.randint(3)] = np.random.randn(1)[0] return G_Noiseimg # Increased Gaussian filtering for data enhancement def Blur(img): blur = cv2.GaussianBlur(img, (7, 7), 1.5) # CV2. Gaussian blur (image, convolution kernel, standard deviation) return blur
2.2 why MobileNet-v3
2.2.1 comparison with other models under the same data set
2.2.2 introduction to mobilenet-v3
MobileNet-v3 integrates The idea of this model:
2.3 type prompt and elimination strategy implementation
These two functions are actually realized in two ways: the "type prompt" function uses crawler technology, and the data source is part of the overview of Baidu Encyclopedia entries; the "elimination strategy" function uses the traditional string return method.
Core code based on crawler technology:
def class_value(content): # Request address url = 'https://baike.baidu.com/item/' + urllib.parse.quote(content) # Request header headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36' } # Using request address and request header to construct request object req = urllib.request.Request(url=url, headers=headers, method='GET') # Send request and get response response = urllib.request.urlopen(req) # Read the response and get the text text = response.read().decode('utf-8') # Construct _elementobject html = etree.HTML(text) # Use xpath to match the data to get a list of matching strings sen_list = html.xpath('//div[contains(@class,"lemma-summary") or contains(@class,"lemmaWgt-lemmaSummary")]//text()') # Filter the data and remove the blank sen_list_after_filter = [item.strip('\n') for item in sen_list] # Concatenates a list of strings into strings and returns return ''.join(sen_list_after_filter)
2.4 demo implementation effect
Model size | speed | Accuracy rate | recall | F1-score |
5.96 MB | 169ms/img | 91.20% | 83.71% | 0.8729 |
2.5 future outlook
Of course, this is only an experimental one Of course, if you want to actually land, it is very different. The biggest reason is that there are too few types of data sets, which is also a point that needs to be improved in the future
three Design and implementation of mouse detection function based on YOLOv5
3.1 collation and acquisition of data sets
Similarly, the dataset obtains 500 common mouse pictures in daily life. The crawler tool used is the baidu picture crawler APP written by yourself mentioned above. Because the essence of this function is target detection, the Make Sense online annotation tool is used.

3.2 introduction to yolov5
3.3 design and implementation of rodent detection
The project is based on the design and implementation of YOLOv5 The open source project source code of Yolov 5 v5.0 on GitHub.
A comprehensive series of explanations from installation to example application For readers interested in GitHub YOLOv5 open source code column, please move to: https://blog.csdn.net/it_charge/category_11244228.html
Final effect demonstration:
Model size | speed | Accuracy rate | recall | mAP_0.5 |
54.4 MB | 69.7s/100img (NVIDIA GeForce GTX 1050 Ti) | 93.17% | 91.65% | 0.9240 |
3.4 future outlook
- Improvement in remote detection
- Improvement in individual testing
4 design and implementation of function packaging and human-computer interaction interface based on PyQt5
4.1 PyQt5 environment installation
Install first PyQt:
pip install pyQt5 -i https://pypi.tuna.tsinghua.edu.cn/simple or pip install -i https://mirror.baidu.com/pypi/simple pyQt5 or pip install pyQt5 -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com or Mixed use
Next install QtDesigner:
pip install pyqt5-tools -i https://pypi.tuna.tsinghua.edu.cn/simple or pip install -i https://mirror.baidu.com/pypi/simple pyqt5-tools or pip install pyqt5-tools-i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com or Mixed use
Note: for readers interested in Python GUI installation rules, please move to: Python GUI quick start RongZi's blog - CSDN blog
4.2 excerpts of key sentences
De border
self.setWindowFlags(Qt.FramelessWindowHint)
Set background image
window_pale = QtGui.QPalette() window_pale.setBrush(self.backgroundRole(), QtGui.QBrush(QtGui.QPixmap("background.jpg"))) self.setPalette(window_pale)
Set pop-up window
QMessageBox.information(self, "i 'm sorry", "This function is under emergency repair...")
Override mouse events
# Override mouse movement events def mouseMoveEvent(self, e: QMouseEvent): if self._tracking: self._endPos = e.pos() - self._startPos self.move(self.pos() + self._endPos) self.setCursor(QCursor(Qt.SizeAllCursor)) # Override mouse press event def mousePressEvent(self, e: QMouseEvent): if e.button() == Qt.LeftButton: self._startPos = QPoint(e.x(), e.y()) self._tracking = True # Override mouse lift event def mouseReleaseEvent(self, e: QMouseEvent): if e.button() == Qt.LeftButton: self._tracking = False self._startPos = None self._endPos = None self.setCursor(QCursor(Qt.ArrowCursor))
display picture
self.classlabel = QLabel(self) self.classlabel.setFixedSize(100, 100) self.classlabel.move(100, 100) self.classlabel.setPixmap(QPixmap("image.png")) self.classlabel.setScaledContents(True)
display text
self.classlabel = QLabel(self) self.classlabel.setText("Text content") self.classlabel.move(100, 100) self.classlabel.setStyleSheet( "QLabel{color:black;font-size:15px;font-weight:bold;font-family:KaiTi;}" )
simulation The system overrides the minimize, shrink interface, and close buttons
# Red button: overrides the close event self.button_red = QPushButton(self) self.button_red.move(900, 11) self.button_red.setFixedSize(18, 18) self.button_red.setStyleSheet("QPushButton{background:#CE0000;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}" "QPushButton:hover{background:red;}" "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}") self.button_red.clicked.connect(self.quit_button) # Yellow button: overrides the zoom out event self.button_orange = QPushButton(self) self.button_orange.move(865, 11) self.button_orange.setFixedSize(18, 18) self.button_orange.setStyleSheet("QPushButton{background:orange;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}" "QPushButton:hover{background:#FFD306}" "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}") # Green button: override minimize event self.button_green = QPushButton(self) self.button_green.move(830, 11) self.button_green.setFixedSize(18, 18) self.button_green.setStyleSheet("QPushButton{background:green;color:white;box-shadow: 1px 1px 3px;border-radius: 9px}" "QPushButton:hover{background:#08BF14}" "QPushButton:pressed{border: 1px solid #3C3C3C!important;background:black}") def quit_button(self): quit()
Realize picture rotation
self.classlabel = QLabel(self) self.classlabel.setFixedSize(777, 333) self.classlabel.move(100, 222) self.classlabel.setPixmap(QPixmap("image.jpg")) self.classlabel.setScaledContents(True) global lu self.n = 1 self.lu = "./images/" + str(self.n) + ".jpg" self.pm = QPixmap(self.lu) self.lbpic = myLabel(self) self.lbpic.setPixmap(self.pm) self.lbpic.resize(777, 333) self.lbpic.move(100, 222) self.lbpic.setScaledContents(True) self.lbpic._signal.connect(self.callbacklog) # joining signal self.timer1 = QTimer(self) self.timer1.timeout.connect(self.timer_TimeOut) self.timer1.start(2000) self.show() def timer_TimeOut(self): self.n += 1 if self.n > 3: self.n = 1 self.lu = "./images/" + str(self.n) + ".jpg" self.pm = QPixmap(self.lu) self.lbpic.setPixmap(self.pm) def callbacklog(self, msg): from PIL import Image import matplotlib.pyplot as plt img = Image.open(self.lu) plt.figure("image") plt.imshow(img) plt.show()
Implement button selection
self.radioButton_1 = QtWidgets.QRadioButton(self) self.radioButton_1.setGeometry(QtCore.QRect(100, 100, 100, 100)) self.radioButton_1.setStyleSheet("color:black;font-size:18px;font-weight:bold;font-family:KaiTi;") self.radioButton_1.setObjectName("radioButton_1") self.radioButton_2 = QtWidgets.QRadioButton(self) self.radioButton_2.setGeometry(QtCore.QRect(100, 100, 100, 100)) self.radioButton_2.setStyleSheet("color:black;font-size:18px;font-weight:bold;font-family:KaiTi;") self.radioButton_2.setObjectName("radioButton_2") translate = QtCore.QCoreApplication.translate self.radioButton_1.setText(translate("Form", "Option one")) self.radioButton_1.setChecked(True) self.radioButton_2.setText(translate("Form", "Option 2"))
Implement drop-down list selection
self.cb1 = QComboBox(self) self.cb1.move(503, 181) self.cb1.addItems(['Option 1']) self.cb2 = QComboBox(self) self.cb2.move(606, 181) self.cb2.addItems(['Option 2.1', 'Option 2.2', 'Option 2.3']) self.cb3 = QComboBox(self) self.cb3.move(693, 181) self.cb3.addItems(['Option 3.1', 'Option 3.2', 'Option 3.3', 'Option 3.4', 'Option 3.5'])
Select a local folder to export pictures
def openimage(self): self.imgName, self.imgType = QFileDialog.getOpenFileName(self, 'Select Picture', '.', 'image file(*.jpg)') jpg_img = QtGui.QPixmap(self.imgName).scaled(self.imglabel.width(), self.imglabel.height()) self.imglabel.setPixmap(jpg_img)
Select a local folder to export video
def button_video_open(self): video_name, _ = QtWidgets.QFileDialog.getOpenFileName(self, "Open video", "", "*.mp4;;*.avi;;All Files(*)") flag = self.cap.open(video_name) if flag == False: QtWidgets.QMessageBox.warning(self, u"Warning", u"Failed to open video", buttons=QtWidgets.QMessageBox.Ok, defaultButton=QtWidgets.QMessageBox.Ok) else: self.out = cv2.VideoWriter('prediction.avi', cv2.VideoWriter_fourcc(*'MJPG'), 20, (int(self.cap.get(3)), int(self.cap.get(4)))) self.timer_video.start(30) self.pushButton_video.setDisabled(True) self.pushButton_img.setDisabled(True) self.pushButton_camera.setDisabled(True)
Achieve camera display effect
def button_camera_open(self): if not self.timer_video.isActive(): # The first local camera is used by default flag = self.cap.open(0) if flag == False: QtWidgets.QMessageBox.warning(self, u"Warning", u"Failed to open camera", buttons=QtWidgets.QMessageBox.Ok, defaultButton=QtWidgets.QMessageBox.Ok) else: self.out = cv2.VideoWriter('prediction.avi', cv2.VideoWriter_fourcc(*'MJPG'), 20, (int(self.cap.get(3)), int(self.cap.get(4)))) self.timer_video.start(30) self.pushButton_video.setDisabled(True) self.pushButton_img.setDisabled(True) self.pushButton_camera.setText(u"Turn off the camera") else: self.timer_video.stop() self.cap.release() self.out.release() self.label.clear() self.init_logo() self.pushButton_video.setDisabled(False) self.pushButton_img.setDisabled(False) self.pushButton_camera.setText(u"Camera detection")
Insert web pages into the implementation interface
# Create a new QWebEngineView() object self.qwebengine = QWebEngineView(self) # Set the position and size of the web page displayed in the window self.qwebengine.setGeometry(100, 100, 100, 100) # Load web address in QWebEngineView self.qwebengine.load(QUrl("http://www.baidu.com"))
4.3 future outlook
- Connect with the database to realize the user registration function, making the project closer to the actual landing expectation.
5 main references
[1] J. Redmon, S. Divvala, R. Girshick, and A. Farhadi. You only look once: Unified, real-time object detection. arXiv preprint arXiv:1506.02640, 2015.
[2] Joseph Redmon, Ali Farhadi. YOLO9000:Better, Faster, Stronger. arXiv preprint arXiv:1612.08242v1, 2015.
[3] J. Redmon and A. Farhadi. Yolov3: An incremental improve_x0002_ment. arXiv, 2018.
[4] Alexey Bochkovskiy, Chien-Yao Wang and Hong-Yuan Mark Liao. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv, 2020.
[5] Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, Hartwig Adam.Searching for MobileNetV3. arXiv, 2019.
Copyright notice: This article is CSDN Blogger「RongZi! The most beautiful baby!」Original articles, follow CC 4.0 BY-SA Copyright agreement. For reprint, please attach the original source link and this statement in an eye-catching position.
Links to other original columns of CSDN blogger "RongZi! The most beautiful!"
Learning notes column of machine learning: https://blog.csdn.net/it_charge/category_9920949.html
Learning notes column of digital image processing: https://blog.csdn.net/it_charge/category_9899861.html
Python Network Data crawling and analysis column: https://blog.csdn.net/it_charge/category_10097766.html
GitHub YOLOv5 open source code project series explanation column: https://blog.csdn.net/it_charge/category_11244228.html
Column of "in depth learning over and over" required articles: https://blog.csdn.net/it_charge/category_11251478.html
Application of 23 design patterns in the glory of the king column: https://blog.csdn.net/it_charge/category_9842143.html
Address of blogger station B: Personal space of RongZi python small class_ Beep beep beep_ Bilibili
Thanks for reading! Thank you for your support! Thanks for your attention!
Welcome to exchange comments and learn together!
year
month
Chinese mainland Chongqing