Python screen capture tool to identify the QR code in the screen

How to quickly identify QR codes when looking at web pages? Most people may take out their cell phones. There are two problems in mobile phone recognition: 1. Facing the screen, there will be interference of moire pattern. 2. The recognition results cannot be used immediately on the computer. If you make a screenshot gadget to identify, it will be much more convenient.

Install Python dependency package

pip install pillow opencv-python dbr pyside2

Screen shot to identify QR code

  1. Build UI with Qt
  2. Create a custom full screen Qt widget
  3. Listen for mouse events and get the area
  4. Use the interface in PIL to capture the picture according to the coordinates
  5. call Dynamsoft Barcode Reader Interface identification QR code

code implementation

Open the UI editor (path: Python 37 \ lib \ site packages \ pyside2 \ designer. Exe) to create the interface.

  • The two buttons are used for area screenshots and full screen screenshots respectively
  • The middle area is the QLabel for displaying pictures

Compile the UI into a python file:

pyside2-uic design.ui -o design.py

Import UI into python project:

from design import Ui_MainWindow

class MainWindow(QMainWindow):

    def __init__(self, license):
        super(MainWindow, self).__init__()
        self.setWindowState(Qt.WindowMaximized)
        self.ui = Ui_MainWindow()
        self.ui.setupUi(self)
        self.setAcceptDrops(True)

Create a new SnippingTool.py file. Create a custom Qt Widget:

import numpy as np
import cv2
from PIL import ImageGrab
from PySide2 import QtWidgets, QtCore, QtGui
from PySide2.QtCore import Qt

class SnippingWidget(QtWidgets.QWidget):
    is_snipping = False

    def __init__(self, parent=None, app=None):
        super(SnippingWidget, self).__init__()
        self.parent = parent
        self.setWindowFlags(Qt.WindowStaysOnTopHint)

        self.screen = app.primaryScreen()
        self.setGeometry(0, 0, self.screen.size().width(), self.screen.size().height())
        self.begin = QtCore.QPoint()
        self.end = QtCore.QPoint()
        self.onSnippingCompleted = None

    def fullscreen(self):
        img = ImageGrab.grab(bbox=(0, 0, self.screen.size().width(), self.screen.size().height()))

        try:
            img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
        except:
            img = None
            
        if self.onSnippingCompleted is not None:
            self.onSnippingCompleted(img)

    def start(self):
        SnippingWidget.is_snipping = True
        self.setWindowOpacity(0.3)
        QtWidgets.QApplication.setOverrideCursor(QtGui.QCursor(QtCore.Qt.CrossCursor))
        self.show()

    def paintEvent(self, event):
        if SnippingWidget.is_snipping:
            brush_color = (128, 128, 255, 100)
            lw = 3
            opacity = 0.3
        else:
            self.begin = QtCore.QPoint()
            self.end = QtCore.QPoint()
            brush_color = (0, 0, 0, 0)
            lw = 0
            opacity = 0

        self.setWindowOpacity(opacity)
        qp = QtGui.QPainter(self)
        qp.setPen(QtGui.QPen(QtGui.QColor('black'), lw))
        qp.setBrush(QtGui.QColor(*brush_color))
        rect = QtCore.QRectF(self.begin, self.end)
        qp.drawRect(rect)

    def mousePressEvent(self, event):
        self.begin = event.pos()
        self.end = self.begin
        self.update()

    def mouseMoveEvent(self, event):
        self.end = event.pos()
        self.update()

    def mouseReleaseEvent(self, event):
        SnippingWidget.is_snipping = False
        QtWidgets.QApplication.restoreOverrideCursor()
        x1 = min(self.begin.x(), self.end.x())
        y1 = min(self.begin.y(), self.end.y())
        x2 = max(self.begin.x(), self.end.x())
        y2 = max(self.begin.y(), self.end.y())

        self.repaint()
        QtWidgets.QApplication.processEvents()
        img = ImageGrab.grab(bbox=(x1, y1, x2, y2))

        try:
            img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
        except:
            img = None
            
        if self.onSnippingCompleted is not None:
            self.onSnippingCompleted(img)

        self.close()

During initialization, we set the widget to full screen size:

self.screen = app.primaryScreen()
self.setGeometry(0, 0, self.screen.size().width(), self.screen.size().height())

The widget is displayed when the start() function is called. At this time, paintEvent() will be triggered, and we will draw a screenshot in it.

By monitoring mouse events, we can obtain the coordinates when clicking and releasing:

    def mousePressEvent(self, event):
        self.begin = event.pos()
        self.end = self.begin
        self.update()

    def mouseMoveEvent(self, event):
        self.end = event.pos()
        self.update()

    def mouseReleaseEvent(self, event):
        SnippingWidget.is_snipping = False
        QtWidgets.QApplication.restoreOverrideCursor()
        x1 = min(self.begin.x(), self.end.x())
        y1 = min(self.begin.y(), self.end.y())
        x2 = max(self.begin.x(), self.end.x())
        y2 = max(self.begin.y(), self.end.y())

        self.repaint()
        QtWidgets.QApplication.processEvents()

Finally, use ImageGrab.grab(bbox=(x1, y1, x2, y2)) to obtain a screenshot of the screen. The coordinates of the full screen are img = ImageGrab.grab(bbox=(0, 0, self.screen.size().width(), self.screen.size().height())).

The obtained image is transferred to the callback function:

        try:
            img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR)
        except:
            img = None
            
        if self.onSnippingCompleted is not None:
            self.onSnippingCompleted(img)

In the main function, initialize the button and set the callback function. When the screen capture is triggered, we minimize the window so that the screen will not be obscured. Restore the program window after obtaining the image:

self.snippingWidget = SnippingTool.SnippingWidget(app=QApplication.instance())
self.snippingWidget.onSnippingCompleted = self.onSnippingCompleted
self.ui.pushButton_area.clicked.connect(self.snipArea)
self.ui.pushButton_full.clicked.connect(self.snipFull)

def snipArea(self):
    self.setWindowState(Qt.WindowMinimized)
    self.snippingWidget.start()    

def snipFull(self):
    self.setWindowState(Qt.WindowMinimized)
    self.snippingWidget.fullscreen()    

After the callback function is triggered, the image data is transmitted to the SDK for decoding. Finally, draw the results on QLabel:

    def onSnippingCompleted(self, frame):
        self.setWindowState(Qt.WindowMaximized)
        if frame is None:
            return 

        frame, self._results = self._barcodeManager.decode_frame(frame)
        self.showResults(frame, self._results)

Find a batch of QR code images in the search engine for testing:

video

Desktop QR code recognition

Source code

https://github.com/yushulx/python-gui-barcode-reader

Tags: Python Qt

Posted on Thu, 14 Oct 2021 03:02:36 -0400 by austine_power007