Tired of playing games? Paddle gesture recognition play a new game!

PaddlePaddle realizes gesture recognition, playing and eating beans!

Article directory:

1. Gesture data collection

2. PaddleX training model

3. Test gesture recognition model

4. Test game gesture control

5. Success~

# Unzip code
!unzip /home/aistudio/data/data41298/code.zip -d /home/aistudio/work/
!pip install paddlex

Fist means go down:

Palms indicate walking up:

The following two are left and right:

Blank means fixed bit by bit:

# Set work path
import os

1. Gesture data collection:

This step requires running the collection folder locally PalmTracker.py Hand gesture data collection is carried out for files;

When running the program, the camera will be turned on, make gestures in the designated area, and press s to save;

# !python collect/PalmTracker.py
collect    data     game.py  pacman.py	test.jpg  utils.py
config.py  demo.py  images   src	tools	  weights

2. PaddleX training model

This step uses ResNet18 provided by PaddleX for training;

The pre training model uses the weight of training on 'IMAGENET', and PaddleX selects the parameter pretrain_weights = IMAGENET;

I have collected about 40 pieces of each gesture here, and the accuracy of the training results is more than 93%;

2.1 defining data sets

from paddlex.cls import transforms
import os
import cv2
import numpy as np
import paddlex as pdx

base = './data'

with open(os.path.join('train_list.txt'), 'w') as f:
    for i, cls_fold in enumerate(os.listdir(base)):
        cls_base = os.path.join(base, cls_fold)
        files = os.listdir(cls_base)
        print('{} train num:'.format(cls_fold), len(files))
        for pt in files:
            img = os.path.join(cls_fold, pt)
            info = img + ' ' + str(i) + '\n'

with open(os.path.join('labels.txt'), 'w') as f:
    for i, cls_fold in enumerate(os.listdir(base)):

train_transforms = transforms.Compose([

train_dataset = pdx.datasets.ImageNet(

2.2 using ResNet18 training model

20 epoch s are trained here, and the initial learning rate is 2e-2

num_classes = len(train_dataset.labels)
model = pdx.cls.ResNet18(num_classes=num_classes)
            lr_decay_epochs=[5, 10, 15],

3 test gesture recognition model:

from paddlex.cls import transforms
import matplotlib.pyplot as plt
import paddlex
import cv2
import warnings


train_transforms = transforms.Compose([

model = paddlex.load_model('weights/final')
im = cv2.imread('test.jpg')
result = model.predict(im, topk=1, transforms=train_transforms)
print("Predict Result:", result)

%matplotlib inline
("Predict Result:", result)

%matplotlib inline
2020-06-23 09:27:29 [INFO]	Model[ResNet18] loaded.
Predict Result: [{'category_id': 1, 'category': 'left', 'score': 0.9999609}]

[failed to transfer the pictures in the external link. The source station may have anti-theft chain mechanism. It is recommended to save the pictures and upload them directly (img-xDGN7LG1-1592877702883)(output_13_1.png)]

4. Test gesture control in the game:

Local operation demo.py That's right;

!python demo.py

5. Success

Then embed the control into the game~

Game code from: https://github.com/hbokmann/Pacman

!python game.py

I put the demo video on Youtube

Link address: https://youtu.be/tlZT2WeaK1U

Update, station B approved!

Link address: https://www.bilibili.com/video/BV1xa4y1Y7Mb/

About the author:

Sophomore of Beijing University of Technology

Interested directions: target detection, face recognition, EEG recognition, etc

You are also welcome to fork, comment and exchange

Author blog home page: https://blog.csdn.net/weixin_44936889

contact us:

If you need weight files or source codes, and alchemists who share the same interests with AI, please privately stamp the author~

Please note AI fans when contacting me, I will pull you into the communication group~

The group will regularly share various source codes and models. Please download the previously shared ones from the group file~

Tags: Python pip github

Posted on Tue, 23 Jun 2020 00:00:54 -0400 by ac1dsp3ctrum