AI Creation Camp: Etiquette robot C-3PO

Etiquette robot C-3PO: help your Star Trek!

In 1970, the first man-made satellite "Dongfanghong 1" was successfully launched!

In 2003, Shenzhou V manned spacecraft successfully launched!

In 2007, Chang'e-1 successfully flew to the moon!

In 2008, Shenzhou VII manned spacecraft realized the astronauts' extravehicular flight

2011 Shenzhou 8 "hand in hand" Tiangong 1 (automatic rendezvous and docking)

In 2013, Shenzhou 10 conducted its first space teaching

In 2016, Shenzhou 11 realized the medium-term manned presence (33 days of space life)

In 2017, Tiangong II received the first "space express"

In 2020, the long march 5B launch vehicle successfully made its first flight

In 2021, China's zhurong spacecraft successfully landed on Mars and the Chinese space station welcomed the first three Chinese astronauts

...

Star Trek, are you ready!

Effect display

Video link of station B

https://www.bilibili.com/video/BV1d44y1q7To?share_source=copy_web

Github link

https://github.com/yongxinliao-hqu/C-3PO_Wechaty_PaddleHub

###Welcome to Star and Fork 😃

Implementation process of the project

Step 1: test the basic function modules

Aurebesh language converter

https://aistudio.baidu.com/aistudio/projectdetail/2171156

Baidu translation API

https://aistudio.baidu.com/aistudio/projectdetail/2171717

Test wechat dialogue robot

https://aistudio.baidu.com/aistudio/clusterprojectdetail/2170096/

The second step is to combine the basic functions to achieve the effect of "interstellar communicator"

References reference items

  1. Server setup and WeChaty Token acquisition

https://aistudio.baidu.com/aistudio/projectdetail/1836012

  1. Wandering in the future: science fiction robot based on WeChaty, PaddleHub and Caiyun Xiaomeng

https://aistudio.baidu.com/aistudio/projectdetail/1896705

  1. Baidu general API translation

http://api.fanyi.baidu.com/product/11

  1. Meitu XiuXiu cut head

https://pc.meitu.com

  1. paddlehub official website address:

https://www.paddlepaddle.org.cn/hub

  1. Wechaty document

https://wechaty.js.org/docs/

  1. Get a Free Wechaty Token

https://wechaty.js.org/docs/puppet-services/

The following is an explanation of the program. You need to create a new script project to run it

Because asyncio.run() cannot be called from a running event loop

After running this project, download the following files and put them into your new script project

  • paddlehub-chatbot.py
  • run.sh
  • StarWarsCharater folder and its files
  • Aurebesh folder and its files

0 install Wechaty and Paddle series and download the Plato mini model

!pip install --upgrade pip
!pip install wechaty==0.7dev17

!pip install paddlehub
!pip install paddlepaddle
!pip install paddlenlp

# Download model
!hub install plato-mini==1.0.0
# Setting environment variables
!export WECHATY_PUPPET=wechaty-puppet-service
!export WECHATY_PUPPET_SERVICE_TOKEN=puppet_paimon_xxxxxx #Your own token

# Set GPU for model prediction
!export CUDA_VISIBLE_DEVICES=0

# Create two folders for saving pictures
!mkdir -p image

1. Import necessary libraries, initialization models and variables

from collections import deque
import os
import asyncio
import cv2
import requests
import random
import json
from hashlib import md5
from PIL import Image
import numpy as np

import random

from wechaty import (
    Contact,
    FileBox,
    Message,
    Wechaty,
    ScanStatus,
)

from wechaty_puppet import MessageType

import paddlehub as hub

# Initialize the paddlehub Plato mini model
model = hub.Module(name='plato-mini', version='1.0.0')
model._interactive_mode = True
model.max_turn = 10
model.context = deque(maxlen=model.max_turn)

# Initialization function module selection variable
function_chosen = 0 # 0 initial state, 1 interstellar communicator, 2 get interstellar name

2. Define relevant methods of Aurebsh language conversion

#Delete the last translation result file
def clean_previous_reslults(path):
    for root,dirs,files in os.walk(path): 
        for file in files: 
            if 'translation' in file:
                os.remove(os.path.join(root,file))
#Load the path of each target letter to target_letters_list
def load_target_letters(target_letters_path):
    target_letter_list = []
    for root,dirs,files in os.walk(target_letters_path): 
        for file in files: 
            target_letter_list.append(os.path.join(root,file))
    print("############# target_letter_list #############")
    print("Number of target letters and symbols: " + str(len(target_letter_list)))
    print(target_letter_list)
    return target_letter_list
# Set Aurebesh letter folder path
aurebesh_letters_path = '/home/aistudio/Aurebesh/'
    
# Load the path of each Aurebesh letter to aurebesh_letters_list
aurebesh_letters_list = load_target_letters(aurebesh_letters_path)
# Load the letters of the source sentence to source_letters_list
def load_source_letters(sentence):
    source_letter_list = list(sentence)
    print("\n############# source_letter_list #############")
    print("Number of source letters: " + str(len(source_letter_list)))
    print(source_letter_list)
    return source_letter_list
# Set source_ letters_ Letters in list and target_ letters_ The letters in the list are mapped and stored in final_letters_list
def translation(source_letters_list, target_letters_path, target_letters_list):
    final_letters_list = []
    for i in range(len(source_letters_list)):
        y = source_letters_list[i]
        for j in range(len(target_letters_list)):
            x = target_letters_list[j].replace(target_letters_path,'')[0]
            if not x == '_': # Those with "" in the file name are punctuation, and the rest are letters
                if x.lower() == y or x.upper() == y: # Case insensitive
                    final_letters_list.append(target_letters_list[j])
            elif y == ' ' and 'Aurebesh/_blank.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == ',' and 'Aurebesh/_comma.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '.' and 'Aurebesh/_period.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '?' and 'Aurebesh/_question_mark.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '!' and 'Aurebesh/_exclamation_mark.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == ':' and 'Aurebesh/_colon.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == ';' and 'Aurebesh/_semicolon.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '-' and 'Aurebesh/_dash.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '\'' and 'Aurebesh/_left_single_quotation_mark.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '\"' and 'Aurebesh/_left_double_quotation_mark.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == '(' and 'Aurebesh/_left_parenthesis.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
            elif y == ')' and 'Aurebesh/_right_parenthesis.png' in target_letters_list[j]:
                final_letters_list.append(target_letters_list[j])
    print("\n############# final_letters_list #############")
    print(final_letters_list)
    return final_letters_list
# Target letter image mosaic
def join_letters(letter_each_line, final_letters_list):   
    line_count = 0 # Total number of rows
    ims = [] # Picture list
    letter_total = len(final_letters_list) # Total letters
    width = 150 # Single image width
    height = 108 # Single image height
    
    # Print the number of pictures to be spliced
    print("\n############# number of letters to join #############")
    print("Number of letters to join: " + str(letter_total))
    
    # The total number of rows is calculated according to the number of letters in each row
    if letter_total % letter_each_line == 0:
        line_count = int(letter_total / letter_each_line)
    else:
        line_count = int(letter_total / letter_each_line + 1)
    
    # Obtain all letter images and convert them to the same size
    for i in range (letter_total):
        im_list = [Image.open(final_letters_list[i])] 
        #The picture is converted to the same size
        for im in im_list:
            new_img = im.resize((width, height), Image.BILINEAR)
            ims.append(new_img)
    
    # Create a large blank image (final output image)
    result = Image.new(ims[0].mode, (width * letter_each_line, height * line_count))
    
    # Create a blank long chart for each line
    for i in range (line_count):
        result_a_line = Image.new(ims[0].mode, (width * letter_each_line, height ))
        # Splice the pictures of each line
        for j in range (letter_each_line):
            if i*letter_each_line+j < len(ims):
                result_a_line.paste(ims[i*letter_each_line+j], box=(j * width,0))
        # Spell the pictures of each line into the final output picture
        result.paste(result_a_line, box=(0,i*height))
   
   
    # Print final output picture size
    print("\n############# result size #############")
    print(result.size)
    
    # Save final output picture
    result.save('/home/aistudio/image/translation.png')
# Execute translation
def do_translation(original_sentence,letter_each_line):
    # letter_ each_ Number of letters per line
    # Judge whether the sentence contains Chinese
    if is_contains_chinese(original_sentence):
        translation_result = translate_zh_to_en(original_sentence)
        processed_sentence = translation_result['trans_result'][0]['dst']
    else:
        processed_sentence = original_sentence
        
    # Load the letters of the source sentence into english_letters_list
    english_letters_list = load_source_letters(processed_sentence)
        
    # English_ letters_ Letters in list and aurabesh_ letters_ The letters in the list are mapped and stored in translation_letters_list
    translation_letters_list = translation(english_letters_list, aurebesh_letters_path, aurebesh_letters_list)
    
    # translation_letters_list
    
    join_letters(letter_each_line, translation_letters_list)
        
    # Display source text and final translation results
    print("\n############# source sentence #############")
    print("original: " + original_sentence)
    print("processed: " + processed_sentence)

3. Define relevant methods for calling baidu general translation API (Chinese to English or English to Chinese)

# Configuration parameters: appid, appkey, url
appid = 'Your own appid'
appkey = 'Your own appkey'
endpoint = 'http://api.fanyi.baidu.com'
path = '/api/trans/vip/translate'
url = endpoint + path
salt = random.randint(32768, 65536)
# Generate salt and sign
def make_md5(s, encoding='utf-8'):
    return md5(s.encode(encoding)).hexdigest()
# Translate English to Chinese
def translate_en_to_zh(query):
    # For list of language codes, please refer to `https://api.fanyi.baidu.com/doc/21`
    from_lang = 'en'
    to_lang =  'zh'
    sign = make_md5(appid + query + str(salt) + appkey)

    # Build request
    headers = {'Content-Type': 'application/x-www-form-urlencoded'}
    payload = {'appid': appid, 'q': query, 'from': from_lang, 'to': to_lang, 'salt': salt, 'sign': sign}

    # Send request
    r = requests.post(url, params=payload, headers=headers)
    result = r.json()
    
    # Show response
    print(json.dumps(result, indent=4, ensure_ascii=False))
    return result
# Translate English to Chinese
def translate_zh_to_en(query):
    # For list of language codes, please refer to `https://api.fanyi.baidu.com/doc/21`
    from_lang = 'zh'
    to_lang =  'en'
    sign = make_md5(appid + query + str(salt) + appkey)

    # Build request
    headers = {'Content-Type': 'application/x-www-form-urlencoded'}
    payload = {'appid': appid, 'q': query, 'from': from_lang, 'to': to_lang, 'salt': salt, 'sign': sign}

    # Send request
    r = requests.post(url, params=payload, headers=headers)
    result = r.json()
    
    # Show response
    print(json.dumps(result, indent=4, ensure_ascii=False))
    return result
# Judge the language of the string (Chinese or English)
def is_contains_chinese(strs):
    for _char in strs:
        if '\u4e00' <= _char <= '\u9fa5':
            return True
    return False

4 customized Wechaty method

#Feedback when receiving information
async def on_message(msg: Message):
    # languages
    language_chosen = 'Aurebesh'
    
    # Chat object list
    star_war_charater_name_list = ['Master Yoda','Anakin Skywalker','Qui-Gon jinn']
    star_war_charater_icon_list = ['StarWarsCharater/MasterYoda.png','StarWarsCharater/AnakinSkywalker.png','StarWarsCharater/Qui-GonJinn.png']
   
    """
    Message Handler for the Bot
    """
    global function_chosen
    
    print("====================== function_chosen get into on message:" + str(function_chosen))
    
    if isinstance(msg.text(), str) and len(msg.text()) > 0 and msg._payload.type == MessageType.MESSAGE_TYPE_TEXT:
        if function_chosen == 1:
            if msg.text() == "title":
                function_chosen = 2
                print("====================== function_chosen Exclusive interstellar name:" + str(function_chosen))
                await msg.say('Hello, please enter your name')
            elif msg.text() == "return":
                function_chosen = 3
                print("====================== function_chosen:" + str(function_chosen))
                await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
            else:
                bot_response = model.predict(data=msg.text())[0]
                    
                do_translation(msg.text(),10)
                file_box_aurebesh_in = FileBox.from_file('./image/translation.png')
                await msg.say('In human language translation...')
                await msg.say(file_box_aurebesh_in)
                    
                await msg.say('send out' + language_chosen + 'Reply and wait for reply...')
                do_translation(bot_response,10)
                file_box_aurebesh_out = FileBox.from_file('./image/translation.png')
                await msg.say(file_box_aurebesh_out)
                await msg.say(language_chosen + 'In language translation...')
                await msg.say(star_war_charater_name_list[1] + ": " + bot_response )  # Return the text generated by PaddleHub chatbot
        elif function_chosen == 2:
            if msg.text() == "communication":
                function_chosen = 1
                print("====================== function_chosen Start the interstellar communicator:" + str(function_chosen))
                the_random_number = random.randint(0,len(star_war_charater_name_list) - 1)
                await msg.say('Your communication object this time is:\n'+ star_war_charater_name_list[the_random_number]+ ' \n Transferring...')
                file_box_charater = FileBox.from_file(star_war_charater_icon_list[the_random_number])
                await msg.say(file_box_charater)
                await msg.say('Successfully connected' +star_war_charater_name_list[the_random_number] )
            elif msg.text() == "return":
                function_chosen = 3
                print("====================== function_chosen:" + str(function_chosen))
                await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
            else:
                do_translation(msg.text(),5)
                file_box_aurebesh_name = FileBox.from_file('./image/translation.png')
                await msg.say('Your name is:')
                await msg.say(file_box_aurebesh_name)
        elif msg.text() == "1" or msg.text() == "communication":
            function_chosen = 1
            print("====================== function_chosen Start the interstellar communicator:" + str(function_chosen))
            the_random_number = random.randint(0,len(star_war_charater_name_list) - 1)
            await msg.say('Your communication object this time is:\n'+ star_war_charater_name_list[the_random_number]+ ' \n Transferring...')
            file_box_charater = FileBox.from_file(star_war_charater_icon_list[the_random_number])
            await msg.say(file_box_charater)
            await msg.say('Successfully connected' +star_war_charater_name_list[the_random_number] )
        elif msg.text() == "2" or msg.text() == "title":
            function_chosen = 2
            print("====================== function_chosen Exclusive interstellar name:" + str(function_chosen))
            await msg.say('Hello, please enter your name')
        elif msg.text() == "3" or msg.text() == "return":
            function_chosen = 3
            print("====================== function_chosen:" + str(function_chosen))
            await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
        else:
            await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
#QR Code processing
async def on_scan(
        qrcode: str,
        status: ScanStatus,
        _data,
):
    """
    Scan Handler for the Bot
    """
    print('Status: ' + str(status))
    print('View QR Code Online: https://wechaty.js.org/qrcode/' + qrcode)

# Login processing
async def on_login(user: Contact):
    """
    Login Handler for the Bot
    """
    print(user)
    # TODO: To be written

# Wechaty main
async def main():
    """
    Async Main Entry
    """
    #
    # Make sure we have set WECHATY_PUPPET_SERVICE_TOKEN in the environment variables.
    #
    if 'WECHATY_PUPPET_SERVICE_TOKEN' not in os.environ:
        print('''
            Error: WECHATY_PUPPET_SERVICE_TOKEN is not found in the environment variables
            You need a TOKEN to run the Python Wechaty. Please goto our README for details
            https://github.com/wechaty/python-wechaty-getting-started/#wechaty_puppet_service_token
        ''')
    bot = Wechaty()

    bot.on('scan',      on_scan)
    bot.on('login',     on_login)
    bot.on('message',   on_message)

    await bot.start()

# The actual operation needs to be in the script project
asyncio.run(main())

Tags: NLP paddlepaddle

Posted on Sat, 06 Nov 2021 22:34:06 -0400 by cursed