Etiquette robot C-3PO: help your Star Trek!
In 1970, the first man-made satellite "Dongfanghong 1" was successfully launched!
In 2003, Shenzhou V manned spacecraft successfully launched!
In 2007, Chang'e-1 successfully flew to the moon!
In 2008, Shenzhou VII manned spacecraft realized the astronauts' extravehicular flight
2011 Shenzhou 8 "hand in hand" Tiangong 1 (automatic rendezvous and docking)
In 2013, Shenzhou 10 conducted its first space teaching
In 2016, Shenzhou 11 realized the medium-term manned presence (33 days of space life)
In 2017, Tiangong II received the first "space express"
In 2020, the long march 5B launch vehicle successfully made its first flight
In 2021, China's zhurong spacecraft successfully landed on Mars and the Chinese space station welcomed the first three Chinese astronauts
...
Star Trek, are you ready!
Effect display
Video link of station B
https://www.bilibili.com/video/BV1d44y1q7To?share_source=copy_web
Github link
https://github.com/yongxinliao-hqu/C-3PO_Wechaty_PaddleHub
###Welcome to Star and Fork 😃
Implementation process of the project
Step 1: test the basic function modules
Aurebesh language converter
https://aistudio.baidu.com/aistudio/projectdetail/2171156
Baidu translation API
https://aistudio.baidu.com/aistudio/projectdetail/2171717
Test wechat dialogue robot
https://aistudio.baidu.com/aistudio/clusterprojectdetail/2170096/
The second step is to combine the basic functions to achieve the effect of "interstellar communicator"
References reference items
- Server setup and WeChaty Token acquisition
https://aistudio.baidu.com/aistudio/projectdetail/1836012
- Wandering in the future: science fiction robot based on WeChaty, PaddleHub and Caiyun Xiaomeng
https://aistudio.baidu.com/aistudio/projectdetail/1896705
- Baidu general API translation
http://api.fanyi.baidu.com/product/11
- Meitu XiuXiu cut head
https://pc.meitu.com
- paddlehub official website address:
https://www.paddlepaddle.org.cn/hub
- Wechaty document
https://wechaty.js.org/docs/
- Get a Free Wechaty Token
https://wechaty.js.org/docs/puppet-services/
The following is an explanation of the program. You need to create a new script project to run it
Because asyncio.run() cannot be called from a running event loop
After running this project, download the following files and put them into your new script project
- paddlehub-chatbot.py
- run.sh
- StarWarsCharater folder and its files
- Aurebesh folder and its files
0 install Wechaty and Paddle series and download the Plato mini model
!pip install --upgrade pip !pip install wechaty==0.7dev17 !pip install paddlehub !pip install paddlepaddle !pip install paddlenlp # Download model !hub install plato-mini==1.0.0
# Setting environment variables !export WECHATY_PUPPET=wechaty-puppet-service !export WECHATY_PUPPET_SERVICE_TOKEN=puppet_paimon_xxxxxx #Your own token # Set GPU for model prediction !export CUDA_VISIBLE_DEVICES=0 # Create two folders for saving pictures !mkdir -p image
1. Import necessary libraries, initialization models and variables
from collections import deque
import os
import asyncio
import cv2
import requests
import random
import json
from hashlib import md5
from PIL import Image
import numpy as np
import random
from wechaty import (
Contact,
FileBox,
Message,
Wechaty,
ScanStatus,
)
from wechaty_puppet import MessageType
import paddlehub as hub
# Initialize the paddlehub Plato mini model model = hub.Module(name='plato-mini', version='1.0.0') model._interactive_mode = True model.max_turn = 10 model.context = deque(maxlen=model.max_turn) # Initialization function module selection variable function_chosen = 0 # 0 initial state, 1 interstellar communicator, 2 get interstellar name
2. Define relevant methods of Aurebsh language conversion
#Delete the last translation result file
def clean_previous_reslults(path):
for root,dirs,files in os.walk(path):
for file in files:
if 'translation' in file:
os.remove(os.path.join(root,file))
#Load the path of each target letter to target_letters_list
def load_target_letters(target_letters_path):
target_letter_list = []
for root,dirs,files in os.walk(target_letters_path):
for file in files:
target_letter_list.append(os.path.join(root,file))
print("############# target_letter_list #############")
print("Number of target letters and symbols: " + str(len(target_letter_list)))
print(target_letter_list)
return target_letter_list
# Set Aurebesh letter folder path
aurebesh_letters_path = '/home/aistudio/Aurebesh/'
# Load the path of each Aurebesh letter to aurebesh_letters_list
aurebesh_letters_list = load_target_letters(aurebesh_letters_path)
# Load the letters of the source sentence to source_letters_list
def load_source_letters(sentence):
source_letter_list = list(sentence)
print("\n############# source_letter_list #############")
print("Number of source letters: " + str(len(source_letter_list)))
print(source_letter_list)
return source_letter_list
# Set source_ letters_ Letters in list and target_ letters_ The letters in the list are mapped and stored in final_letters_list
def translation(source_letters_list, target_letters_path, target_letters_list):
final_letters_list = []
for i in range(len(source_letters_list)):
y = source_letters_list[i]
for j in range(len(target_letters_list)):
x = target_letters_list[j].replace(target_letters_path,'')[0]
if not x == '_': # Those with "" in the file name are punctuation, and the rest are letters
if x.lower() == y or x.upper() == y: # Case insensitive
final_letters_list.append(target_letters_list[j])
elif y == ' ' and 'Aurebesh/_blank.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == ',' and 'Aurebesh/_comma.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '.' and 'Aurebesh/_period.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '?' and 'Aurebesh/_question_mark.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '!' and 'Aurebesh/_exclamation_mark.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == ':' and 'Aurebesh/_colon.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == ';' and 'Aurebesh/_semicolon.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '-' and 'Aurebesh/_dash.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '\'' and 'Aurebesh/_left_single_quotation_mark.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '\"' and 'Aurebesh/_left_double_quotation_mark.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == '(' and 'Aurebesh/_left_parenthesis.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
elif y == ')' and 'Aurebesh/_right_parenthesis.png' in target_letters_list[j]:
final_letters_list.append(target_letters_list[j])
print("\n############# final_letters_list #############")
print(final_letters_list)
return final_letters_list
# Target letter image mosaic
def join_letters(letter_each_line, final_letters_list):
line_count = 0 # Total number of rows
ims = [] # Picture list
letter_total = len(final_letters_list) # Total letters
width = 150 # Single image width
height = 108 # Single image height
# Print the number of pictures to be spliced
print("\n############# number of letters to join #############")
print("Number of letters to join: " + str(letter_total))
# The total number of rows is calculated according to the number of letters in each row
if letter_total % letter_each_line == 0:
line_count = int(letter_total / letter_each_line)
else:
line_count = int(letter_total / letter_each_line + 1)
# Obtain all letter images and convert them to the same size
for i in range (letter_total):
im_list = [Image.open(final_letters_list[i])]
#The picture is converted to the same size
for im in im_list:
new_img = im.resize((width, height), Image.BILINEAR)
ims.append(new_img)
# Create a large blank image (final output image)
result = Image.new(ims[0].mode, (width * letter_each_line, height * line_count))
# Create a blank long chart for each line
for i in range (line_count):
result_a_line = Image.new(ims[0].mode, (width * letter_each_line, height ))
# Splice the pictures of each line
for j in range (letter_each_line):
if i*letter_each_line+j < len(ims):
result_a_line.paste(ims[i*letter_each_line+j], box=(j * width,0))
# Spell the pictures of each line into the final output picture
result.paste(result_a_line, box=(0,i*height))
# Print final output picture size
print("\n############# result size #############")
print(result.size)
# Save final output picture
result.save('/home/aistudio/image/translation.png')
# Execute translation
def do_translation(original_sentence,letter_each_line):
# letter_ each_ Number of letters per line
# Judge whether the sentence contains Chinese
if is_contains_chinese(original_sentence):
translation_result = translate_zh_to_en(original_sentence)
processed_sentence = translation_result['trans_result'][0]['dst']
else:
processed_sentence = original_sentence
# Load the letters of the source sentence into english_letters_list
english_letters_list = load_source_letters(processed_sentence)
# English_ letters_ Letters in list and aurabesh_ letters_ The letters in the list are mapped and stored in translation_letters_list
translation_letters_list = translation(english_letters_list, aurebesh_letters_path, aurebesh_letters_list)
# translation_letters_list
join_letters(letter_each_line, translation_letters_list)
# Display source text and final translation results
print("\n############# source sentence #############")
print("original: " + original_sentence)
print("processed: " + processed_sentence)
3. Define relevant methods for calling baidu general translation API (Chinese to English or English to Chinese)
# Configuration parameters: appid, appkey, url appid = 'Your own appid' appkey = 'Your own appkey' endpoint = 'http://api.fanyi.baidu.com' path = '/api/trans/vip/translate' url = endpoint + path salt = random.randint(32768, 65536)
# Generate salt and sign
def make_md5(s, encoding='utf-8'):
return md5(s.encode(encoding)).hexdigest()
# Translate English to Chinese
def translate_en_to_zh(query):
# For list of language codes, please refer to `https://api.fanyi.baidu.com/doc/21`
from_lang = 'en'
to_lang = 'zh'
sign = make_md5(appid + query + str(salt) + appkey)
# Build request
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
payload = {'appid': appid, 'q': query, 'from': from_lang, 'to': to_lang, 'salt': salt, 'sign': sign}
# Send request
r = requests.post(url, params=payload, headers=headers)
result = r.json()
# Show response
print(json.dumps(result, indent=4, ensure_ascii=False))
return result
# Translate English to Chinese
def translate_zh_to_en(query):
# For list of language codes, please refer to `https://api.fanyi.baidu.com/doc/21`
from_lang = 'zh'
to_lang = 'en'
sign = make_md5(appid + query + str(salt) + appkey)
# Build request
headers = {'Content-Type': 'application/x-www-form-urlencoded'}
payload = {'appid': appid, 'q': query, 'from': from_lang, 'to': to_lang, 'salt': salt, 'sign': sign}
# Send request
r = requests.post(url, params=payload, headers=headers)
result = r.json()
# Show response
print(json.dumps(result, indent=4, ensure_ascii=False))
return result
# Judge the language of the string (Chinese or English)
def is_contains_chinese(strs):
for _char in strs:
if '\u4e00' <= _char <= '\u9fa5':
return True
return False
4 customized Wechaty method
#Feedback when receiving information
async def on_message(msg: Message):
# languages
language_chosen = 'Aurebesh'
# Chat object list
star_war_charater_name_list = ['Master Yoda','Anakin Skywalker','Qui-Gon jinn']
star_war_charater_icon_list = ['StarWarsCharater/MasterYoda.png','StarWarsCharater/AnakinSkywalker.png','StarWarsCharater/Qui-GonJinn.png']
"""
Message Handler for the Bot
"""
global function_chosen
print("====================== function_chosen get into on message:" + str(function_chosen))
if isinstance(msg.text(), str) and len(msg.text()) > 0 and msg._payload.type == MessageType.MESSAGE_TYPE_TEXT:
if function_chosen == 1:
if msg.text() == "title":
function_chosen = 2
print("====================== function_chosen Exclusive interstellar name:" + str(function_chosen))
await msg.say('Hello, please enter your name')
elif msg.text() == "return":
function_chosen = 3
print("====================== function_chosen:" + str(function_chosen))
await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
else:
bot_response = model.predict(data=msg.text())[0]
do_translation(msg.text(),10)
file_box_aurebesh_in = FileBox.from_file('./image/translation.png')
await msg.say('In human language translation...')
await msg.say(file_box_aurebesh_in)
await msg.say('send out' + language_chosen + 'Reply and wait for reply...')
do_translation(bot_response,10)
file_box_aurebesh_out = FileBox.from_file('./image/translation.png')
await msg.say(file_box_aurebesh_out)
await msg.say(language_chosen + 'In language translation...')
await msg.say(star_war_charater_name_list[1] + ": " + bot_response ) # Return the text generated by PaddleHub chatbot
elif function_chosen == 2:
if msg.text() == "communication":
function_chosen = 1
print("====================== function_chosen Start the interstellar communicator:" + str(function_chosen))
the_random_number = random.randint(0,len(star_war_charater_name_list) - 1)
await msg.say('Your communication object this time is:\n'+ star_war_charater_name_list[the_random_number]+ ' \n Transferring...')
file_box_charater = FileBox.from_file(star_war_charater_icon_list[the_random_number])
await msg.say(file_box_charater)
await msg.say('Successfully connected' +star_war_charater_name_list[the_random_number] )
elif msg.text() == "return":
function_chosen = 3
print("====================== function_chosen:" + str(function_chosen))
await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
else:
do_translation(msg.text(),5)
file_box_aurebesh_name = FileBox.from_file('./image/translation.png')
await msg.say('Your name is:')
await msg.say(file_box_aurebesh_name)
elif msg.text() == "1" or msg.text() == "communication":
function_chosen = 1
print("====================== function_chosen Start the interstellar communicator:" + str(function_chosen))
the_random_number = random.randint(0,len(star_war_charater_name_list) - 1)
await msg.say('Your communication object this time is:\n'+ star_war_charater_name_list[the_random_number]+ ' \n Transferring...')
file_box_charater = FileBox.from_file(star_war_charater_icon_list[the_random_number])
await msg.say(file_box_charater)
await msg.say('Successfully connected' +star_war_charater_name_list[the_random_number] )
elif msg.text() == "2" or msg.text() == "title":
function_chosen = 2
print("====================== function_chosen Exclusive interstellar name:" + str(function_chosen))
await msg.say('Hello, please enter your name')
elif msg.text() == "3" or msg.text() == "return":
function_chosen = 3
print("====================== function_chosen:" + str(function_chosen))
await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
else:
await msg.say('Interstellar communicator is starting \n Reply to "communication" and start the interstellar communicator \n Reply to "title" to obtain the exclusive star name \n Reply "back" to return to the function menu')
#QR Code processing
async def on_scan(
qrcode: str,
status: ScanStatus,
_data,
):
"""
Scan Handler for the Bot
"""
print('Status: ' + str(status))
print('View QR Code Online: https://wechaty.js.org/qrcode/' + qrcode)
# Login processing
async def on_login(user: Contact):
"""
Login Handler for the Bot
"""
print(user)
# TODO: To be written
# Wechaty main
async def main():
"""
Async Main Entry
"""
#
# Make sure we have set WECHATY_PUPPET_SERVICE_TOKEN in the environment variables.
#
if 'WECHATY_PUPPET_SERVICE_TOKEN' not in os.environ:
print('''
Error: WECHATY_PUPPET_SERVICE_TOKEN is not found in the environment variables
You need a TOKEN to run the Python Wechaty. Please goto our README for details
https://github.com/wechaty/python-wechaty-getting-started/#wechaty_puppet_service_token
''')
bot = Wechaty()
bot.on('scan', on_scan)
bot.on('login', on_login)
bot.on('message', on_message)
await bot.start()
# The actual operation needs to be in the script project asyncio.run(main())