Paddle dialogue story continues to create your infinite world
- If you can, which movie or novel do you want to go through most?
- This time, we use GPT2 text generation and wechaty dialogue to continue the story, interact with AI and jointly create the plot
- Preset the world scene you want to interact with, for example, in the face of the attack of the trisomy, play your brain hole and input any action or plot outline you think of. The continuation of the model ensures that you like (hei) not (ren) and never (wen) mouth (hao)
Project introduction
Local test
github There is a specific native deployment process in
The code in this notebook is run on the command line
inspiration source
-
Running group: a desktop role-playing game. The host describes the plot and others promote the plot. Everyone has his own career, skills and so on.
-
Infinite flow: through various films, animation and novels to interact with the original plot. Reference works: phantom hero, cube building, infinite terror.
-
AI Dungeon: This is the world's first pure AI generated text adventure game. Through this AIDungeon 2, players will be exposed to a completely free virtual world. The whole game world will be expanded and continued around the player's input. It can be said that Everything is in my hands
Generate sample presentation
- Example 1:
Main god ABA: cold, shaking... When you wake up, you jump up from the ground and look around in panic. The office environment in your mind is confused with the environment in front of you. A few seconds later, you wake up from the confusion. Now you come to a city, zombies roaring in the streets, and you realize that the nuclear bomb will come in an hour. Me: look around and see if there are any self-defense weapons God Abba: is it possible to survive. Me: suddenly I found a pistol nearby, full of bullets. At this time, a zombie rushed out The LORD God ABA: without hesitation, he shot the zombie in the head, and then the zombie fell to the ground and died. Me: look around to see if there are any cars that can start and escape the city as soon as possible Main god Abba: Zombies surround you. You have to run to the next building. At this time, you come to an underground laboratory, and the cold alarm is reminding you T The virus has leaked. A tube of virus reagent is in front of you.
- Example 2:
Me: finally, I began to counter attack the trisomy world God Abba: with the help of Technology,I finally became super British and catch up with the United States,Human firm eyes twinkled in scarlet eyes,The super British base will be captured by me,Next second,Countless zombies rushed at me,Remember to use a flamethrower,Um,I am a revolutionary soldier,Pick up the chainsaw in your hand,Unity of all things,Point to the direction of the zombie migration...... I: finally catch up with Britain and the United States to achieve world peace God Abba: achieve world hegemony,The rainbow road to the mighty lion......(It seems that it doesn't rhyme anymore) Come here first,Make up the ending when you have time<eod>Change careers IT Is there a future? I am a programmer who has graduated for a year ,The hard life makes me find a job a challenge. There is nowhere to escape. You run to the team of the "natural selection" spacecraft. You realize that the trisomy fleet is about to arrive. Zhang Beihai will hijack the spacecraft to escape the solar system. Me:???
Station B link
Installation dependency
# Note that you must install this version to experience gpt2 !pip install paddlehub==2.0.4 paddlenlp==2.0.0rc14 sentencepiece==0.1.92
Create a Model and load the official pre training Model
- Many pre training semantic model parameters are built in PaddleNLP, including Chinese GPT2 model
- Through gpt2fortraining.from_ Pre trained ('gpt2 base CN ') can call the built-in pre training model parameters
- If you install a higher version of paddlenlp, you can gptforpretraining.from_ Pre trained ('gpt CPM small CN distill ') experience CPM and other models
import paddle
import paddle.nn as nn
import numpy as np
import paddlehub as hub
from paddlenlp.transformers import *
import queue
class Model():
def __init__(self):
# self.model = GPTForPretraining.from_pretrained('gpt-cpm-small-cn-distill')
self.model = GPT2ForPretraining.from_pretrained('gpt2-base-cn')
self.model.eval()
# self.tokenizer = GPTChineseTokenizer.from_pretrained('gpt-cpm-small-cn-distill')
self.tokenizer = GPT2ChineseTokenizer.from_pretrained('gpt2-base-cn')
self.tokenizer.encode('.') # initialization
def top_k_top_p_filtering(self, logits, top_k=0, top_p=1.0, filter_value=-float('Inf')):
top_k = min(top_k, logits.shape[-1]) # Safety check
logits_np = logits.numpy()
if top_k > 0:
# Remove all tokens with a probability less than the last token of the top-k
indices_to_remove = logits_np < np.sort(logits_np)[-top_k]
logits_np[indices_to_remove] = filter_value
if top_p < 1.0:
sorted_logits = paddle.sort(logits, descending=True)
sorted_indices = paddle.argsort(logits, descending=True).numpy()
cumulative_probs = paddle.cumsum(paddle.nn.functional.softmax(
sorted_logits, axis=-1), axis=-1).numpy()
# Remove tokens with cumulative probability above the threshold
sorted_indices_to_remove = cumulative_probs > top_p
# Shift the indices to the right to keep also the first token above the threshold
sorted_indices_to_remove[...,
1:] = sorted_indices_to_remove[..., :-1]
sorted_indices_to_remove[..., 0] = 0
indices_to_remove = sorted_indices[sorted_indices_to_remove]
logits_np[indices_to_remove] = filter_value
return paddle.to_tensor(logits_np)
def sample(self, text, max_len=32, end_word='. ', repitition_penalty=1.0, temperature=1.0, top_p=0.9):
with paddle.no_grad():
# Termination flag
if end_word is not None:
stop_id = self.tokenizer.encode(end_word)
if 'input_ids' in stop_id: # Judge the use of cpm
stop_id = stop_id['input_ids']
stop_id = stop_id[-1]
ids = self.tokenizer.encode(text)
if 'input_ids' in ids: # Judge the use of cpm
ids = ids['input_ids']
input_ids = paddle.to_tensor(ids).unsqueeze(0)
else:
input_ids = paddle.to_tensor(np.array(ids).reshape(1, -1).astype('int64'))
output, cached_kvs = self.model(input_ids, use_cache=True)
next_token_logits = output[0, -1, :]
for id in set(ids):
next_token_logits[id] /= repitition_penalty
next_token_logits = next_token_logits / temperature
filtered_logits = self.top_k_top_p_filtering(next_token_logits, top_k=0, top_p=1.0)
next_token = paddle.multinomial(paddle.nn.functional.softmax(filtered_logits, axis=-1), num_samples=1).numpy()
ids += [int(next_token)]
response = [int(next_token)] # As long as it is generated later
for i in range(max_len):
input_id = paddle.to_tensor(np.array([next_token]).reshape(1, -1).astype('int64'))
output, cached_kvs = self.model(input_id, use_cache=True, cache=cached_kvs)
next_token_logits = output[0, -1, :]
for id in set(ids):
next_token_logits[id] /= repitition_penalty
next_token_logits = next_token_logits / temperature
filtered_logits = self.top_k_top_p_filtering(next_token_logits, top_k=0, top_p=1.0)
next_token = paddle.multinomial(paddle.nn.functional.softmax(filtered_logits, axis=-1), num_samples=1).numpy()
ids += [int(next_token)]
response += [int(next_token)]
# Stop prediction according to the termination flag
if (end_word is not None) and (int(next_token) == stop_id):
break
# If cpm is used, the tokenizer as of August 3 has no deocde method
if hasattr(self.tokenizer, 'convert_ids_to_string'):
return self.tokenizer.convert_ids_to_string(response)
return self.tokenizer.decode(response)
Set a world view
background_events = [
'Now you come to a city where zombies roar in the streets. You seem to have come to Raccoon City, and the nuclear bomb will come in an hour.',
'After thousands of hardships, you come to an underground laboratory, next to the cold alarm is reminding you T The virus has leaked. A tube of virus reagent is in front of you.',
'You came to the three body world. The virus you were infected with accidentally leaked to this world. Now the whole world is facing a biochemical crisis.',
'There is nowhere to escape. You run to the team of the "natural selection" spacecraft. You realize that the trisomy fleet is about to arrive. Zhang Beihai will hijack the spacecraft to escape the solar system.',
]
class World():
def __init__(self):
self.init = False
self.model = Model()
def sample(self, sentence):
outputs = self.model.sample(
sentence, # Enter text
max_len=128, # Maximum length of generated text
end_word='. ', # End of period generation
)
return outputs
def start(self):
self.context_list = [
'Cold, shaking... The moment you wake up, you jump up from the ground and look around in panic. The office environment in your mind is confused with the environment in front of you. A few seconds later, you wake up from the confusion.',
]
self.lastPoint = 1
self.init = True
self.worlds_event = queue.Queue(maxsize=0)
for i in background_events:
i = i.replace('\n','')
self.worlds_event.put(i)
event = self.worlds_event.get()
self.context_list.append(event)
return ''.join(self.context_list)
def receive(self, user_input):
if self.worlds_event.empty():
print('Your adventure is over')
return 'Your adventure is over'
user_output = ''
inputs = user_input + '，'
self.context_list.append(inputs)
context = ''.join(self.context_list) # You can select a complete history
# context = ''.join(self.context_list[-2:]) # Select recent chat history
# context = self.summary(self.context_list) # Make a text summary of the previous dialogue history
user_output = self.sample(context)
self.context_list.append(user_output)
self.lastPoint += 1
if self.lastPoint % 4 == 0:
if self.worlds_event.empty():
print(user_output, 'It's over')
return 'Your adventure is over'
event = self.worlds_event.get()
self.context_list.append(event)
user_output = user_output + event
print(user_output)
return user_output
Load model
# Here to download a gpt 13 G, wait patiently world = World()
Start the game
- Due to the problem of hardware video memory, you may encounter * * * out of memory * * * during multiple runs, and you need to restart the actuator
print(world.start())
while True:
ans = input()
print(ans)
if world.receive(ans) == 'Your adventure is over':
break
Cold, shaking... The moment you wake up, you jump up from the ground and look around in panic. The office environment in your mind is confused with the environment in front of you. A few seconds later, you wake up from the confusion. Now you come to a city where zombies roar in the streets. You seem to have come to Raccoon City, and the nuclear bomb will come in an hour. Is Alice there? If not, I'll run out of town and hit dozens of zombies on the way We can beat back their attack. In fact, I don't think I can beat them back. There are too many of them. What if they explode later Why don't you look at the countdown,Six seconds later,I managed to escape...... The signs on the map have become a red radius. ？？Where did I go? Why don't I run to the toilet So far away,You're not like playing games and running maps,You won't get lost. After thousands of hardships, you come to an underground laboratory, next to the cold alarm is reminding you T The virus has leaked. A tube of virus reagent is in front of you. The rich rely on technology and the poor rely on variation. With this injection, I can evolve Direct monster,That's it. This is a biochemical crisis, not a sadistic prototype. Now I can kill it all the way Pretend to be a pig and eat a tiger. Then I finally ran outside the city and survived Luckily I remember everything,But why is the girl in front of me?Everywhere,The feeling of suffocation,Just close your eyes,Life is really unfair,You can know what people think at first sight. I got an injection and became a girl, but what I want to become is superman Scarlet eyes,Hot blood gushed out. You came to the three body world. The virus you were infected with accidentally leaked to this world. Now the whole world is facing a biochemical crisis. Let me find a safe place and start towards the north and south poles Frio's house is very close. Find an underground base in the Arctic and let me see how to prevent the end of the world In this way, we can have more time with the precious human beings in the world. Finally, I began to counter attack the three body world With the help of Technology,I finally became super British and catch up with the United States,Human firm eyes twinkled in scarlet eyes,The super British base will be captured by me,Next second,Countless zombies rushed at me,Remember to use a flamethrower,Um,I am a revolutionary soldier,Pick up the chainsaw in your hand,Unity of all things,Point to the direction of the zombie migration......Fate,Don't let me do bad things all the way,The dark animal skin on his back faces north,These paved the way for my success Finally, catch up with Britain and the United States to achieve world peace Achieve world hegemony,The rainbow road to the mighty lion......(It seems that it doesn't rhyme anymore) Come here first,Make up the ending when you have time<eod>Change careers IT Is there a future? I am a programmer who has graduated for a year ,The hard life makes me find a job a challenge. There is nowhere to escape. You run to the team of the "natural selection" spacecraft. You realize that the trisomy fleet is about to arrive. Zhang Beihai will hijack the spacecraft to escape the solar system. OK, OK, I see. Then I'll start the wandering earth program Your adventure is over
summary
- Using paddelnlp and its built-in Chinese pre training model, a simple text generation task can be realized
- In order to ensure that the plot is smooth and sustainable, some key plots need to be defined in advance, such as "what should you do when a tube of virus reagent is placed in front of you" and so on.
- At present, there is no finetune, and the quality of stories generated is uneven. If there is a specific corpus, the fine-tuning effect will be better. However, after the latest test, the difficulty of data acquisition is underestimated, and there is no appropriate data set above, action and below
- Through the self labeled samples to finetune GPT-2, the test found that the finetune effect of dozens of samples was almost 0
- The original Ai Dungeon has shown us what level this game can reach with good training process and training data. Whether the idea of such projects is the direction of future game development or not, there is no doubt that they will play an important role in some places. For example, let NPC s in the game have a certain degree of interactivity, brainstorm with writers, provide creative inspiration, and so on.
Reference items
about
LMc Lab of language and psychological computing, Tianjin University, with research interests in intelligent dialogue and psychological computing, personalized recommendation, etc
Welcome everyone to leave a message with questions, exchange and learn, and make progress and growth together