Introduction to the word segmentation

TEST 1

'''
Created on 2018 December 8th 2013

@author: admin
'''

import jieba

'''
cut Method has two parameters
1)The first parameter is the string we want to segment
2)Second parameter cut_all It is used to control whether full mode is adopted
'''

#Full mode
word_list = jieba.cut("What a nice day today. Xiao Ming, let's go hiking!",cut_all=True)
print("Full mode:","|".join(word_list))
#Precise mode, the default is precise mode
word_list = jieba.cut("What a nice day today. Xiao Ming, let's go hiking!",cut_all=False)
print("Precise mode:","|".join(word_list))
#Search engine mode
word_list = jieba.cut_for_search("What a nice day today. Xiao Ming, let's go hiking!")
print("Search Engines:","|".join(word_list))
#Default mode
word_list = jieba.cut("What a nice day today. Xiao Ming, let's go hiking!")
print("Default mode:","|".join(word_list))

TEST 2

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''


#-*- coding:utf-8 -*-
import jieba

jieba.load_userdict("./dict.txt")
word_list = jieba.cut("Do you go hiking today? Let's change places! How about the garden?No problem, bean sprouts")
print("|".join(word_list))

TEST 3

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''


import jieba.analyse as al
 
content = open("./topk.txt","rb").read()
word_topk = al.extract_tags(content,topK=4)
print("|".join(word_topk))

TEST 4

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''


import jieba.posseg as pseg

words = pseg.cut("Qingdao Beijing is a good place")
for word in words:
    print(word.word,word.flag)

TEST 5

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''


import jieba
#Turn on parallel word segmentation mode. The parameter is the number of processes participating in parallel word segmentation
#jieba.enable_parallel(2)
#Turn off parallel participle
#jieba.disable_parallel()
content = open("./topk.txt","rb").read()
words = jieba.cut(content)
print("|".join(words))

TEST 6

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''


#-*- coding:utf-8 -*-
 
import jieba
jieba.set_dictionary("./dict.txt")
content = open("./topk.txt","rb").read()
words = jieba.cut(content)
print("|".join(words))

TEST 7

# *-*coding=utf8*-*
'''
Created on 2018 December 8th 2013

@author: admin
'''

import jieba

result = jieba.tokenize(u'What a nice day today. Honey, let's go hiking!')
for token in result:
    print("word %s\t\t start: %d \t\t end:%d" % (token[0],token[1],token[2]))

dict.txt

Garden Garden 5
 Bean sprouts 3 nr

topk.txt

I feel very happy after reading Mr. Cao Juren's "killing the wrong person", but I think it's outrageous to talk about some of them, so I want to raise a few objections——


Yuan Shikai [3] after the revolution of 1911, he killed the party members in a big way. From Yuan Shikai's point of view, he was not wrong at all, because he was just an anti revolutionary of the fake revolution.


What's wrong is that the revolutionist was deceived into thinking that he was really a somersault. He changed from a minister of Beiyang to a revolutionist, so he was brought into the same tune, shed everyone's blood, and floated him to the throne of the president. By the time of the second revolution, it seemed as if he was a wrestler again, changing from a "national servant" to a vampire king. In fact, he didn't, but he showed himself.


So kill, kill, kill. In Beijing, even hotels and inns are full of detectives. In addition, the "military and political law enforcement office" only sees the young people arrested for being suspected and sent in, but never sees them walk out alive. In addition, in the Government Gazette, we see the advertisements of Party members breaking away from the party every day, saying that they were pulled by friends before and entered the party by mistake. Now they know their own fallacies, so we need to change our minds I'm a good man.


It soon proved that Yuan Shikai was not wrong in killing people. He was going to be emperor.


In a flash, it has been twenty years. Now, the young people in their twenties are still sucking. How fast the time is.


However, Yuan Shikai wanted to be an emperor himself. Why did he leave his real counterpart, the old emperor? This need not be discussed, as long as we look at the current warlord scuffle. They fought to the death of each other, as if they were fighting against each other. But later, as long as one "went wild", they would be polite. However, for revolutionaries, even if they had not fought, they would never let one go. They know very well.


So I think that the reason why the Chinese revolution was like this is not because they "killed the wrong people", but because we saw the wrong people.


At the end of the day, I also have some objections to the idea of "killing more people above middle age". But since I was already above middle age, I just looked at the ground to avoid suspicion.


April 10 Cao Juren.


I remember that under the original "polite", there are still sentences with the meaning of "maybe when I go abroad, I will hold a farewell party", which were deleted later.


April 12 diary Cao Juren.
81 original articles published, 133 praised, 200000 visitors+
Private letter follow

Posted on Tue, 17 Mar 2020 02:22:45 -0400 by carleihar