Shell pipe directive pipe

Pipe command pipe

1. The pipeline command only processes the standard output, and the standard error output will be omitted.
2. The pipeline command must be able to accept the data from the previous command as standard input to continue processing

Select the commands cut and grep

After analyzing a piece of data, take out what we want. Or take out what we want by analyzing keywords

Selection information is usually analyzed line by line

cut takes out the required information

Take out the information we want from a line of information, and process the information in behavioral units

grammar
Used for characters with specific segmentation
cut -d 'split character' - f field number (counting from 1)

It is used for neatly arranged information to take out a fixed character interval in character units
cut -c character interval

Options and parameters
-d followed by a separator character, used with - f
-f fields take out the number of segments according to the number of segments divided by - d
-c take out the fixed character interval in character units

case

Take out the third field of PATH

[ranan@c105 ~]$ echo $PATH
/home/mpi/bin:/home/ranan/.local/bin:/home/ranan/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin
[ranan@c105 ~]$ echo $PATH | cut -d ':' -f 3,4
/home/ranan/bin
[ranan@c105 ~]$ echo $PATH | cut -d ':' -f 3-5
/home/ranan/bin:/usr/local/bin:/usr/bin

Take out the first 5 characters of each line

[ranan@c105 ~]$ echo $PATH | cut -c 1-5
/home

grep fetches the required rows and filters the unnecessary rows

Analyze a line of information. If there is any information we need, take out a line.
You can first grep out a row, and then use cut to get the required information.

grammar

grep [-A] [-B] [-acinv] [--color=auto] 'find character' filename

Options and parameters
-a find data in the form of text file by binary text
-c count calculates the time when 'find character' is found
-i ignore ignore case
-n output line number
-v reverses the output of lines without characters
--color=auto you can add color to the found part. After centOS7, it will be displayed by default

-Numbers can be added after A. after not only outputs this line, but also the following n lines
-B can be followed by numbers. before not only outputs this line, but also the previous n lines

[ranan@c105 ~]$ last -n 5
ranan    pts/1        192.168.10.1     Fri Dec  3 15:53   still logged in
ranan    pts/0        192.168.10.1     Fri Dec  3 10:11    gone - no logout
reboot   system boot  4.18.0-240.22.1. Fri Dec  3 10:11   still running
ranan    pts/0        192.168.10.1     Thu Dec  2 20:49 - 09:52  (13:03)
reboot   system boot  4.18.0-240.22.1. Thu Dec  2 20:49 - 09:52  (13:03)

wtmp begins Sun May 23 20:25:21 2021

[ranan@c105 ~]$ last -n 5 | grep -n -A1  '192'
1:ranan    pts/1        192.168.10.1     Fri Dec  3 15:53   still logged in
2:ranan    pts/0        192.168.10.1     Fri Dec  3 10:11    gone - no logout
3-reboot   system boot  4.18.0-240.22.1. Fri Dec  3 10:11   still running
4:ranan    pts/0        192.168.10.1     Thu Dec  2 20:49 - 09:52  (13:03)
5-reboot   system boot  4.18.0-240.22.1. Thu Dec  2 20:49 - 09:52  (13:03)

Sort commands sort, wc, uniq

Usage scenario: calculate the total number of data in the same form in the data once

Sort sort

sort can be sorted according to different data forms. sort takes each line of the file as a unit and compares them with each other. The comparison principle is to compare the ASCII code values from the first character back, and finally output them in ascending order.

grammar

Sort [- fbmnrtuk] [-o < output file >] [- T < split character >] [+ < start field > - < end field >] [file or stdin]

Common parameters

-f ignore case
-b ignore the space character at the beginning of each line
-M sorts by month name
-m integrate several sorted files
-n sort using the size of numeric values
-O < output file > save the sorted results to the specified file
-r descending order, default ascending order
-c check whether the documents have been sorted in order
-u uniq, de duplication with - c, strict verification, if not with - c, only one sorting result is output, which is generally replaced by uniq.
-T < separator character > specifies the field separator character used in sorting, which is separated by [Tab] by default
-k select the column or the character of the column to sort

case

All personal accounts are recorded in / etc/passwd, and the accounts are sorted

[ranan@c105 ~]$ cat /etc/passwd | sort

/The content of etc/passwd is divided by: we use the third column

[ranan@c105 ~]$ cat /etc/passwd | sort -t ':' -k 3
# Arrange by number
[ranan@c105 ~]$ cat /etc/passwd | sort -t ':' -k 3 -n

The default is ASCII

Sort by number

Assume three digits, sorted by ten digits from small to large, and single digits from large to small.

[ranan@c105 ~]$ cat number.txt
991
454
522
258
322
[ranan@c105 ~]$ sort -n -k 1.2 -k 1.3nr number.txt # Why not? Wrong writing
322
522
454
258
991
[ranan@c105 ~]$ sort -n -k 1.2,1.2 -k 1.3nr number.txt #The second character in the first column, from small to large, the third character in the first column, from large to small, where r is - r
322
522
258
454
991

Note that the first way of sorting here is wrong. The End part is omitted, which means that you will sort the string from the second letter to the last character of the field. If the End part is not set, it is considered that End is set as the End of the line, but only ten bits are required from small to large, so it needs to be limited.

Output the sorting results to the source file

[ranan@c105 ~]$ cat number.txt | sort -k 1.2 > number.txt
[ranan@c105 ~]$ cat number.txt #number.txt is empty, indicating that the redirection method is not available
# Use - o
[ranan@c105 ~]$ cat number.txt 
34
45
52
29
91
[ranan@c105 ~]$ cat number.txt | sort -k 1.2 -o number.txt
[ranan@c105 ~]$ cat number.txt
91
52
34
45
29

-Combination of u and - k

-u only identifies the fields set with - k. if they are found to be the same, the subsequent same lines will be deleted.

If there are two levels of sorting, - u will weigh all - k options, and will delete those that are the same. As long as there is a level different, they will not be deleted easily.

Note: if cross domain is encountered, the cross domain setting is an illusion and cross domain comparison will not be performed

[ranan@c105 ~]$ cat facebook.txt 
google 110 5000
baidu 100 5000
guge 50 3000
sohu 100 4500
[ranan@c105 ~]$ sort -n -k 2 facebook.txt 
guge 50 3000
baidu 100 5000
sohu 100 4500
google 110 5000
[ranan@c105 ~]$ sort -n -k 2 -u facebook.txt 
guge 50 3000
baidu 100 5000
google 110 5000

uniq de duplication

If the sorting is completed, only one display will be listed for the duplicate data. uniq can be used

grammar
uniq [-ic]

-i ignore case
-c count the repeated occurrences
-u displays only non repeating rows
-d displays only duplicate rows

uniq is best used in conjunction with sorting, because unqi will only delete adjacent duplicate rows

[ranan@c105 ~]$ last | cut -d ' ' -f 1
ranan
reboot
ranan
ranan
[ranan@c105 ~]$ last | cut -d ' ' -f 1 | uniq
ranan
reboot
ranan
[ranan@c105 ~]$ last | cut -d ' ' -f 1 | sort |uniq

ranan
reboot

wc statistics

wc can count the number of words, lines and characters in a file, which can help us calculate the overall data of the output information

grammar
wc [-lwm]

-l train travel
-w how many words (English letters) are listed
-m how many characters

be careful
If only wc is used, it is output in lwm order by default
You can't connect files directly behind!, You can use standard input

wc -l < nowcoder.txt

Tags: Linux shell

Posted on Sun, 05 Dec 2021 16:29:22 -0500 by willeadie