SHELL script - text three swordsman

SHELL script - text three swordsman

grep

Common options

optiondescribesupplement
-EExtended regular (ERE)–extended–regexp
-PBuilt in regularPerl
-iignore case–ignore–case
-nPrint line number–line–number
-oPrint only matching content–only–matching
-cPrint only the number of lines that match each file–count
-BPrint the first few lines that match–before–context=NUM
-APrint the last few lines that match–after–context=NUM
-CPrint the first and last lines of the match–context=NUM
–color=autoHighlight matchesDefault settings
-vReverse to print unmatched lines–invert–match
-eMultipoint operation

Use with regular expressions

Basic regularity

Symboldescriberemarks
.Match any single character (must exist)
^Matches a line that begins with a character
$A line ending with what character
^$Match blank lines
*Matches the previous character 0 or more times
.*Represents any character of any length
[list]Match any single character in the list
[ ^list]Match any single character in the list
^[list]Matches characters that begin with any single character in the list
^[ ^list]Matches the beginning of any single character except the list
<What does it start withThere are in the symbol\
>What does it end withThere are in the symbol\
<>Exact matchThere are in the symbol\
{n}Matches the preceding character n consecutive timesThere are in the symbol\
{n,}Matches the preceding character at least n timesThere are in the symbol\
{m,n}Indicates that the matching character appears at least m times and at most n timesThere are in the symbol\
(strings)Save the matched characters, which can be replaced by the label \ 1There are in the symbol\

Perl built-in regular (- P)

Symboldescriberemarks
\dMatch number[0-9]
\wMatch alphanumeric underscores[a-zA-Z0-9_]
\sMatch spaces, tabs, page breaks[\t\r\n]

Extended regular expression (- E)

Symboldescriberemarks
+Matches one or more preceding characters
?Matches zero or one preceding character
|or
()Group character(taobao|baidui).com
{n}The leading character is repeated n timesNot in symbol\
{n,}The leading character is repeated at least n timesNot in symbol\
{n,m}The leading character is repeated n to m timesNot in symbol\

Case: remove all comments and blank lines when viewing the configuration file

[root@server1 ~]# grep -Ev "^#|^$" /etc/ssh/sshd_config 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server

sed

summary

  • You can edit files non interactively, which is often used in shell scripts
  • sed saves the lines currently being processed in a temporary buffer (also known as mode space), and then processes the lines in the temporary buffer without directly modifying the original file (if you want to modify the original file, you can add corresponding parameters). After completion, it sends the processed lines to the screen.

grammar

Command line usage

sed [option]  'Address location|sed Command of' filename

Common options
-e	Make multiple edits
-n	Cancel default output
-f	appoint sed The file name of the script
-r  Using extended regular expressions
-i	Modify source file

Common commands
p	Print line
d	Delete row
i	Insert on the previous line of the current line
$i	Insert on the last line
a	Insert on the next line of the current line
$a	Insert (append) on the next line of the last line
c	Replace entire line
r	read
w	Save as
!	Reverse
s	Find replace
g	Replace whole line
&	Replace the contents of the reference lookup string in the string
=	Print line number
[root@server1 ~]# head -5 /etc/passwd > test.txt

print document
[root@server1 ~]# sed 'p' test.txt 
root:x:0:0:root:/root:/bin/bash
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[root@server1 ~]# sed -n 'p' test.txt #Cancel default output
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

Print 1-3 that 's ok
[root@server1 ~]# sed -n '1,3p' test.txt  
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin

Delete row
[root@server1 ~]# sed  '1d' test.txt  
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[root@server1 ~]# sed -n '1d' test.txt  #Cancel default output

Single line insertion
[root@server1 ~]# sed  '1ihello' test.txt 
hello
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

Multiline insertion
[root@server1 ~]# sed  '1ihello\nworld' test.txt 
hello
world
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[root@server1 ~]# sed  '1ihello\
> linux\
> world' test.txt
#Insert multiple lines with newline

Add
[root@server1 ~]# sed  '$ahello' test.txt 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
hello

replace
[root@server1 ~]# sed  '1chello world' test.txt 
hello world
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

Match replacement
#Matching syntax: / matching content/
[root@server1 ~]# sed  '/bin/chello' test.txt #Find the line with bin and replace with hello
hello
hello
hello
hello
hello

read
[root@server1 ~]# sed  '$r /etc/hosts' test.txt 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.226.40 admin 
192.168.226.10 node1 
192.168.226.20 node2 
192.168.226.30 node3 

Save as
[root@server1 ~]# sed '1,3w 111' test.txt 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[root@server1 ~]# cat 111 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
[root@server1 ~]# sed -n  '/root/w 111' test.txt #Save the row with root in 111
[root@server1 ~]# cat 111 
root:x:0:0:root:/root:/bin/bash

Reverse
[root@server1 ~]# sed -n  '1,3!p' test.txt 
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

Find replace
[root@server1 ~]# sed -n  's/root/ROOT/p' test.txt #Replace first by default
ROOT:x:0:0:root:/root:/bin/bash
[root@server1 ~]# sed -n  's/root/ROOT/gp' test.txt #Replace whole line
ROOT:x:0:0:ROOT:/ROOT:/bin/bash

Custom delimiter (generally used to find or replace content)/Separator (when)
[root@server1 ~]# sed -n  's#/sbin/nologin#/bin/bash#gp' test.txt 
bin:x:1:1:bin:/bin:/bin/bash
daemon:x:2:2:daemon:/sbin:/bin/bash
adm:x:3:4:adm:/var/adm:/bin/bash
lp:x:4:7:lp:/var/spool/lpd:/bin/bash

Batch notes
[root@server1 ~]# sed -n  '1,3s/^/#/p' test.txt 
#root:x:0:0:root:/root:/bin/bash
#bin:x:1:1:bin:/bin:/sbin/nologin
#daemon:x:2:2:daemon:/sbin:/sbin/nologin

Remove all special characters from the file
[root@server1 ~]# sed -n  '1,3s/[/:]//gp' test.txt 
rootx00rootrootbinbash
binx11binbinsbinnologin
daemonx22daemonsbinsbinnologin

Save the lookup string for reference in the replacement string
[root@server1 ~]# sed -n  '1,3s/\(root\)/\1 hello/gp' test.txt 
root hello:x:0:0:root hello:/root hello:/bin/bash
[root@server1 ~]# sed -n  '1,3s/root/& hello/gp' test.txt 
root hello:x:0:0:root hello:/root hello:/bin/bash

Print line number
[root@server1 ~]# sed -n '/bash$/= ' test.txt 
1

Multieditor
[root@server1 ~]# sed -ne '/bash$/= ' -ne '/bash$/p' test.txt 
1
root:x:0:0:root:/root:/bin/bash

Modify original document
#If the - n option cannot be added, the file will be emptied
#If you cannot add the p command, the file will have more lines to print
[root@server1 ~]# sed -i 's/root/ROOT/g' test.txt 
[root@server1 ~]# cat test.txt 
ROOT:x:0:0:ROOT:/ROOT:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
Options / commandsexplainusage
-nCancel default output
-eMake multiple editssed -ne '/nologin$/=' -ne '/nologin$​/p' 1.txt
-fspecify a filenameNo
-rExtended regularity
-iModify source fileCannot be used with the - i option and the p command
pPrint linesed -n '1,5p' a.txt
dDelete rowsed -n '1d' a.txt
iInsert on the previous line of the current linesed '1ihello\nworld' test.txt
$iInsert on the last line
aInsert on the next line of the current linesed '5ahello' a.txt
$aAdd
cReplace entire linesed '1,5chello world' a.txt
rreadsed '3r /etc/hosts' 2.txt
wSave assed '/root/w a.txt' 2.txt
!Reversesed -n '1!p' 1.txt
sFind replacesed -n 's/root/ROOT/p' 1.txt
gReplace whole linesed -n 's/root/ROOT/gp' 1.txt
&Replace the contents of the reference lookup string in the stringsed -n 's/root/#&/p' a.txt
=Print line numbersed -n '/bash$/=' 1.txt

case

  • Remove all comments and blank lines when viewing the configuration file
Method 1
[root@server1 ~]# sed -e '/^#/d' -e '/^$/d' /etc/ssh/sshd_config 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server

Method 2
[root@server1 ~]# sed '/^#/d;/^$/d' /etc/ssh/sshd_config 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server

Method 3
[root@server1 ~]# sed -r '/^#|^$/d' /etc/ssh/sshd_config 
HostKey /etc/ssh/ssh_host_rsa_key
HostKey /etc/ssh/ssh_host_ecdsa_key
HostKey /etc/ssh/ssh_host_ed25519_key
SyslogFacility AUTHPRIV
AuthorizedKeysFile	.ssh/authorized_keys
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials no
UsePAM yes
X11Forwarding yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
Subsystem	sftp	/usr/libexec/openssh/sftp-server
  • Get IP address
Method 1
[root@server1 ~]# ifconfig ens33 |sed '2!d'
        inet 192.168.226.10  netmask 255.255.255.0  broadcast 192.168.226.255
[root@server1 ~]# ifconfig ens33 |sed '2!d'|cut -d' ' -f10,13,16
192.168.226.10 255.255.255.0 192.168.226.255

Method 2
[root@server1 ~]# ifconfig ens33 |sed '2!d'|sed -n 's/ /\n/gp'|sed '/^$|[a-z]/d'

inet
192.168.226.10

netmask
255.255.255.0

broadcast
192.168.226.255
[root@server1 ~]# ifconfig ens33 |sed '2!d'|sed -n 's/ /\n/gp'|sed -r '/^$|[a-z]/d'
192.168.226.10
255.255.255.0
192.168.226.255

Used in scripts

  • When writing sed script, you need to add its corresponding magic characters (command interpreter)
  • Just write the content between '······'
  • One command per line
  • Note that there should be no spaces after the command
[root@server1 ~]# vim sed.sh
#!/bin/sed -f
1,3d
s/root/hello/g
2iworld
[root@server1 ~]# cat test.txt 
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[root@server1 ~]# sed -f sed.sh test.txt 
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

[root@server1 ~]# sed -f sed.sh -i test.txt #Modify original document
[root@server1 ~]# cat test.txt 
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin

awk

summary

  • Awk is a programming language. The basic function of awk language is to browse and extract information based on specified rules in files or strings. After awk extracts information, other text operations can be carried out.
  • Complete awk scripts are often used to format information in text files.
  • Generally, awk is based on the behavior of files. Awk receives each line of the file and then executes the corresponding command to process the text
  • awk represents the first letter of the last names of the three authors, Alfred Aho, Brian Kernighan and Peter Weinberger.
  • Gawk is an open source version of awk. Awk has been linked to gawk in linux system.
[root@server1 ~]# ll /bin/awk
lrwxrwxrwx. 1 root root 4 7 June 24-17:32 /bin/awk -> gawk

Command mode usage

grammar

awk option 'Regular, address status{awk sentence}' file name

Common options
-F	Specify a separator. You can specify one or more. The default separator is a space
-v	Define variables and assign values
  • ’'command' is written in standard

    Regular matching
    '/^root/{awk sentence}' #Match start with root
    [root@server1 ~]# cat test.txt 
    root:x:0:0:root:/root:/bin/bash
    bin:x:1:1:bin:/bin:/sbin/nologin
    daemon:x:2:2:daemon:/sbin:/sbin/nologin
    adm:x:3:4:adm:/var/adm:/sbin/nologin
    lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
    [root@server1 ~]# awk -F: '/^root/{print $0}' test.txt 
    root:x:0:0:root:/root:/bin/bash
    
    Address location
    'NR==1,NR==5{awk sentence}' #Match lines 1-5
    [root@server1 ~]# awk -F: 'NR==3{print $0}' test.txt 
    daemon:x:2:2:daemon:/sbin:/sbin/nologin
    
    Multiple awk Statement use;separate
    [root@server1 ~]# awk -F: 'NR==3{print $1,$3}' test.txt 
    daemon 2
    [root@server1 ~]# awk -F: 'NR==3{print $1;print $3}' test.txt 
    daemon
    2
    
    BEGIN END use
    'BEGIN{awk sentence};{Processing};END{awk sentence}'
    'BEGIN{awk sentence};{Processing}'
    '{Processing};END{awk sentence}'
    [root@server1 ~]# awk -F: 'BEGIN{print "Login_shell\tLogin_home\n********"};{print $NF"\t\t"$(NF-1)};END{print "***********"}' /etc/passwd
    Login_shell	Login_home
    ********
    /bin/bash		/root
    /sbin/nologin		/bin
    /sbin/nologin		/sbin
    /sbin/nologin		/var/adm
    /sbin/nologin		/var/spool/lpd
    /bin/sync		/sbin
    /sbin/shutdown		/sbin
    /sbin/halt		/sbin
    /sbin/nologin		/var/spool/mail
    /sbin/nologin		/root
    /sbin/nologin		/usr/games
    /sbin/nologin		/var/ftp
    /sbin/nologin		/
    /sbin/nologin		/
    /sbin/nologin		/
    /sbin/nologin		/
    /sbin/nologin		/var/empty/sshd
    /sbin/nologin		/var/spool/postfix
    /sbin/nologin		/usr/share/httpd
    /sbin/nologin		/dev/null
    /sbin/nologin		/var/lib/zabbix
    /bin/bash		/home/cephu
    /sbin/nologin		/etc/ntp
    ***********
    

Internal variable

variableVariable descriptionremarks
$0All records of the current processing line
$1,$2,$3...$nDifferent fields in the file that are separated by an interval symbol for each line'{print $1,$3}'
NFNumber of fields (columns) of the current record'{print NF}'
$NFLast column$(NF-1) indicates the penultimate column
NRLine number'NR==2{print $7}'
FNRLine number (same as above)
FSDefine input field separator, default space'BEGIN{FS=":"};{print $1,$3}'
OFSDefine output field separator, default space'BEGIN{OFS="\t"};print $1,$3}'
RSDefines the input record delimiter, which defaults to line feed'BEGIN{RS="\t"};{print $0}'
ORSDefines the output record delimiter, which defaults to line feed'BEGIN{ORS="\n\n"};{print $1,$3}'
FILENAMECurrently entered file name
Define input separator
#Method 1: specify option - F
[root@server1 ~]# awk -F: '{print $1,$NF}' test.txt 
root /bin/bash
bin /sbin/nologin
daemon /sbin/nologin
adm /sbin/nologin
lp /sbin/nologin
#Method 2: with the help of internal variable FS
[root@server1 ~]# awk 'BEGIN{FS=":"};{print $1,$NF}' test.txt 
root /bin/bash
bin /sbin/nologin
daemon /sbin/nologin
adm /sbin/nologin
lp /sbin/nologin

Define output separator
#default
[root@server1 ~]# awk -F: '{print $1,$NF}' test.txt
root /bin/bash
bin /sbin/nologin
daemon /sbin/nologin
adm /sbin/nologin
lp /sbin/nologin
#Method 1: specify between fields by writing between ""
[root@server1 ~]# awk -F: '{print $1"\t"$NF}' test.txt 
root	/bin/bash
bin	/sbin/nologin
daemon	/sbin/nologin
adm	/sbin/nologin
lp	/sbin/nologin
#Method 2: with the help of internal variable OFS
[root@server1 ~]# awk -F: 'BEGIN{OFS="\t"};{print $1,$NF}' test.txt 
root	/bin/bash
bin	/sbin/nologin
daemon	/sbin/nologin
adm	/sbin/nologin
lp	/sbin/nologin

Define input record separator
[root@server1 ~]# vim 1.txt
  1 123^I456^I789$
  2 234 567$
  3 sdvffrrvf$
  4 ddfd sdg^Ifhh^Idd$
[root@server1 ~]# awk 'BEGIN{RS="\t"};{print $0}' 1.txt  #The default line break is still in effect
123
456
789
234 567
sdvffrrvf
ddfd sdg
fhh
dd

Define output record separator
[root@server1 ~]# awk '{print $0}' 1.txt #default
123	456	789
234 567
sdvffrrvf
ddfd sdg	fhh	dd
[root@server1 ~]# awk 'BEGIN{ORS="\t"};{print $0}' 1.txt
123	456	789	234 567	sdvffrrvf	ddfd sdg	fhh	dd	[root@server1 ~]# 

[external chain picture transfer failed. The source station may have anti-theft chain mechanism. It is recommended to save the picture and upload it directly (img-lqbgtnrm-163214794425) (... /% E5%9B%BE%E7%89%87%E6%94%BE%E7%BD%AE%E7%82%B9/image-20210918154138794.png)]

Workflow

Take awk -F: '{print ,$NF}' /etc/passwd as an example

  • awk takes a line as input and assigns it to the internal variable $0. Each line is a record and ends with a newline character (RS)
  • Each line is broken down into fields by the spacer colon (FS), and each field is stored in a numbered variable, starting at $1 and ending at $NF
  • awk uses the print function to print fields. The printed fields are separated by spaces because there is a comma between $1 and $NF. Comma is special. It is mapped to another internal variable OFS. OFS defaults to space
  • After awk processes one line, it will get another line from the file and store it in $0, overwrite the original content, and then separate the new string into fields and process it. This process will continue until all rows are processed

Combined with regular use

operatorexplain
==be equal to
!=Not equal to
>greater than
<less than
>=Greater than or equal to
<=Less than or equal to
~matching
!~Mismatch
!Logical non
&&Logic and
||Logical or
From lp The first line matches to line 10  
[root@server1 ~]# awk '/^lp/,NR==10{print $0}' /etc/passwd
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin

Print 6-10 that 's ok
[root@server1 ~]# awk 'NR>=6 && NR<=10{print $0}' /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin

Print 6-10 OK, to nologin Ending line
[root@server1 ~]# awk 'NR>=6 && NR<=10 && /nologin$/{print $0}' /etc/passwd
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin

Format output printf

%s	Character type
%d	value type
-	Indicates left alignment without default right alignment
\n	Line feed, printf The default line end does not wrap
[root@server1 ~]# awk -F: '{printf "%-15s %-20s\n", $NF,$(NF-1)}' /etc/passwd
/bin/bash       /root               
/sbin/nologin   /bin                
/sbin/nologin   /sbin               
/sbin/nologin   /var/adm            
/sbin/nologin   /var/spool/lpd      
/bin/sync       /sbin               
/sbin/shutdown  /sbin               
/sbin/halt      /sbin               
/sbin/nologin   /var/spool/mail     
/sbin/nologin   /root               
/sbin/nologin   /usr/games          
/sbin/nologin   /var/ftp            
/sbin/nologin   /                   
/sbin/nologin   /                   
/sbin/nologin   /                   
/sbin/nologin   /                   
/sbin/nologin   /var/empty/sshd     
/sbin/nologin   /var/spool/postfix  
/sbin/nologin   /usr/share/httpd    
/sbin/nologin   /dev/null           
/sbin/nologin   /var/lib/zabbix     
/bin/bash       /home/cephu         
/sbin/nologin   /etc/ntp            

Script programming

Script execution

[root@server1 ~]# vim awk.sh
#!/bin/awk -f

#Just write the content in ''
BEGIN{FS=":"}
NR==1,NR==3{print $1"\t"$NF}

[root@server1 ~]# awk -f awk.sh /etc/passwd
root	/bin/bash
bin	/sbin/nologin
daemon	/sbin/nologin

Define variables

[root@server1 ~]# awk -v num=1 '{print num}' 1.txt 
1
1
1
1
[root@server1 ~]# awk -v num=1 'BEGIN{print num}'
1

Select execute

awk option 'Regular, address location{if((condition){Statement 1; Statement 2;...}}' file name
awk option 'Regular, address location{if((condition){Statement 1; Statement 2;...} else{Statement 1; Statement 2;...}}' file name
awk option 'Regular, address location{if(Condition 1){Statement 1; Statement 2;...} else if(Condition 2){Statement 1; Statement 2;...} else{Statement 1; Statement 2;...}}' file name

Case: print the of ordinary users UID And user name
[root@server1 ~]# awk -F: '{if($3>=1000){print $1,$3}}' /etc/passwd
cephu 1000
zhnagsan 1001
lisi 1002
[root@server1 ~]# Awk - F: '{if ($3 > = 1000) {print $1 "\ T" $3} else {print $1 "\ t non ordinary user"}}' / etc/passwd
root	Non ordinary users
bin		Non ordinary users
daemon	Non ordinary users
adm		Non ordinary users
lp		Non ordinary users
sync	Non ordinary users
shutdown Non ordinary users
halt	Non ordinary users
mail	Non ordinary users
operator Non ordinary users
games	Non ordinary users
ftp		Non ordinary users
nobody	Non ordinary users
systemd-network	Non ordinary users
dbus	Non ordinary users
polkitd	Non ordinary users
sshd	Non ordinary users
postfix	Non ordinary users
apache	Non ordinary users
tss		Non ordinary users
zabbix	Non ordinary users
cephu	1000
zhnagsan	1001
lisi	1002
[root@server1 ~]# Awk - F: '{if ($3 > = 1000) {print $1 "\ tnormal user"} else if ($3 = = 0) {print $1 "\ tadministrator"} else {print $1 "\ tsystem user"}' / etc/passwd
root	administrators
bin		System user
daemon	System user
adm		System user
lp		System user
sync	System user
shutdown System user
halt	System user
mail	System user
operator System user
games	System user
ftp		System user
nobody	System user
systemd-network	System user
dbus	System user
polkitd	System user
sshd	System user
postfix	System user
apache	System user
tss		System user
zabbix	System user
cephu	Ordinary users
ntp		System user
zhnagsan Ordinary users
[root@server1 ~]# awk -F: '{if($3>=1000){i++} else if($3==0){j++} else{k++}}; End {print J "administrators \ n"k "system users \ n"i "ordinary users"} '/ etc/passwd
1 Administrators
21 System users
3 Ordinary users

Circular statement

Print 1,3,5,7,9
[root@server1 ~]# awk 'BEGIN{for(i=1;i<=10;i+=2){print i}}'
1
3
5
7
9
[root@server1 ~]# awk 'BEGIN{i=1;while(i<=10){print i;i+=2}}'
1
3
5
7
9

arithmetic operation

Decimal operations can be performed
+ - * / %(model) **(power)Can be calculated
[root@server1 ~]# awk 'BEGIN{print 2**3}'
8
[root@server1 ~]# awk 'BEGIN{print 2/3}'
0.666667

case

Print ip address
[root@server1 ~]# ifconfig ens33 |awk 'NR==2{print $2}'
192.168.226.10
[root@server1 ~]# ifconfig ens33 |awk 'NR==2{print $2"\n"$4"\n"$6}'
192.168.226.10
255.255.255.0
192.168.226.255

Statistics/etc/passwd Various types in shell Number of
[root@server1 ~]# awk -F: '{shells[$NF]++};END{for(i in shells){print i,shells[i]}}' /etc/passwd
/bin/sync 1
/bin/bash 4
/sbin/nologin 18
/sbin/halt 1
/sbin/shutdown 1

Tags: Linux Operation & Maintenance regex awk perl

Posted on Mon, 20 Sep 2021 20:54:24 -0400 by dapuxter