Learn the new after reviewing the past, let’s review the awk tool today

Overview

• Awk is an excellent text processing tool, one of the most powerful data processing engines available in Linux and Unix environments. The new version nawk, gawk generated by improved awk, now the default linux system is daily use gawk
  
• Awk function: style loading, flow control, mathematical operators, process control statements and even built-in variables and functions can be carried out
• There are two special expressions in awk: BEGIN and END. Both of these can be used in the pattern. The purpose of providing BEGIN and END is to give the program an initial state and perform some finishing work after the program ends. Any operations listed after BEGIN (within {}) will be performed before awk starts scanning for input, and operations listed after END will be performed after scanning all input. Therefore, BEGIN is usually used to display variables and preset (initialized) variables, and END is used to output the final result

working principle

• Read text line by line, separated by space or tab key separator by default, save the separated fields to built-in variables, and execute editing commands according to the mode or condition
• The sed command is often used to process a whole line, while the awk command tends to divide a line into multiple "fields" and then process it.
• The reading of awk information is also read line by line, and the execution result can be printed and displayed through the print function.
• In the process of using the awk command, you can use the logical operator "&&" to mean "and", "||" means "or", "!" means "not, no"; it can also be simple Mathematical operations, such as "+", "-", "*", "/", "%", "^" represent the meaning of addition, subtraction, multiplication, division, remainder and power respectively

Command format

• awk option'mode or condition {operation}' file 1 file 2...
• awk -f script file file 1 file 2…
  

Common built-in variables of awk (can be used directly) are shown in the following table

variable	Description
FS	Column separator specifies the field separator for each line of text, the default is a space or a tab character, which has the same effect as "-F"
RS	When the line separator awk reads data from the file, it will cut the data into many records according to the definition of RS, and awk only reads one record at a time for processing. The default value is'\n'
NF	The number of fields in the current processing line
NO	The line number (ordinal) of the currently processed line
$0	The entire line of the current processing line
$n	The nth field (column n) of the currently processed row
FILENAME	File name being processed

Common built-in variable command emoticons

Output text by field (command analysis)

[root@localhost ~]# awk -F ":" '!($3<200) {print $1,$3}' /etc/passwd
【输出第3个字段的值不小于200的行中第1个和第3个字段内容】
polkitd 999
libstoragemgmt 998
colord 997
saslauth 996
chrony 995
nfsnobody 65534
geoclue 994
setroubleshoot 993
sssd 992
gnome-initial-setup 991
qz 1000
[root@localhost ~]# awk -F ":" '$3<6{print $1,$3}' /etc/passwd
【输出第3个字段的值小于6的第1个和第3个字段内容】
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
[root@localhost ~]# awk -F ":" '{print $1,$3}' /etc/passwd
【输出每行中的第1个和第3个字段】
root 0
bin 1
...略...
...略...
postfix 89
tcpdump 72
qz 1000
[root@localhost ~]# awk -F "/" '{print $2}' /etc/passwd
【输出每行中以“/”分隔的第2个字段】
root:
bin:
...略...
...略...
var
:
home
[root@localhost ~]# awk -F ":" '{print $1}' /etc/passwd
【输出每行中以“:”分隔的第1个字段】
root
bin
...略...
...略...
qz
[root@localhost ~]# awk 'BEGIN {FS="/"};{if($3>1000){print$2,$3}}' /etc/passwd
【先处理完BEGIN的内容，将每行中的默认分隔符更换成以“/”分隔，再打印出当第3列值小于1000的行中的第2列和第3列内容】
root: bin
bin: sbin
...略...
...略...
var spool
: sbin
home qz:
[root@localhost ~]# awk -F ":" '{qz=($3>=$4)?$3:$4;{print qz}}' /etc/passwd
【($3>=$4)?$3:$4是三元运算符，如果第3个字段的值≥第4个字段的值，则把第3个字段的值赋给qz,否则把第4个字段的值赋给qz，
最后打印出被赋值后的变量qz
“?”代表当前面“$3>=$4”条件成立则输出"?"后面的语句
":"代表当前面“$3>=$4”条件不成立则输出":"后面的语句】
0
1
2
...略...
...略...
72
1000
[root@localhost ~]# awk -F ":" '$7~"/bash"{print $1}' /etc/passwd
【输出以冒号分隔且第7个字段中包含/bash的行里的第1个字段】
root
qz
[root@localhost ~]# awk -F ":" '{print NR,$0}' /etc/passwd
【输出内容的内容和行号。每次处理完一条记录，NR的值都会加1】
1 root:x:0:0:root:/root:/bin/bash
2 bin:x:1:1:bin:/bin:/sbin/nologin
...略...
...略...
40 tcpdump:x:72:72::/:/sbin/nologin
41 qz:x:1000:1000:qz:/home/qz:/bin/bash
[root@localhost ~]# awk -F ":" '($1~"root")&&(NF==7){print $1,$2}' /etc/passwd
【输出第1个字段中包含root字符且共有7个字段的行里面的第1个和第2个字段】
root x
[root@localhost ~]# awk -F ":" '($7!="/bin/bash")&&($7!="/sbin/nologin"){print}' /etc/passwd
【输出第7个字段中即不为/bin/bash也不是/sbin/nologin字符的所有行】
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt

Invoke Shell commands through pipe symbols and double quotes

[root@localhost ~]# echo $PATH | awk 'BEGIN{RS=":"};END{print NR}'
【先echo出$PATH的内容，然后统计以“:”分隔的文本段落数，END{
    
    }语句块中，通常会放入打印结果等语句】
5
[root@localhost ~]# echo $PATH 
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin
[root@localhost ~]# awk -F: '/nologin$/{print | "wc -l"}' /etc/passwd
【调用“wc -l”命令统计以nologin结尾的用户个数，等同于grep -c "nologin$"】
36
[root@localhost ~]# grep -c "nologin$" /etc/passwd
36

View the current memory idle rate, CPU usage and restart time

[root@localhost ~]# top -b -n 1 | grep Cpu | awk -F ',' '{print $4}' | awk '{print $1}'
【查看当前CPU空闲率，（-b -n 1表示只需要1次的输出结果）】
100.0
[root@localhost ~]# free -m | awk '/Mem:/{print int($3/($3+$4)*100)"%"}'
【查看当前内存使用百分比】
21%
[root@localhost ~]# date -d "$(awk -F "." '{print $1}' /proc/uptime) second ago" +"%F %H:%M:%S"
2021-03-16 08:44:48
[root@localhost ~]# init 6
[root@localhost ~]# date -d "$(awk -F "." '{print $1}' /proc/uptime) second ago" +"%F %H:%M:%S"
2021-03-16 11:30:16
【显示系统上次系统重启时间，等同于uptime；second ago为显示多少秒前的时间；+"%F %H:%M:%S"等同于+"%Y-%M-%D %H:%M:%S"的时间格式】

Count the number of online users and the current hostname

[root@localhost ~]# awk 'BEGIN {q=0;while ("w" | getline)q++;{print q-2}}'
1
[root@localhost ~]# awk 'BEGIN {q=0;while ("w" | getline)q++;{print q-2}}'
2
[root@localhost ~]# awk 'BEGIN {q=0;while ("w" | getline)q++;{print q-2}}'
3
【调用w命令来统计当前在线用户数】
[root@localhost ~]# awk 'BEGIN {"hostname" | getline;{print $0}}'
localhost.localdomain
【调用hostname命令输出当前的主机名】
[root@localhost ~]# hostname
localhost.localdomain
[root@localhost ~]# seq 5 | awk '{getline;print $0}'
2
4
5
[root@localhost ~]# seq 5 | awk '{print $0;getline}'
1
3
5
【当getline左右无重定向符“<”或“|”时，awk首先读取到第一行，就是1，然后getline就得到了1下面的第二行，就是2.因为getline
之后，awk会改变对应的NF、NR、FNR和$0等内部变量，所以此时的$0的值就不再是1，而是2了，然后将它打印出来】
【当getline左右有重定向符“<”或“|”时，getline则作用于定向输入文件，由于该文件是刚打开，还没有被awk读入一行，只是getline
读入，所以getline返回的则是该文件的第一行，而不是隔行】