Three Musketeers shell awk
In Linux / UNIX systems, awk is a powerful tool for editing, the input text is read line by line, and to find the matching according to the specified pattern to meet the requirements of the output format or content filtering process can not interact the case of implementing fairly complex text manipulation, are widely used in Shell script to complete a variety of automated configuration tasks.
Awk command format used is shown below, wherein the single quotation marks plus braces "{}" of the processing operation for setting the data. awk can deal directly with the target file, the target file can be processed by the "-f" read the script.
awk command format tool
awk 选项 '模式或条件 {编辑指令}' 文件 1 文件 2 „ //过滤并输出文件符条件的内容
awk -f 脚本文件 文件 1 文件 2 „ //从脚本中调用编辑指令,过滤并输出内容
7awk tend splitting a line into a plurality of "field" and then processed, and the default field in the case where the separator is a space or tab keys. awk execution result can be displayed by the print data printing field function. In using awk command can use the logical operator "&&" indicates "and", "||" indicates "or" means "not", "!"; Also can perform simple mathematical operations, such as +, -, *, /,%, ^ respectively addition, subtraction, multiplication, division, exponentiation, and take the remainder.
Find / etc / passwd user name, user ID, group ID, column, execute the following command to awk.
[root@localhost ~]# awk -F : '{print $1,$3,$4}' /etc/passwd
root 0 0
bin 1 1
daemon 2 2
adm 3 4
lp 4 7
sync 5 0
shutdown 6 0
halt 7 0
mail 8 12
operator 11 0
games 12 100
ftp 14 50
nobody 99 99
systemd-network 192 192
dbus 81 81
polkitd 999 998
abrt 173 173
libstoragemgmt 998 996
rpc 32 32
colord 997 995
saslauth 996 76
rtkit 172 172
pulse 171 171
chrony 995 991
rpcuser 29 29
nfsnobody 65534 65534
ntp 38 38
tss 59 59
usbmuxd 113 113
geoclue 994 989
qemu 107 107
radvd 75 75
setroubleshoot 993 988
sssd 992 987
gdm 42 42
gnome-initial-setup 991 986
sshd 74 74
avahi 70 70
postfix 89 89
tcpdump 72 72
chen 1000 1000
awk reads from the standard input file or input information, and as sed, the information is read line by line read. Awk except that the line of the text file as a record, and a part of the row (column) as a field (field) in a record. In order to operate these various fields, a method shell similar to the position of the variable awk borrowed by $ 1, $ 2, $ 3 "sequentially showing lines of different fields (records). Further awk by $ 0 represents the entire row (record). Different between the fields are separated by a specified .awk default character delimiter is a space to allow .awk command line with the "-F separator" specified in the form delimiters.
awk contains several special built-in variables (which can be directly used) as follows:
(. 1) the FS: Specify each line of text field separator, the default position is a space or tab.
(2) NF: number of columns of the row currently being processed.
(3) NR: line number of the currently processed (ordinal).
(4) $ 0: the entire line of the row currently being processed.
(6) FILENAME: File name to be processed.
(7) RS9: recording data separated by default \ n-, i.e. the behavior of each record.
Use a .awk
(1) outputs all the contents, is equivalent to the cat chen.txt,
[root@localhost ~]# awk '{print $0}' chen.txt
[root@localhost ~]# awk '{print}' chen.txt //这条命令和上面一条是一样的效果
#version=DEVEL
#System authorization information
aulth --enableshadow --passalgo=sha512
#Use CDROM installation media
cdlrom.
thethethe.(2) output lines 1 to 3 content
[root@localhost ~]# awk 'NR==1,NR==3{print}' chen.txt //这条命令和下面命令是一样的效果
[root@localhost ~]# awk '(NR>=1)&&(NR<=3){print}' chen.txt
#version=DEVEL
#System authorization information
aulth --enableshadow --passalgo=sha512Content (3) outputs all odd rows
[root@localhost ~]# awk '(NR%2)==1{print}' chen.txt
Use CDROM installation media
cdrom.
thethethe.
THE
THEASDHAS
Use graphical install.
graphical.
best
test
ASSDJFXYxyzC
AxyzxyzxyzC
keyboard --vckeymap=cn --xlayouts='cn'
System languageContent (4) outputs all even rows
[root@localhost ~]# awk '(NR%2)==0{print}' chen.txt
Use graphical install.
graphical.
best
test
ASSDJFXYxyzC
AxyzxyzxyzC
keyboard --vckeymap=cn --xlayouts='cn'
System language
lang zh_CN.UTF-8
Network information
network --bootproto=dhcp --device=ens33 --onboot=off --ipv6=auto --no-activate
network --hostname=localhost.localdomain
Root password
rootpw --iscrypted $6$L.egxzosoP/0k9Nj$wna7vPXZjeH0jFcNZUymYKF8ySXq5HxQuvxTFxIpEAAxuDj7MQJtXBds5E0LxAftI1H5JbJuYpN44d5n6t1AZ.
System servicesContent (5) outputs all even rows
awk '/^The/{print}' chen.txt
THE
THEASDHAS(6) output limit. Ending row
[root@localhost ~]# awk '/limit.$/{print}' chen.txt
Use graphical install limit.(7) In the statistical / bin rows / bash end equivalent to greo -c "/ bin / bash" chen.txt
[root@localhost ~]# awk 'BEGIN {x=0} ; /\/bin\/bash$/{x++};END{print x}' /etc/passwd
(8) In the statistical number of the paragraph-delimited text
[root@localhost ~]# awk 'BEGIN {RS="the"};END{print NR}' chen.txt
(B) the output text field
[root@localhost ~]# awk -F : '{print $3}' /etc/passwd
0
1
2
3
4
5
6
7
8
9
10
11(1) in each row of the output field 3
[root@localhost ~]# awk -F : '{print $1,$3}' /etc/passwd
root 0
bin 1
daemon 2
adm 3
lp 4
sync 5
shutdown 6
halt 7
mail 8
operator 11
games 12
ftp 14
nobody 99
systemd-network 192(2) Output code is "*" to record the user's shadow
[root@localhost ~]# awk -F : '$2== "*"{print}' /etc/shadow
bin:*:17110:0:99999:7:::
daemon:*:17110:0:99999:7:::
adm:*:17110:0:99999:7:::
lp:*:17110:0:99999:7:::
sync:*:17110:0:99999:7:::
shutdown:*:17110:0:99999:7:::
halt:*:17110:0:99999:7:::
mail:*:17110:0:99999:7:::
operator:*:17110:0:99999:7:::
games:*:17110:0:99999:7:::The first field (3) and the output divider colon seventh field contains / bash row
[root@localhost ~]# awk '($1~"nfs")&&(NF=7){print $1,$2}' /etc/passwd
nfsnobody:x:65534:65534:Anonymous NFS(4) the output of the seventh field is neither / bin / bash nor is / sbin / nologin all rows
[root@localhost ~]# awk -F : '($7!="/bin/bash")&&($7!="/sbin/nologin")' /etc/passwd
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt(Ii) through the pipeline, double quotes invoke shell commands
Wc -l command to call the number using statistical bash users, equivalent to grep -c "bash $" / etc / passwd
[root@localhost ~]# awk -F : '/bash$/{print | "wc -l"}' /etc/passwd
2(1) calls w command, and used to count the number of online users
[root@localhost ~]# awk 'BEGIN {while ("w" | getline) n++ ;{print n-2}}'
2
(2) call hostname, and the output current host name
[root@localhost ~]# awk 'BEGIN {"hostname" | getline ; print $0}'
localhost.localdomainThree .sort Tools
sort is a tool in units of the contents of the file to be sorted, it can also be sorted according to different data types. Such as data and character Board is not the same. The syntax sort command is "sort [options] parameters", which are commonly used options include the following.
(1) -f: ignore case;
(2) -b: Ignore spaces in front of each line;
(. 3) -M: sorted by month;
(. 4) -n: The digital sorting;
(. 5) -R & lt : reverse order;
(. 6) -u: equivalent to the uniq, showing the same data as the single line;
(. 7) -t: Specifies delimiter, using [Tab] key default partition;
(. 8) -o <output file> : the sorted results to the designated dump file;
(. 9) -k: Specifies the sorting area.
1: / etc / passwd file accounts sorted.
Collation is the beginning of alphabetical order, beginning are the same if it is small to large in accordance with the second letter
[root@localhost ~]# sort /etc/passwd
abrt:x:173:173::/etc/abrt:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
avahi:x:70:70:Avahi mDNS/DNS-SD Stack:/var/run/avahi-daemon:/sbin/nologin
bin:x:1:1:bin:/bin:/sbin/nologin
chen:x:1000:1000:chen:/home/chen:/bin/bash
chrony:x:995:991::/var/lib/chrony:/sbin/nologin
colord:x:997:995:User for colord:/var/lib/colord:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin(1) The / etc / passwd file, the third column reverse sequence
[root@localhost ~]# sort -t : -rk 3 /etc/passwd
nobody:x:99:99:Nobody:/:/sbin/nologin
polkitd:x:999:998:User for polkitd:/:/sbin/nologin
libstoragemgmt:x:998:996:daemon account for libstoragemgmt:/var/run/lsm:/sbin/nologin
colord:x:997:995:User for colord:/var/lib/colord:/sbin/nologin
saslauth:x:996:76:Saslauthd user:/run/saslauthd:/sbin/nologin
chrony:x:995:991::/var/lib/chrony:/sbin/nologin
geoclue:x:994:989:User for geoclue:/var/lib/geoclue:/sbin/nologin
setroubleshoot:x:993:988::/var/lib/setroubleshoot:/sbin/nologin
sssd:x:992:987:User for sssd:/:/sbin/nologin
gnome-initial-setup:x:991:986::/run/gnome-initial-setup/:/sbin/nologin
postfix:x:89:89::/var/spool/postfix:/sbin/nologin
dbus:x:81:81:System message bus:/:/sbin/nologin(2) the / etc / passwd file, sort the third column, and outputs the stored content to a file user.txt
[root@localhost ~]# sort -t : -k 3 /etc/passwd -o chench.txt
[root@localhost ~]# cat chench.txt
root:x:0:0:root:/root:/bin/bash
chen:x:1000:1000:chen:/home/chen:/bin/bash
qemu:x:107:107:qemu user:/:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
bin:x:1:1:bin:/bin:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
pulse:x:171:171:PulseAudio System Daemon:/var/run/pulse:/sbi(D) uniq tool
Uniq tool often used in conjunction with the sort command in Linux systems, for reporting or ignore file duplicates. Specific command syntax is: uniq [options] parameters. Which include the following common options.
(1) -c: counting;
(2) -d: Only duplicate rows;
(. 3) -u: display line appears only once;
1: delete a file duplicates.
[root@localhost ~]# cat a.txt
centos7 6.2
centos7 6.2
centos7 6.2
centos7 6.2
centos7 6.2
centos7 6.6
centos7 6.2
centos7 6.2
centos7 6.3
centos7 6.5
linux 1
linux 2
linux 3
linux 4
linux 5
linux 6
[root@localhost ~]# uniq a.txt
centos7 6.2
centos7 6.6
centos7 6.2
centos7 6.3
centos7 6.5
linux 1
linux 2
linux 3
linux 4
linux 5
linux 6
uniq but only tool to remove duplicate delete duplicate continuous
(2) delete a file duplicate rows, and the number of the bank's recurring beginning of the line display
[root@localhost ~]# uniq -c a.txt
1
5 centos7 6.2
1 centos7 6.6
2 centos7 6.2
1 centos7 6.3
1 centos7 6.5
1 linux 1
1 linux 2
1 linux 3
1 linux 4
1 linux 5
1 linux 6
1 (3) Find duplicate rows testfile file.
[root@localhost ~]# uniq -d a.txt
centos7 6.2
centos7 6.2