uniq remove duplicate lines or statistics

uniq Remove duplicate lines or statistics
Main options
-u (unique) and -d (duplicate) -c (statistics) -f (number of skipped fields, default tab marked) -s (same as f, unit character)
1. The same record is printed Once (must be sorted)
sort a.txt |uniq or
sort -u a.txt -o b.txt

[root@M165 root]# cat a.txt          
ab 2
ab 4
ab 2
ad 4
ab 4
count duplicate lines
[root@M165 root]# sort a.txt |uniq -dc
      2 ab 2
      2 ab 4
count separate lines, I don't know why there is uniq -uc
[root@M165 root]# sort a.txt |uniq -uc
      1 ad 4
Statistics
[root@M165 root]# sort a.txt |uniq -c
      2 ab 2
      2 ab 4
      1 ad 4

[root@M165 root]# tail -5 install.log.syslog
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:29 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 22 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
<14>Aug 22 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
不对比第一字段<13>Aug
[root@M165 root]# tail -5 install.log.syslog |uniq -f 1
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:29 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 22 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
不对比第二字段21
[root@M165 root]# tail -5 install.log.syslog |uniq -f 2
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:29 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
不对比第二字段 16:01:28
[root@M165 root]# tail -5 install.log.syslog |uniq -f 3
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
[root@M165 root]# tail -5 install.log.syslog |uniq -f 4
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
用字符作单位
[root@M165 root]# tail -5 install.log.syslog |uniq -s 9
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:29 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 22 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd
[root@M165 root]# tail -5 install.log.syslog |uniq -s 10
<13>Aug 21 16:01:28 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:29 kudzu: aliased usb-controller1 as ehci-hcd
<13>Aug 21 16:01:20 kudzu: aliased usb-controller1 as ehci-hcd

转自： http://blog.csdn.net/magicbreaker/article/details/3225672

花下眠： http://huaxiamian.cc