5.运行mapreduce
map 映射
reduce 规约
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hadoop jar ./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar pi 5 10
词频统计
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ vi a.log
ruoze
jepson
www.ruozedata.com
dashu
adai
fanren
1
a
b
c
a b c ruoze jepon
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ vi b.txt
a b d e f ruoze
1 1 3 5
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -mkdir /wordcount
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -mkdir /wordcount/input
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -put a.log /wordcount/input
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -put b.txt /wordcount/input
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -ls /wordcount/input/
Found 2 items
-rw-r--r-- 1 hadoop supergroup 76 2019-02-16 21:59 /wordcount/input/a.log
-rw-r--r-- 1 hadoop supergroup 24 2019-02-16 21:59 /wordcount/input/b.txt
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hadoop jar \
./share/hadoop/mapreduce2/hadoop-mapreduce-examples-2.6.0-cdh5.7.0.jar \
wordcount /wordcount/input /wordcount/output1
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -cat /wordcount/output1/part-r-00000
19/02/16 22:05:46 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
1 3
3 1
5 1
a 3
adai 1
b 3
c 2
d 1
dashu 1
e 1
f 1
fanren 1
jepon 1
jepson 1
ruoze 3
www.ruozedata.com 1
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ hdfs dfs -get /wordcount/output1/part-r-00000 ./
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$ cat part-r-00000
1 3
3 1
5 1
a 3
adai 1
b 3
c 2
d 1
dashu 1
e 1
f 1
fanren 1
jepon 1
jepson 1
ruoze 3
www.ruozedata.com 1
[hadoop@hadoop002 hadoop-2.6.0-cdh5.7.0]$
把每个单词出现的次数统计出来