Spark里WordCount程序三种写法

方法一:
val g=sc.textFile("/hosts").flatMap(x=>x.split(" ")).map(x=>(x,1)).reduceByKey((x,y)=>x+y)


方法二:
scala> val rr=sc.textFile("/test2").flatMap(x=>x.split(" ")).countByValue()


方法三:

val t1=sc.textFile("/root/testFile/test2").flatMap(x=>x.split(" ")).toDF("word").registerTempTable("words")
sqlContext.sql("select word,count(*) from words group by word ").show

猜你喜欢

转载自blog.csdn.net/zhouzhuo_csuft/article/details/80651141