- 安装sqlite3
brew install sqlite
2.sqlite3 命令操作sqlite3 命令操作
sqlite3 china-poi.sqlite
.help 显示帮助
.quit 退出
sqlite> .output ChinaPOI.csv
sqlite> select * from ChinaPOI;
sqlite> .output stdout
sqlit>.quit
4.#给对应的表添加表头 默认分隔符号是|
vi ChinaPOI.csv
shopID|cityTelCode|cityName|name|address|tel|latitude|longitude|channel
:wq 保存退出.
[hadoop@hadoop-namenode3 server]$ head ChinaPOI.csv
shopID|cityTelCode|cityName|name|address|tel|latitude|longitude|channel
500001|021|上海|萨莎|东平路11号|021-64746628
64746170|31.20742|121.45477|food
…
500201|021|上海|富豪环球白玉兰西餐厅|衡山路516号富豪环球东亚酒店1楼|64155588-2756|31.20016|121.44867|food
[hadoop@hadoop-namenode3 server]$
5.上传数据到在hadoop 集群上
进入hadoop-namenode ( active )那台:这里我是hadoop-namenode3 是活动的。
hdfs dfs -mkdir /data/poi/
hdfs dfs -put ChinaPOI.csv /data/poi/
打开zeppelin 试试是否可以开始操作刚才导入的数据:
//val df = spark.read.format(“json”).load(“hdfs://10.20.2.3:8020/study/spark/Spark-The-Definitive-Guide/data/flight-data/json/2015-summary.json”)
val df = spark.read.format(“csv”).option(“header”, “true”).option(“delimiter”, “|”).option(“inferSchema”, “true”).load(“hdfs://10.20.2.3:8020/data/poi/ChinaPOI.csv”)
df.printSchema()
df.createOrReplaceTempView(“dfTable”)
df.head(5)
