转载请注明出处:http://blog.csdn.net/dongdong9223/article/details/86438951
本文出自【我是干勾鱼的博客】
Ingredients:
-
Java:Java SE Development Kit 8u162(Oracle Java Archive),Linux下安装JDK修改环境变量
-
Hadoop:hadoop-2.9.1.tar.gz(Apache Hadoop Releases Downloads, All previous releases of Hadoop are available from the Apache release archive site)
-
Hive:hive-2.3.4(mirrors.tuna.tsinghua.edu.cn,Mirror site for Hive)
之前在易筋经Hive——Hive安装及简单使用中讲解了Hive的环境搭建及简单操作方法,今天来讲解一下Hive中数据库、数据表以及数据信息的插入。
Hive具体数据的操作可以在官网User Documentation中找到。
1 启动Hive服务器
运行hiveserver2命令:
hiveserver2
2 创建数据库、数据表
在UserInfo.hql文件中保存HQL脚本命令,内容如下:
create database if not exists mydata;
use mydata;
drop table if exists UserInfo;
create table UserInfo(
id int, --id
name string, --姓名
hobby array<string>, --个人爱好
address map<string,string> --地址信息
)
row format delimited
fields terminated by ','
collection items terminated by '-'
map keys terminated by ':';
hive命令导入数据:
# hive -f UserInfo.hql
which: no hbase in (/opt/hive/apache-hive-2.3.4-bin/bin:/opt/java/jdk1.8.0_162/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Logging initialized using configuration in jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/hive-common-2.3.4.jar!/hive-log4j2.properties Async: true
OK
Time taken: 5.497 seconds
OK
Time taken: 0.026 seconds
OK
Time taken: 0.118 seconds
OK
Time taken: 0.669 seconds
进入Hive查看:
3 上传数据文件
先要在Hadoop的hdfs-site.xml文件中加入:
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
否则在插入数据的时候会出现权限的问题。
在HDFS上创建一个文件夹:
/opt/hadoop/hadoop-2.9.1/bin/hadoop fs -mkdir -p /tmp/myfile/
可以使用Hue文件浏览器,上传本地csv文件。假设路径为/tmp/hive, 文件名为: UserInfo.csv。或者直接使用命令上传文件:
/opt/hadoop/hadoop-2.9.1/bin/hadoop fs -put /opt/git/WebCrawler/UserInfo.csv /tmp/myfile/
给文件夹授权:
/opt/hadoop/hadoop-2.9.1/bin/hadoop fs -chmod 755 /tmp/myfile
查看:
# /opt/hadoop/hadoop-2.9.1/bin/hadoop fs -ls /tmp/myfile/
Found 1 items
-rw-r--r-- 1 root supergroup 146 2019-01-13 22:48 /tmp/myfile/UserInfo.csv
能够看到csv文件已经保存到HDFS中了。
4 插入数据
在客户端连接Hive服务器,运行命令:
beeline -u jdbc:hive2://localhost:10000
如下:
# beeline -u jdbc:hive2://localhost:10000
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/hive/apache-hive-2.3.4-bin/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop/hadoop-2.9.1/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Connecting to jdbc:hive2://localhost:10000
Connected to: Apache Hive (version 2.3.4)
Driver: Hive JDBC (version 2.3.4)
Transaction isolation: TRANSACTION_REPEATABLE_READ
Beeline version 2.3.4 by Apache Hive
插入数据:
0: jdbc:hive2://localhost:10000> load data inpath '/user/root/myfile/UserInfo.csv' overwrite into table mydata.UserInfo;
No rows affected (2.411 seconds)
查看数据:
0: jdbc:hive2://localhost:10000> select * from userinfo;
+--------------+----------------+-----------------------+---------------------------------------------+
| userinfo.id | userinfo.name | userinfo.hobby | userinfo.address |
+--------------+----------------+-----------------------+---------------------------------------------+
| 1 | xiaoming | ["book","TV","code"] | {"beijing":"chaoyang","shagnhai":"pudong"} |
| 2 | lilei | ["book","code"] | {"nanjing":"jiangning","taiwan":"taibei"} |
| 3 | lihua | ["music","book"] | {"heilongjiang":"haerbin"} |
+--------------+----------------+-----------------------+---------------------------------------------+
3 rows selected (1.411 seconds)
参考
Permission denied: user=administrator, access=WRITE, inode="/":root:supergroup:drwxr-xr-x