记一次centos7 下安装sqoop2 server全过程（步骤详细！！！）

文章目录

下载解压缩
配置环境变量
修改sqoop 配置
启动sqoop

Sqoop是一个用来将Hadoop和关系型数据库中的数据相互转移的工具，可以将一个关系型数据库（例如： MySQL,Oracle,Postgres等）中的数据导进到Hadoop的HDFS中，也可以将HDFS的数据导进到关系型数据库中。

对于某些NoSQL数据库它也提供了连接器。Sqoop，类似于其他ETL工具，使用元数据模型来判断数据类型并在数据从数据源转移到Hadoop时确保类型安全的数据处理。Sqoop专为大数据批量传输设计，能够分割数据集并创建Hadoop任务来处理每个区块。

下载解压缩

进入sqoop 官网，点击sqoop2 的download，进入以下页面：

点击官网推荐的镜像源，进入以下页面：

红色下划线指示的就是所需的文件，右键复制链接地址

开始下载安装：

[root@miao pack]# wget https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz
--2019-01-16 11:50:52--  https://mirrors.tuna.tsinghua.edu.cn/apache/sqoop/1.99.7/sqoop-1.99.7-bin-hadoop200.tar.gz
Resolving mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)... 101.6.8.193, 2402:f000:1:408:8100::1
Connecting to mirrors.tuna.tsinghua.edu.cn (mirrors.tuna.tsinghua.edu.cn)|101.6.8.193|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102436055 (98M) [application/x-gzip]
Saving to: ‘sqoop-1.99.7-bin-hadoop200.tar.gz’

100%[================================================================>] 102,436,055 12.9MB/s   in 7.3s

2019-01-16 11:50:59 (13.3 MB/s) - ‘sqoop-1.99.7-bin-hadoop200.tar.gz’ saved [102436055/102436055]

[root@miao pack]# tar -zxvf sqoop-1.99.7-bin-hadoop200.tar.gz
sqoop-1.99.7-bin-hadoop200/bin/sqoop-sys.sh
sqoop-1.99.7-bin-hadoop200/bin/sqoop.sh
sqoop-1.99.7-bin-hadoop200/bin/sqoop2-server
sqoop-1.99.7-bin-hadoop200/bin/sqoop2-shell
sqoop-1.99.7-bin-hadoop200/bin/sqoop2-tool
......

配置环境变量

建立sqoop 的额外包、日志文件的存储路径：

mkdir /usr/local/soft/sqoop/extra
mkdir /usr/local/soft/sqoop/logs

修改环境变量：

vi /etc/profile

// 加入以下内容
# sqoop2
export SQOOP_HOME=/usr/local/soft/sqoop
export PATH=$PATH:$SQOOP_HOME/bin
export SQOOP_SERVER_EXTRA_LIB=$SQOOP_HOME/extra
export CATALINA_BASE=$SQOOP_HOME/server
export LOGDIR=$SQOOP_HOME/logs

// 别忘了使改动生效
source /etc/profile

修改sqoop 配置

sqoop 配置文件在sqoop 安装目录下的conf 目录下：sqoop.properties。

找到以下这个配置，并将其改为自己的hadoop 配置目录：

扫描二维码关注公众号，回复： 4941899 查看本文章

# Hadoop configuration directory
org.apache.sqoop.submission.engine.mapreduce.configuration.directory=/usr/local/soft/hadoop/etc/hadoop

在conf 目录下新建catalina.properties 文件，加入以下内容：(记得将hadoop 路径改为自己的)

common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,${catalina.home}/../lib/*.jar,/usr/local/soft/hadoop/share/hadoop/common/*.jar,/usr/local/soft/hadoop/share/hadoop/common/lib/*.jar,/usr/local/soft/hadoop/share/hadoop/hdfs/*.jar,/usr/local/soft/hadoop/share/hadoop/hdfs/lib/*.jar,/usr/local/soft/hadoop/share/hadoop/mapreduce/*.jar,/usr/local/soft/hadoop/share/hadoop/mapreduce/lib/*.jar,/usr/local/soft/hadoop/share/hadoop/tools/lib/*.jar,/usr/local/soft/hadoop/share/hadoop/yarn/*.jar,/usr/local/soft/hadoop/share/hadoop/yarn/lib/*.jar,/usr/local/soft/hadoop/share/hadoop/httpfs/tomcat/lib/*.jar

启动sqoop

启动sqoop2 server

[root@master ~]# sqoop2-server start
Setting conf dir: /usr/local/soft/sqoop/bin/../conf
Sqoop home directory: /usr/local/soft/sqoop
Starting the Sqoop2 server...
0    [main] INFO  org.apache.sqoop.core.SqoopServer  - Initializing Sqoop server.
8    [main] INFO  org.apache.sqoop.core.PropertiesConfigurationProvider  - Starting config file poller thread
Sqoop2 server started.

启动sqoop2 shell

[root@master ~]# sqoop2-shell
Setting conf dir: /usr/local/soft/sqoop/bin/../conf
Sqoop home directory: /usr/local/soft/sqoop
Sqoop Shell: Type 'help' or '\h' for help.

sqoop:000>