1 前言
2 理论基础
2.1 部署软件架构
2.2 部署步骤和安装方法
2.2.1 安装方法
A）Cloudera Manager安装程序安装（容易）
B）yum源方式安装（中等）
C）源代码安装（难）
注：本教程使用方法B
2.2.2 部署步骤
1）安装JDK
2）安装并配置数据库
3）安装Cloudera管理服务端
4）安装Cloudera管理代理端
5）安装CDH管理服务软件
6）创建、启动和配置CDH并管理服务
2.3 Cloudera Manager端的相关文件
|
1
|
rpm -ql cloudera-manager-server
|
显示如下：
|
1
2
3
4
5
6
7
8
9
10
|
/etc/cloudera-scm-server
/etc/cloudera-scm-server/db
.properties
/etc/cloudera-scm-server/log4j
.properties
/etc/default/cloudera-scm-server
/etc/rc
.d
/init
.d
/cloudera-scm-server
/opt/cloudera/csd
/opt/cloudera/parcel-repo
/usr/sbin/cmf-server
/var/log/cloudera-scm-server
/var/run/cloudera-scm-server
|
文件与目录功能如下：
1）其中/etc/的2-4行为Cloudera Manager服务端配置文件
2）/opt/cloudera/parcel-repo为下载安装包存放目录
3 实践部分
3.1 环境信息
3.1.1 系统信息
注：系统请使用最小化安装，否则可能Sqoop服务可能无法启动
3.1.2 主机信息
Cloudera Manager:
ip address=10.168.0.120
hostname=cdm-m.cmdschool.org
Cloudera Host1:
ip address=10.168.0.121
hostname=cdm-h1.cmdschool.org
Cloudera Host2:
ip address=10.168.0.122
hostname=cdm-h2.cmdschool.org
Cloudera Host3:
ip address=10.168.0.123
hostname=cdm-h3.cmdschool.org
Cloudera Host4:
ip address=10.168.0.124
hostname=cdm-h4.cmdschool.org
3.2 运行环境配置
In Cloudera Manager & Cloudera Host[1-4]
3.2.1 关闭selinux
|
1
|
getenforce
|
如果显示如下：
|
1
|
Enforcing
|
则执行：
|
1
2
|
setenforce 0
sed
-i
's/SELINUX=enforcing/SELINUX=disabled/g'
/etc/selinux/config
|
3.2.2 配置hosts
vim编辑/etc/hosts
|
1
2
3
4
5
|
10.168.0.120 cdm-m.cmdschool.org
10.168.0.121 cdm-h1.cmdschool.org
10.168.0.122 cdm-h2.cmdschool.org
10.168.0.123 cdm-h3.cmdschool.org
10.168.0.124 cdm-h4.cmdschool.org
|
3.2.3 检查主机名称
|
1
|
hostname
|
注：主机名称与上面不一致会影响服务的启动
3.2.4 配置sudo(单用户模式适用，可选)
|
1
|
visudo
|
增加如下组
|
1
|
%cloudera-scm ALL=(ALL) NOPASSWD: ALL
|
确认包含如下行：
|
1
|
Defaults secure_path =
/sbin
:
/bin
:
/usr/sbin
:
/usr/bin
|
vim编辑/etc/pam.d/su，确保包含如下行
|
1
|
session required pam_limits.so
|
3.2.5 关闭防火墙并设置开机不启动
|
1
2
|
/etc/init
.d
/iptables
stop
chkconfig iptables off
|
3.2.6 优化虚拟内存需求率
1)检查虚拟内存需求率
|
1
|
cat
/proc/sys/vm/swappiness
|
显示如下：
|
1
|
|
2)临时降低虚拟内存需求率
|
1
|
sysctl vm.swappiness=0
|
3)永久降低虚拟内存需求率
vim编辑/etc/sysctl.conf
|
1
2
|
kernel.shmall = 4294967296
vm.swappiness = 0
|
并运行如下命令使生效
|
1
|
sysctl -p
|
3.2.7 解决透明大页面问题
1)检查透明大页面问题
|
1
|
cat
/sys/kernel/mm/transparent_hugepage/defrag
|
如果显示为：
|
1
|
[always] madvise never
|
2)临时关闭透明大页面问题
|
1
|
echo
never >
/sys/kernel/mm/transparent_hugepage/defrag
|
确认配置生效：
|
1
|
cat
/sys/kernel/mm/transparent_hugepage/defrag
|
应该显示为：
|
1
|
always madvise [never]
|
3)配置开机自动生效
vim编辑/etc/rc.local,加入如下内容
|
1
|
echo
never >
/sys/kernel/mm/transparent_hugepage/defrag
|
3.3 yum源的安装配置
3.3.1 公共yum源配置
In Cloudera Manager & Cloudera Host[1-4]
1)配置yum源
下载默认yum源
|
1
|
wget -P
/etc/yum
.repos.d/ https:
//archive
.cloudera.com
/cm5//6/x86_64/cm/cloudera-manager
.repo
|
修改为指定版本yum源
vim编辑/etc/yum.repos.d/cloudera-manager.repo修改如下参数：
|
1
|
baseurl=https:
//archive
.cloudera.com
/cm5/redhat/6/x86_64/cm/5
.6.0/
|
2)安装配置工具
|
1
|
yum
install
-y vim wget openssh-clients
|
3)安装jdk
|
1
|
yum
install
-y oracle-j2sdk1.7
|
4)安装python
|
1
|
yum
install
-y python
|
5)安装ntpd
|
1
|
yum
install
-y ntp
|
3.3.2 Cloudera Manager端yum源配置
In Cloudera Manager
1)安装Cloudera Manager包
|
1
|
yum
install
-y cloudera-manager-daemons cloudera-manager-server
|
2)安装mysql
|
1
|
yum
install
-y mysql-server mysql-devel mysql
|
3.3.3 Cloudera Manager Agent端yum源配置
In Cloudera Host[1-4]
安装Cloudera Manager Agent包
|
1
|
yum
install
-y cloudera-manager-agent cloudera-manager-daemons
|
3.4 依赖于yum源的环境配置
3.4.1 配置jdk变量环境
In Cloudera Manager & Cloudera Host[1-4]
1)vim编辑/etc/profile，末尾加入如下内容
|
1
2
3
4
|
export
JAVA_HOME=
/usr/java/jdk1
.7.0_67-cloudera
export
JRE_HOME=${JAVA_HOME}
/jre
export
CLASSPATH=.:${JAVA_HOME}
/lib
:${JRE_HOME}
/lib
export
PATH=${JAVA_HOME}
/bin
:$PATH
|
2)导入java环境变量
|
1
|
source
/etc/profile
|
3)测试jdk的配置
|
1
|
java -version
|
3.4.2 权限检查(单用户模式适用，可选)
In Cloudera Manager & Cloudera Host[1-4]
检查以下目录cloudera-scm用户具有完全的权限
检查当前目录权限:
|
1
|
ls
-ld
/opt/cloudera/
|
显示如下
|
1
|
drwxr-xr-x. 4 cloudera-scm cloudera-scm 4096 May 23 13:51
/opt/cloudera/
|
检查子目录权限:
|
1
|
ls
-lR
/opt/cloudera/
|
显示如下
|
1
2
3
4
5
6
7
8
9
10
|
/opt/cloudera/
:
total 8
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Feb 12 11:28 csd
drwxr-xr-x. 2 cloudera-scm cloudera-scm 4096 Feb 12 11:28 parcel-repo
/opt/cloudera/csd
:
total 0
/opt/cloudera/parcel-repo
:
total 0
|
同样，检查服务器或客户端目录权限
|
1
2
3
4
|
ls
-ld
/var/log/cloudera-scm-server/
ls
-lR
/var/log/cloudera-scm-server/
ls
-ld
/var/lib/cloudera-scm-agent/
ls
-lR
/var/lib/cloudera-scm-agent/
|
3.4.3 检查线程限制配置
In Cloudera Manager & Cloudera Host[1-4]
|
1
|
cat
/etc/security/limits
.d
/cloudera-scm
.conf
|
显示如下：
|
1
2
3
4
5
6
7
8
|
#
# (c) Copyright 2014 Cloudera, Inc.
#
cloudera-scm soft nofile 32768
cloudera-scm soft nproc 65536
cloudera-scm hard nofile 1048576
cloudera-scm hard nproc unlimited
cloudera-scm hard memlock unlimited
|
3.4.4 Cloudera Manager端配置
In Cloudera Manager
1)临时校对时间
|
1
|
ntpdate 0.centos.pool.ntp.org
|
2)启动并配置ntpd服务自动启动
|
1
2
|
/etc/init
.d
/ntpd
start
chkconfig ntpd on
|
3.4.5 Cloudera Manager Agen端配置
In Cloudera Host[1-4]
1)临时校对时间
|
1
|
ntpdate 10.168.0.120
|
2)vim编辑/etc/ntp.conf
注释掉外网时间服务器并增加内网时间服务器地址
|
1
2
3
4
5
|
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server 10.168.0.120 iburst
|
3)启动并配置ntpd服务自动启动
|
1
2
|
/etc/init
.d
/ntpd
start
chkconfig ntpd on
|
3.4.7 安装MySQL JDBC Driver
In Cloudera Manager & Cloudera Host[1-4]
|
1
2
3
4
|
wget http:
//dev
.mysql.com
/get/Downloads/Connector-J/mysql-connector-java-5
.1.39.
tar
.gz
tar
zxvf mysql-connector-java-5.1.39.
tar
.gz
mkdir
/usr/share/java/
cp
mysql-connector-java-5.1.39
/mysql-connector-java-5
.1.39-bin.jar
/usr/share/java/mysql-connector-java
.jar
|
3.4.8 配置公钥认证
In Cloudera Manager
In Cloudera Manager：
|
1
|
ssh
-keygen -t rsa
|
注：以上一路回车
In Cloudera Manager Agen：
|
1
2
3
4
5
|
|
In Cloudera Manager：
|
1
2
3
4
5
|
ssh
10.168.0.120
ssh
10.168.0.121
ssh
10.168.0.122
ssh
10.168.0.123
ssh
10.168.0.124
|
注：以上如果无需密码登记即成功
3.5 Cloudera Manager安装配置
In Cloudera Manager
3.5.1 修改mysql参数
1)关闭数据库
|
1
|
/etc/init
.d
/mysqld
stop
|
2)备份ib_logfile文件
|
1
2
3
|
mkdir
/var/lib/backup
cd
/var/lib/mysql/
mv
ib_logfile*
/var/lib/backup/
|
3)vim编辑/etc/my.cnf
加入如下参数
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
|
[mysqld]
transaction-isolation = READ-COMMITTED
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links = 0
key_buffer = 16M
key_buffer_size = 32M
max_allowed_packet = 32M
thread_stack = 256K
thread_cache_size = 64
query_cache_limit = 8M
query_cache_size = 64M
query_cache_type = 1
max_connections = 550
#expire_logs_days = 10
#max_binlog_size = 100M
#log_bin should be on a disk with enough free space. Replace '/var/lib/mysql/mysql_binary_log' with an appropriate path for your system
#and chown the specified folder to the mysql user.
log_bin=
/var/lib/mysql/mysql_binary_log
# For MySQL version 5.1.8 or later. Comment out binlog_format for older versions.
binlog_format = mixed
read_buffer_size = 2M
read_rnd_buffer_size = 16M
sort_buffer_size = 8M
join_buffer_size = 8M
# InnoDB settings
innodb_file_per_table = 1
innodb_flush_log_at_trx_commit = 2
innodb_log_buffer_size = 64M
innodb_buffer_pool_size = 4G
innodb_thread_concurrency = 8
innodb_flush_method = O_DIRECT
innodb_log_file_size = 512M
[mysqld_safe]
log-error=
/var/log/mysqld
.log
pid-
file
=
/var/run/mysqld/mysqld
.pid
sql_mode=STRICT_ALL_TABLES
|
3.5.2 启动并设置开机自动启动
|
1
2
|
/etc/init
.d
/mysqld
start
chkconfig mysqld on
|
3.5.3 初始化数据库
|
1
|
mysql_secure_installation
|
向导如下：
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
[...]
Enter current password
for
root (enter
for
none):
OK, successfully used password, moving on...
[...]
Set root password? [Y
/n
] y
New password:
Re-enter new password:
Remove anonymous
users
? [Y
/n
] y
[...]
Disallow root login remotely? [Y
/n
] n
[...]
Remove
test
database and access to it [Y
/n
] y
[...]
Reload privilege tables now? [Y
/n
] y
All
done
!
|
3.5.4 准备scm库
1)方法一
数据库配置：
|
1
2
3
4
|
mysql -uroot -p
create database scm default character
set
utf8;
grant all privileges on *.* to scm@
'cdm-m.cmdschool.org'
identified by
'scm'
;
flush privileges;
|
vim编辑/etc/cloudera-scm-server/db.properties修改如下参数：
|
1
2
3
4
5
|
com.cloudera.cmf.db.
type
=mysql
com.cloudera.cmf.db.host=cdm-m.cmdschool.org
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=scm
|
2)方法二(官方建议)
授权temp权限：
|
1
2
3
|
mysql -uroot -p
grant all privileges on *.* to
'temp'
@
'%'
identified by
'temp'
with grant option;
flush privileges;
|
生成配置文件：
|
1
|
/usr/share/cmf/schema/scm_prepare_database
.sh mysql -h cdm-m.cmdschool.org -utemp -ptemp --scm-host cdm-m.cmdschool.org scm scm scm
|
显示如下：
|
1
2
3
4
5
6
|
JAVA_HOME=
/usr/java/jdk1
.7.0_67-cloudera
Verifying that we can write to
/etc/cloudera-scm-server
Creating SCM configuration
file
in
/etc/cloudera-scm-server
Executing:
/usr/java/jdk1
.7.0_67-cloudera
/bin/java
-
cp
/usr/share/java/mysql-connector-java
.jar:
/usr/share/java/oracle-connector-java
.jar:
/usr/share/cmf/schema/
..
/lib/
* com.cloudera.enterprise.dbutil.DbCommandExecutor
/etc/cloudera-scm-server/db
.properties com.cloudera.cmf.db.
[main] DbCommandExecutor INFO Successfully connected to database.
All
done
, your SCM database is configured correctly!
|
确认生成的结果：
|
1
|
cat
/etc/cloudera-scm-server/db
.properties
|
显示如下：
|
1
2
3
4
5
6
7
8
9
10
|
# Auto-generated by scm_prepare_database.sh on Tue May 24 19:08:19 CST 2016
#
# For information describing how to configure the Cloudera Manager Server
# to connect to databases, see the "Cloudera Manager Installation Guide."
#
com.cloudera.cmf.db.
type
=mysql
com.cloudera.cmf.db.host=cdm-m.cmdschool.org
com.cloudera.cmf.db.name=scm
com.cloudera.cmf.db.user=scm
com.cloudera.cmf.db.password=scm
|
确认库访问权限:
|
1
2
|
mysql -uroot -p
show grants
for
scm@
'cdm-m.cmdschool.org'
;
|
显示如下：
|
1
2
3
4
5
6
7
|
+----------------------------------------------------------------------------------------------------------------------+
+----------------------------------------------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO
'scm'
@
'cdm-m.cmdschool.org'
IDENTIFIED BY PASSWORD
'*45E6E3C68BDF1AC7EBB5C5A3BCBD5E9437B293BE'
|
| GRANT ALL PRIVILEGES ON `scm`.* TO
'scm'
@
'cdm-m.cmdschool.org'
|
+----------------------------------------------------------------------------------------------------------------------+
2 rows
in
set
(0.00 sec)
|
清理数据库用户授权：
|
1
2
|
drop user
'temp'
@
'%'
;
flush privileges;
|
3.5.5 创建附加数据库(可选)
1)附加数据库列表
| Role | Database | User | Password |
| Activity Monitor | amon | amon | amon_password |
| Reports Manager | rman | rman | rman_password |
| Hive Metastore Server | metastore | hive | hive_password |
| Sentry Server | sentry | sentry | sentry_password |
| Cloudera Navigator Audit Server | nav | nav | nav_password |
| Cloudera Navigator Metadata Server |
navms | navms | navms_password |
2)创建数据库并配置管理账号密码
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
mysql -uroot -p
create database amon default character
set
utf8;
grant all privileges on amon.* to
'amon'
@
'%'
identified by
'amon_password'
;
create database rman default character
set
utf8;
grant all privileges on rman.* to
'rman'
@
'%'
identified by
'rman_password'
;
create database metastore default character
set
utf8;
grant all privileges on metastore.* to
'hive'
@
'%'
identified by
'hive_password'
;
create database sentry default character
set
utf8;
grant all privileges on sentry.* to
'sentry'
@
'%'
identified by
'sentry_password'
;
create database nav default character
set
utf8;
grant all privileges on nav.* to
'nav'
@
'%'
identified by
'nav_password'
;
create database navms default character
set
utf8;
grant all privileges on navms.* to
'navms'
@
'%'
identified by
'navms_password'
;
flush privileges;
|
3.5.6 配置Oozie库(可选)
1)数据库权限配置
|
1
2
3
4
5
|
mysql -uroot -p
create database oozie default character
set
utf8;
grant all privileges on oozie.* to
'oozie'
@
'localhost'
identified by
'oozie'
;
grant all privileges on oozie.* to
'oozie'
@
'%'
identified by
'oozie'
;
flush privileges;
|
2)配置Oozie库所需软连接
|
1
2
|
cd
/opt/cloudera/parcels/CDH/lib/oozie/lib/
ln
-s
/usr/share/java/mysql-connector-java
.jar mysql-connector-java.jar
|
3.5.7 启动服务并配置开机启动
|
1
2
|
/etc/init
.d
/cloudera-scm-server
start
chkconfig cloudera-scm-server on
|
3.5.8 故障排除
|
1
|
tail
-f
/var/log/cloudera-scm-server/cloudera-scm-server
.out
|
3.6 Cloudera Manager Agent安装
In Cloudera Host[1-4]
3.6.1 创建压缩包存放目录
|
1
2
|
mkdir
-p
/opt/cloudera/parcels
chown
cloudera-scm:cloudera-scm
/opt/cloudera/parcels
|
3.6.2 指定管理服务器和指定包存放目录
vim编辑/etc/cloudera-scm-agent/config.ini确保参数如下并启用：
|
1
2
3
|
server_host=cdm-m.cmdschool.org
server_port=7182
parcel_dir=
/opt/cloudera/parcels
|
3.6.3 指定运行单用户模式的用户名(仅用于单用户模式，不配置)
vim编辑/etc/default/cloudera-scm-agent并取消以下行的注释
|
1
|
USER=
"cloudera-scm"
|
3.6.4 启动服务并配置服务器开机启动
|
1
2
|
/etc/init
.d
/cloudera-scm-agent
start
chkconfig cloudera-scm-agent on
|
3.6.5 故障排除
如下命令监控启动服务的错误输出
|
1
|
tail
-f
/var/log/cloudera-scm-agent/cloudera-scm-agent
.out
|
3.7 登陆配置
In Cloudera WEB Manager
界面配置部分本章节省略……