CentOS7下CDH5.11.1集群离线部署

本文章仅作为个人笔记

CDH官网地址：https://www.cloudera.com/
CDH镜像地址：
- cm5.11.1地址：
- http://archive.cloudera.com/cm5/repo-as-tarball/5.11.1/cm5.11.1-centos7.tar.gz
- http://archive.cloudera.com/cm5/repo-as-tarball/5.11.1/cm5.11.1-centos7.tar.gz.sha1
- cdh5.11.1地址：
- http://archive.cloudera.com/cdh5/parcels/5.11.1/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel
- http://archive.cloudera.com/cdh5/parcels/5.11.1/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel.sha1
- http://archive.cloudera.com/cdh5/parcels/5.11.1/manifest.json
- jdk1.8.121官网下载地址：
- http://download.oracle.com/otn/java/jdk/8u121-b13/e9e7ea248e2c4826b92b3f075a80e441/jdk-8u121-linux-i586.rpm
- mysql5.6.36官网下载地址：
- https://dev.mysql.com/get/Downloads/MySQL-5.6/mysql-5.6.36-linux-glibc2.5-x86_64.tar.gz
- mysql连接驱动下载地址：
- https://dev.mysql.com/downloads/connector/j/
笔者使用VMware进行模拟安装，为了满足基础配置，这里分配3台虚拟机（hadoopb01(4g/4核/40g/192.168.75.136/master),hadoopb02(2g/2核/40g/192.168.75.144/slave),hadoopb03(2g/2核/40g/192.168.75.145/slave)），另外wget下载文件非常缓慢并且不方便反复测试及安装，笔者这里下载好所有资源文件放在本地（.sha1文件结尾的文件统一重命名为.sha）。（如果虚拟机开机后无网络（使用net桥接方式），可与命令行运行dhclient分配ip，若后续断开了可运行dhclient -r后再运行dhclient）
本地虚拟机，本地文件
在3台服务器共同操作：
- 安装基本工具： yum install vim wget net-tools unzip -y
- 关闭SELINUX：
- 运行 setenforce 0 临时关闭
- 永久关闭SELINUX：
  echo “SELINUX=disabled” > /etc/selinux/config
  echo “SELINUXTYPE=targeted” >> /etc/selinux/config
- 关闭iptables防火墙：
  - 运行 systemctl stop firewalld.service 停止防火墙
  - 运行 systemctl status firewalld.service 查看防火墙状态
  - 运行 systemctl disable firewalld.service 关闭防火墙开机启动
- 更改主机名：
  - 运行 hostname hadoopb01 更改主机名，根据不同主机选择不同主机名（hadoopb01/hadoopb02/hadoopb03）
  - 运行 echo “HOSTNAME=hadoopb01” > /etc/sysconfig/network ，根据不同主机选择不同主机名（hadoopb01/hadoopb02/hadoopb03）
  - 运行 echo “hadoopb01” > /etc/hostname ，根据不同主机选择不同主机名（hadoopb01/hadoopb02/hadoopb03）
- 安装jdk（参考http://www.jianshu.com/p/c5fc4dd47754）
- 密钥生成：
  - 运行 ssh-keygen 一路回车回车生成
- ip与hostname绑定：
  echo “192.168.75.136 hadoopb01” >> /etc/hosts
  echo “192.168.75.144 hadoopb02” >> /etc/hosts
  echo “192.168.75.145 hadoopb03” >> /etc/hosts
- 运行python –version 检查python版本，如果不是python2.6.6建议改为python2.6.6
- 时间同步：
  - 运行 yum install ntp -y 安装ntp服务
  - 运行 systemctl enable ntpd 使ntp服务开机启动
- 在从服务器操作（hadoopb02(2g/2核/40g/192.168.75.144/slave),hadoopb03(2g/2核/40g/192.168.75.145/slave）：
- 配置时钟同步：
  echo “driftfile /var/lib/ntp/ntp.drift” > /etc/ntp.conf
  echo “statistics loopstats peerstats clockstats” >> /etc/ntp.conf
  echo “filegen loopstats file loopstats type day enable” >> /etc/ntp.conf
  echo “filegen peerstats file peerstats type day enable” >> /etc/ntp.conf
  echo “filegen clockstats file clockstats type day enable” >> /etc/ntp.conf
  echo “server 192.168.75.136 prefer” >> /etc/ntp.conf
  echo “fudge 192.168.75.136 stratum 5” >> /etc/ntp.conf
  echo “restrict -4 default kod notrap nomodify nopeer noquery” >> /etc/ntp.conf
  echo “restrict -6 default kod notrap nomodify nopeer noquery” >> /etc/ntp.conf
  echo “restrict 127.0.0.1” >> /etc/ntp.conf
  echo “restrict ::1” >> /etc/ntp.conf
- 运行 service ntpd restart 重启ntpd并使配置生效
- 运行 ntpdate –u 192.168.75.136 同步时间
在主服务器操作（hadoopb01(4g/4核/40g/192.168.75.136/master）：
- 运行
  ssh-copy-id hadoopb01
  ssh-copy-id hadoopb02
  ssh-copy-id hadoopb03 完成各服务器免密登录
- 配置时钟同步：
  echo “driftfile /var/lib/ntp/ntp.drift” > /etc/ntp.conf
  echo “restrict 192.168.75.0 mask 255.255.255.0 nomodify notrap” >> /etc/ntp.conf
  echo “server 210.72.145.44” >> /etc/ntp.conf
  echo “server 202.112.10.36 perfer” >> /etc/ntp.conf
  echo “server 59.124.196.83” >> /etc/ntp.conf
  echo “restrict 210.72.145.44 nomodify notrap noquery” >> /etc/ntp.conf
  echo “restrict 202.112.10.36 nomodify notrap noquery” >> /etc/ntp.conf
  echo “restrict 59.124.196.83 nomodify notrap noquery” >> /etc/ntp.conf
  echo “server 127.127.1.0” >> /etc/ntp.conf
  echo “fudge 127.127.1.0 stratum 10” >> /etc/ntp.conf
- 运行 service ntpd restart 重启ntpd并使配置生效
- 运行 ntpdate –u 202.112.10.36 同步时间
- 使用filezilla将上文所述文件传输至主服务器（放置于root用户~目录）
- 安装配置mysql5.6（可参考http://www.jianshu.com/p/c5fc4dd47754）
- 创建相关用户及数据库：
  create database cmf default character set utf8
  grant all on cmf.* to ‘cmf’@’localhost’ identified by ‘cmf_password’
  grant all on cmf.* to ‘cmf’@’%’ identified by ‘cmf_password’
  grant all on cmf.* to ‘cmf’@’ ${m_hosts[${c_index}]}’ identified by ‘cmf_password’ drop database cmf create database cmf create database hive default character set utf8 grant all on hive.* to ‘hive’@’localhost’ identified by ‘hive_password’ grant all on hive.* to ‘hive’@’%’ identified by ‘hive_password’ grant all on hive.* to ‘hive’@’$ {m_hosts[ ${c_index}]}’ identified by ‘hive_password’ drop database hive create database hive create database hue default character set utf8 grant all on hue.* to ‘hue’@’localhost’ identified by ‘hue_password’ grant all on hue.* to ‘hue’@’%’ identified by ‘hue_password’ grant all on hue.* to ‘hue’@’$ {m_hosts[ ${c_index}]}’ identified by ‘hue_password’ drop database hue create database hue create database am default character set utf8 grant all on am.* to ‘am’@’localhost’ identified by ‘am_password’ grant all on am.* to ‘am’@’%’ identified by ‘am_password’ grant all on am.* to ‘am’@’$ {m_hosts[ ${c_index}]}’ identified by ‘am_password’ drop database am create database am create database os default character set utf8 grant all on os.* to ‘os’@’localhost’ identified by ‘os_password’ grant all on os.* to ‘os’@’%’ identified by ‘os_password’ grant all on os.* to ‘os’@’$ {m_hosts[${c_index}]}’ identified by ‘os_password’
  drop database os
  create database os
  flush privileges
- 安装并启动httpd服务：
- 运行 yum install -y httpd 安装httpd服务
- 运行 systemctl enable httpd 设置httpd服务开机启动
- 运行 service httpd start 启动httpd服务
- 创建parcels服务（依次运行如下命令）：
  cd /var/www/html
  mkdir parcels
  cd parcels
  mv ~/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel ./
  mv ~/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel.sha ./
  mv ~/manifest.json ./
- 创建cm5服务（依次运行如下命令）：
  mkdir /opt/rpminstall
  cd /opt/rpminstall
  mv ~/cm5.11.1-centos7.tar.gz ./
  mv ~/cm5.11.1-centos7.tar.gz.sha ./
  tar -zxf cm5.11.1-centos7.tar.gz -C /var/www/html/
  cd /var/www/html/
  mkdir -p cm5/redhat/7/x86_64/
  mv cm cm5/redhat/7/x86_64/
  chown apache:apache -R /var/www/html
  cd /var/www/html/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64
  yum install -y cloudera-manager-daemons-5.11.1-1.cm5111.p0.9.el7.x86_64.rpm
  yum install -y cloudera-manager-server-5.11.1-1.cm5111.p0.9.el7.x86_64.rpm
  mkdir /usr/share/java
  cd /usr/share/java
  mv ~/mysql-connector-java-5.1.42.zip ./
  unzip mysql-connector-java-5.1.42.zip
  cd mysql-connector-java-5.1.42
  cp mysql-connector-java-5.1.42-bin.jar ../mysql-connector-java.jar
  echo “com.cloudera.cmf.db.type=mysql” > /etc/cloudera-scm-server/db.properties
  echo “com.cloudera.cmf.db.host=localhost” >> /etc/cloudera-scm-server/db.properties
  echo “com.cloudera.cmf.db.name=cmf” >> /etc/cloudera-scm-server/db.properties
  echo “com.cloudera.cmf.db.user=cmf” >> /etc/cloudera-scm-server/db.properties
  echo “com.cloudera.cmf.db.password=cmf_password” >> /etc/cloudera-scm-server/db.properties
- 运行 service cloudera-scm-server start 启动 cloudera-scm 服务
- 运行 tail -f /var/log/cloudera-scm-server/cloudera-scm-server.log 查看运行日志，待日志停止且显示Started Jetty server.表示服务运行完成。
- 与浏览器输入 http://192.168.75.136:7180开始正式的集群部署
- 默认用户名密码都为admin
- 同意并继续
- 选择适合自己的版本并继续
- 点击继续
- 填入3台服务器的ip并点击搜索
- 确认可行后点击继续
- 与浏览器访问 http://192.168.75.136/parcels/确认parcels服务，若出现如图则表示ok
- 与浏览器访问 http://192.168.75.136/cm5/redhat/7/x86_64/cm/5/ 确认cm5服务，若出现如图则表示ok
- 点击更多选项配置parcel服务
- 填入之前部署的本地parcel服务并删除多余的url点击保存更改
- 点击自定义存储库配置cm5服务
- 配置cm5服务并点击继续
- 直接点击继续
- 直接点击继续
- 选择合适的登陆配置（密码或私钥），点击继续
- 等待安装完成点击继续
- 等待安装完parcel点击继续
- 检测完成后点击完成（这里忽略这两个警告，可自行修复）
- 这里选择所有服务以测试是否可以完成所有服务的安装
- 这里直接点继续，如果需要自定义可自行定义。
- 填写创建好的数据库信息测试链接ok后点击继续
- 这里直接点击继续（用户可自行配置）
- 等都通过后点击继续，如果hive的安装出现问题可参考文件末尾教程再操作。
- 点击完成，完成安装
- 至此完成安装
文章的最后附上个人的脚本（吐槽下简书的书写为了格式会导致粘贴部分文案行不对号，如果要使用这个脚本需要更改（主机ip，主机名））：
最后为了使脚本能够运行，需要运行 chmod +x filepath 为文件添加权限，运行时执行 bash filepath n 例如（bash cdh.sh 0），运行此脚本可能mysql初始化不完全，如果可以，最好还是弄懂所有的步骤。
====================================================================================================master
#!/bin/bash
c_index=$1
m_names=(“master” “slave01” “slave02”)
m_hosts=(“192.168.1.131” “192.168.1.129” “192.168.1.226”)
function auto_ssh_keygen(){
expect -c “set timeout -1;
spawn ssh-keygen;
expect {
id_rsa {
send \n\r;
exp_continue;
};
Overwrite {
send y\n\r;
exp_continue;
};
passphrase {
send \n\r;
exp_continue;
};
again {
send \n\r;
exp_continue;
};
};
”
}
function auto_ssh_copy(){
expect -c “set timeout -1;
spawn ssh-copy-id $1;
expect {
yes/no {
send yes\n\r;
exp_continue;
};
password {
send $2\n\r; exp_continue; }; }; ” } #初始化必要工具 yum install expect -y yum install unzip -y #关闭selinux setenforce 0 echo “SELINUX=disabled” > /etc/selinux/config echo “SELINUXTYPE=targeted” >> /etc/selinux/config #关闭防火墙 systemctl stop firewalld.service systemctl status firewalld.service systemctl disable firewalld.service #定义主机名 hostname$ {m_names[ ${c_index}]} echo “HOSTNAME=$ {m_names[ ${c_index}]}” > /etc/sysconfig/network #ip与hostname绑定 echo “$ {m_names[ ${c_index}]}” > /etc/hostname t_index=0 for m_name in$ {m_names[@]};
do
echo “ ${m_hosts[t_index]}$ m_name” >> /etc/hosts
let t_index++
done
#时间同步
yum install ntp -y
systemctl enable ntpd
if [ ${c_index} == 0 ] then echo “driftfile /var/lib/ntp/ntp.drift” > /etc/ntp.conf echo “restrict 192.168.75.0 mask 255.255.255.0 nomodify notrap” >> /etc/ntp.conf echo “server 210.72.145.44” >> /etc/ntp.conf echo “server 202.112.10.36 perfer” >> /etc/ntp.conf echo “server 59.124.196.83” >> /etc/ntp.conf echo “restrict 210.72.145.44 nomodify notrap noquery” >> /etc/ntp.conf echo “restrict 202.112.10.36 nomodify notrap noquery” >> /etc/ntp.conf echo “restrict 59.124.196.83 nomodify notrap noquery” >> /etc/ntp.conf echo “server 127.127.1.0” >> /etc/ntp.conf echo “fudge 127.127.1.0 stratum 10” >> /etc/ntp.conf service ntpd restart ntpdate –u 202.112.10.36 #免密登陆 for m_name in$ {m_names[@]};
do
auto_ssh_copy $m_name root done else echo “driftfile /var/lib/ntp/ntp.drift” > /etc/ntp.conf echo “statistics loopstats peerstats clockstats” >> /etc/ntp.conf echo “filegen loopstats file loopstats type day enable” >> /etc/ntp.conf echo “filegen peerstats file peerstats type day enable” >> /etc/ntp.conf echo “filegen clockstats file clockstats type day enable” >> /etc/ntp.conf echo “server$ {m_hosts[0]} prefer” >> /etc/ntp.conf
echo “fudge ${m_hosts[0]} stratum 5” >> /etc/ntp.conf echo “restrict -4 default kod notrap nomodify nopeer noquery” >> /etc/ntp.conf echo “restrict -6 default kod notrap nomodify nopeer noquery” >> /etc/ntp.conf echo “restrict 127.0.0.1” >> /etc/ntp.conf echo “restrict ::1” >> /etc/ntp.conf service ntpd restart ntpdate –u$ {m_hosts[0]}
fi
#生成私钥
rm -rf /root/.ssh/id_rsa
rm -rf /root/.ssh/id_rsa.pub
auto_ssh_keygen
#安装jdk
rpm -Uvh jdk-8u121-linux-x64.rpm
echo “JAVA_HOME=/usr/java/jdk1.8.0_121” > /etc/profile.d/java.sh
echo “PATH=$PATH:$JAVA_HOME/bin” >> /etc/profile.d/java.sh
echo “CLASSPATH=.:$JAVA_HOME/lib/jt.jar:$JAVA_HOME/lib/tools.jar” >> /etc/profile.d/java.sh
echo “export JAVA_HOME PATH CLASSPATH” >> /etc/profile.d/java.sh
source /etc/profile
if [ ${c_index} == 0 ] then #安装并初始化配置mysql tar -zxvf mysql-5.6.36-linux-glibc2.5-x86_64.tar.gz mv mysql-5.6.36-linux-glibc2.5-x86_64 /usr/local/mysql touch /etc/my.cnf echo “[mysql]” > /etc/my.cnf echo “default-character-set=utf8” >> /etc/my.cnf echo “socket=/var/lib/mysql/mysql.sock” >> /etc/my.cnf echo “[mysqld]” >> /etc/my.cnf echo “skip-name-resolve” >> /etc/my.cnf echo “port=3306” >> /etc/my.cnf echo “socket=/var/lib/mysql/mysql.sock” >> /etc/my.cnf echo “basedir=/usr/local/mysql” >> /etc/my.cnf echo “datadir=/usr/local/mysql/data” >> /etc/my.cnf echo “max_connections=200” >> /etc/my.cnf echo “character-set-server=utf8” >> /etc/my.cnf echo “default-storage-engine=INNODB” >> /etc/my.cnf echo “lower_case_table_names=1” >> /etc/my.cnf echo “max_allowed_packet=16M” >> /etc/my.cnf mkdir -p /var/lib/mysql/mysql groupadd mysql useradd -g mysql mysql chown -R mysql:mysql /usr/local/mysql chown -R mysql:mysql /var/lib/mysql yum install -y perl-Module-Install.noarch /usr/local/mysql/scripts/mysql_install_db –user=mysql –basedir=/usr/local/mysql chown -R mysql:mysql /usr/local/mysql chown 777 /etc/my.cnf cp /usr/local/mysql/support-files/mysql.server /etc/rc.d/init.d/mysqld chmod +x /etc/rc.d/init.d/mysqld chkconfig –add mysqld echo “export PATH=\$PATH:/usr/local/mysql/bin” > /etc/profile.d/mysql.sh source /etc/profile service mysqld restart /usr/local/mysql/bin/mysql -e “delete from mysql.user where user = ”” /usr/local/mysql/bin/mysql -e “update mysql.user set password = (select password(‘root’)) where user = ‘root’” /usr/local/mysql/bin/mysql -e “create database cmf default character set utf8” /usr/local/mysql/bin/mysql -e “grant all on cmf.* to ‘cmf’@’localhost’ identified by ‘cmf_password’” /usr/local/mysql/bin/mysql -e “grant all on cmf.* to ‘cmf’@’%’ identified by ‘cmf_password’” /usr/local/mysql/bin/mysql -e “grant all on cmf.* to ‘cmf’@’$ {m_hosts[ ${c_index}]}’ identified by ‘cmf_password’” /usr/local/mysql/bin/mysql -e “drop database cmf” /usr/local/mysql/bin/mysql -e “create database cmf” /usr/local/mysql/bin/mysql -e “create database hive default character set utf8” /usr/local/mysql/bin/mysql -e “grant all on hive.* to ‘hive’@’localhost’ identified by ‘hive_password’” /usr/local/mysql/bin/mysql -e “grant all on hive.* to ‘hive’@’%’ identified by ‘hive_password’” /usr/local/mysql/bin/mysql -e “grant all on hive.* to ‘hive’@’$ {m_hosts[ ${c_index}]}’ identified by ‘hive_password’” /usr/local/mysql/bin/mysql -e “drop database hive” /usr/local/mysql/bin/mysql -e “create database hive” /usr/local/mysql/bin/mysql -e “create database hue default character set utf8” /usr/local/mysql/bin/mysql -e “grant all on hue.* to ‘hue’@’localhost’ identified by ‘hue_password’” /usr/local/mysql/bin/mysql -e “grant all on hue.* to ‘hue’@’%’ identified by ‘hue_password’” /usr/local/mysql/bin/mysql -e “grant all on hue.* to ‘hue’@’$ {m_hosts[ ${c_index}]}’ identified by ‘hue_password’” /usr/local/mysql/bin/mysql -e “drop database hue” /usr/local/mysql/bin/mysql -e “create database hue” /usr/local/mysql/bin/mysql -e “create database am default character set utf8” /usr/local/mysql/bin/mysql -e “grant all on am.* to ‘am’@’localhost’ identified by ‘am_password’” /usr/local/mysql/bin/mysql -e “grant all on am.* to ‘am’@’%’ identified by ‘am_password’” /usr/local/mysql/bin/mysql -e “grant all on am.* to ‘am’@’$ {m_hosts[ ${c_index}]}’ identified by ‘am_password’” /usr/local/mysql/bin/mysql -e “drop database am” /usr/local/mysql/bin/mysql -e “create database am” /usr/local/mysql/bin/mysql -e “create database os default character set utf8” /usr/local/mysql/bin/mysql -e “grant all on os.* to ‘os’@’localhost’ identified by ‘os_password’” /usr/local/mysql/bin/mysql -e “grant all on os.* to ‘os’@’%’ identified by ‘os_password’” /usr/local/mysql/bin/mysql -e “grant all on os.* to ‘os’@’$ {m_hosts[${c_index}]}’ identified by ‘os_password’”
/usr/local/mysql/bin/mysql -e “drop database os”
/usr/local/mysql/bin/mysql -e “create database os”
/usr/local/mysql/bin/mysql -e “flush privileges”
#安装httpd服务
yum install -y httpd
systemctl enable httpd
service httpd restart
cd /var/www/html
mkdir parcels
cd parcels
mv ~/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel ./
mv ~/CDH-5.11.1-1.cdh5.11.1.p0.4-el7.parcel.sha ./
mv ~/manifest.json ./
mkdir /opt/rpminstall
cd /opt/rpminstall
mv ~/cm5.11.1-centos7.tar.gz ./
mv ~/cm5.11.1-centos7.tar.gz.sha ./
tar -zxf cm5.11.1-centos7.tar.gz -C /var/www/html/
cd /var/www/html/
mkdir -p cm5/redhat/7/x86_64/
mv cm cm5/redhat/7/x86_64/
chown apache:apache -R /var/www/html
cd /var/www/html/cm5/redhat/7/x86_64/cm/5/RPMS/x86_64
yum install -y cloudera-manager-daemons-5.11.1-1.cm5111.p0.9.el7.x86_64.rpm
yum install -y cloudera-manager-server-5.11.1-1.cm5111.p0.9.el7.x86_64.rpm
mkdir /usr/share/java
cd /usr/share/java
mv ~/mysql-connector-java-5.1.42.zip ./
unzip mysql-connector-java-5.1.42.zip
cd mysql-connector-java-5.1.42
cp mysql-connector-java-5.1.42-bin.jar ../mysql-connector-java.jar
echo “com.cloudera.cmf.db.type=mysql” > /etc/cloudera-scm-server/db.properties
echo “com.cloudera.cmf.db.host=localhost” >> /etc/cloudera-scm-server/db.properties
echo “com.cloudera.cmf.db.name=cmf” >> /etc/cloudera-scm-server/db.properties
echo “com.cloudera.cmf.db.user=cmf” >> /etc/cloudera-scm-server/db.properties
echo “com.cloudera.cmf.db.password=cmf_password” >> /etc/cloudera-scm-server/db.properties
service cloudera-scm-server restart
source /etc/profile
fi
======================================================================================================end
坑锦集：
- 脚本运行完成后可能需要再次于master运行source /etc/profile 使环境变量生效。
- 如果遇到hive安装失败的问题可能需要再开个浏览器窗口访问cdh搭建主网站更改hive配置，且需要使用独立的mysql数据库及用户。
- 更改hive配置
- ntp时间同步服务如果报警，可于crontab -e添加定时同步时间。
- 当3台机器服务时，可将master也添加namenode节点。

CentOS7下CDH5.11.1集群离线部署

本文章仅作为个人笔记

猜你喜欢