【Hadoop离线基础总结】大数据集群环境准备

大数据集群环境准备

---

三台虚拟机关闭防火墙

centOS 7
	service firewalld stop			->关闭防火墙
	chkconfig firewalld off			->开机关闭防火墙
	systemctl status firewalld.service			->查看当前防火墙状态

---

三台虚拟机关闭selinux

vim /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
#SELINUX=enforcing			->把这一行注释掉
SELINUX=disabled			->改成disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

---

三台虚拟机更改主机名

centOS 7
	vim /etc/hostname

三台机器分别添加
	node01.hadoop.com
	node02.hadoop.com
	node03.hadoop.com

---

三台虚拟机做主机名和IP地址的映射

vim /etc/hosts

三台机器都添加
	192.168.0.10 node01.hadoop.com node01
	192.168.0.20 node02.hadoop.com node02
	192.168.0.30 node03.hadoop.com node03

---

三台虚拟机关闭重启

reboot -h now

---

三台虚拟机免密码登录

• 第一步
  生成公钥和私钥

ssh-keygen 	-t rsa

之后各按三次回车

• 第二步
  将每台机器的私钥拷贝到第一台机器上

ssh-copy-id node01

• 第三步
  将第一台机器的authorized-keys拷贝到第二台第三台机器上

scp /root/.ssh/authorized_keys node02:/root/.ssh
scp /root/.ssh/authorized_keys node03:/root/.ssh

---

三台虚拟机时钟同步

通过网络连接外网进行时钟同步，必须保证虚拟机连上外网
ntpdate us.pool.ntp.org;

#阿里云时钟同步服务器
ntpdate ntp4.aliyun.com

三台机器定时任务
crontab -e
*/1 * * * * /usr/bin/ntpdate us.pool.ntp.org
或者直接与阿里云服务器进行时钟同步
*/1 * * * * /usr/bin/ntpdate ntp4.aliyun.com

---

三台虚拟机安装jdk

查看自带的open jdk
rpm -qa | grep java

卸载系统自带jdk
rpm -e xxx --nodeps

所有软件的安装在
mkdir -p /export/softwares
所有的软件压缩在
mkdir -p /export/servers

上传jdk到/export/softwares路径下，并解压
tar -zxvf jdk-8u141-linux-x64.tar.gz -C ../servers/	

安装工具，方便直接上传文件
yum -y install lrzsz

备份/etc/profile文件
cp /etc/profile /export/servers

shell脚本一键安装
创建一个shell脚本
vim /etc/servers/shells/install_jdk/sh

#!/bin/bash

# 解压jdk包到servers文件夹
tar -zxvf /export/softwares/jdk-8u141-linux-x64.tar.gz -C /export/servers/

# 到jdk1.8.0_141文件夹中，并把其绝对路径定义为home变量
cd /export/servers/jdk1.8.0_141
home=`pwd`

# 回显home变量
echo $home

# 配置环境变量
## 将"export JAVA_HOME=${home}"追加到/etc/profile中
echo "export JAVA_HOME=${home}"  >> /etc/profile
## 将"export PATH=:\$PATH:\$JAVA_HOME/bin"追加到/etc/profile
echo "export PATH=:\$PATH:\$JAVA_HOME/bin" >> /etc/profile


# 用for循环给机器node02和机器node03配置环境变量
for m in  2 3
do
scp -r /export/servers/jdk1.8.0_141 node0$m:/export/servers/
ssh node0$m "echo 'export JAVA_HOME=/export/servers/jdk1.8.0_141' >> /etc/profile; echo 'export PATH=:\$PATH:\$JAVA_HOME/bin' >> /etc/profile"

done