1. 卸载自带的java
查看已经安装的java
rpm -qa | grep java
java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.i686
java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.i686
删除openjdk版本
rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.i686
rpm -e --nodeps java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.i686
2. jdk安装 解压:tar -zxvf jdk-8u181-linux-x64.tar.gz -C ~/bigdata 添加到系统环境变量: ~/.bash_profile
export JAVA_HOME=/root/bigdata/jdk1.8.0_181
export PATH=$JAVA_HOME/bin:$PATH
使得环境变量生效: source ~/.bash_profile 验证java是否配置成功: java -v
3. 安装ssh(免密登陆)
sudo yum install ssh
ssh-keygen -t rsa
cp ~/.ssh/id_rsa.pub ~/.ssh/authorized_keys
4. 下载并解压hadoop 下载:直接去cdh网站下载 解压:tar -zxvf hadoop-2.6.0-cdh5.7.0.tar.gz -C ~/bigdata , 添加到系统环境变量: ~/.bash_profile
export HADOOP_HOME=/root/bigdata/hadoop-2.6.0-cdh5.7.0
export PATH=$HADOOP_HOME/bin:$PATH
5. hadoop配置文件的修改(/root/bigdata/hadoop-2.6.0-cdh5.7.0/etc/hadoop)
hadoop-env.sh
export JAVA_HOME=/root/bigdata/jdk1.8.0_181
core-site.xml
<property>
<name>fs.defaultFS</name>
<value>hdfs://hadoop:8020</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/root/bigdata/tmp</value>
</property>
hdfs-site.xml
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
slaves
说明:上面配置的,hdfs://hadoop:8020中hadoop是虚拟机名,需要在虚拟机/etc/hosts中添加映射关系,192.168.199.100 hadoop
6. 启动hdfs 格式化文件系统(仅第一次执行即可,不要重复执行):hdfs/hadoop namenode -format 启动hdfs:sbin/start-dfs.sh 验证是否启动成功:
jps
DataNode
SecondaryNameNode
NameNode
浏览器访问方式: http://hadoop:50070
7. 停止hdfs sbin/stop-dfs.sh
YARN环境搭建
- 1.复制文件,路径:/hadoop_home/etc/hadoop/。例(本人):/root/bigdata/hadoop-2.6.0-cdh5.7.0/etc/hadoop
cp mapred-site.xml.template mapred-site.xml
- 2.修改配置
- mapred-site.xml
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
- yarn-site.xml
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
- 3.启动YARN相关的进程 sbin/start-yarn.sh
- 4.验证
jps
ResourceManager
NodeManager
http://hadoop:8088
- 5.停止YARN相关的进程 sbin/stop-yarn.sh