原创转载请注明出处：http://agilestyle.iteye.com/blog/2291823
前期准备：
修改Linux主机名，IP，SSH无密登录，JDK安装，环境变量配置等，统统省略。
集群规划：
| hostname |
IP |
Software |
Process |
| hadoop-0000 |
192.168.5.200 |
JDK |
NameNode |
| hadoop-0001 |
192.168.5.201 |
JDK |
NameNode |
| hadoop-0002 |
192.168.5.202 |
JDK |
ResourceManager |
| hadoop-0003 |
192.168.5.203 |
JDK |
ResourceManager |
| hadoop-0004 |
192.168.5.204 |
JDK |
DataNode |
| hadoop-0005 |
192.168.5.205 |
JDK |
DataNode |
| hadoop-0006 |
192.168.5.206 |
JDK |
DataNode |
安装步骤：
2. 配置Hadoop集群
添加环境变量（添加保存退出之后，需要scp到其他6台机器上）
vi ~/.bashrc
#setup Java & Hadoop environment
export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77
export HADOOP_HOME=/home/hadoop/app/hadoop-2.6.4
export HIVE_HOME=/home/hadoop/app/apache-hive-1.2.1-bin
export HBASE_HOME=/home/hadoop/app/hbase-1.1.4
export ZOOKEEPER_HOME=/home/hadoop/app/zookeeper-3.4.8
export PATH=$PATH:${JAVA_HOME}/bin:${HADOOP_HOME}/bin:${HADOOP_HOME}/sbin:${HIVE_HOME}/bin:${HBASE_HOME}/bin:${ZOOKEEPER_HOME}/bin
source ~/.bashrc
vi hadoop-env.sh
export JAVA_HOME=/home/hadoop/app/jdk1.8.0_77
vi core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://ns1/</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>/home/hadoop/app/hadoop-2.6.4/tmp</value>
</property>
<property>
<name>ha.zookeeper.quorum</name>
<value>hadoop-0004:2181,hadoop-0005:2181,hadoop-0006:2181</value>
</property>
</configuration>
vi hdfs-site.xml
<configuration>
<property>
<name>dfs.nameservices</name>
<value>ns1</value>
</property>
<property>
<name>dfs.ha.namenodes.ns1</name>
<value>nn1,nn2</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn1</name>
<value>hadoop-0000:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn1</name>
<value>hadoop-0000:50070</value>
</property>
<property>
<name>dfs.namenode.rpc-address.ns1.nn2</name>
<value>hadoop-0001:9000</value>
</property>
<property>
<name>dfs.namenode.http-address.ns1.nn2</name>
<value>hadoop-0001:50070</value>
</property>
<property>
<name>dfs.namenode.shared.edits.dir</name>
<value>qjournal://hadoop-0004:8485;hadoop-0005:8485;hadoop-0006:8485/ns1</value>
</property>
<property>
<name>dfs.journalnode.edits.dir</name>
<value>/home/hadoop/app/hadoop-2.6.4/journaldata</value>
</property>
<property>
<name>dfs.ha.automatic-failover.enabled</name>
<value>true</value>
</property>
<property>
<name>dfs.client.failover.proxy.provider.ns1</name>
<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>
<property>
<name>dfs.ha.fencing.methods</name>
<value>
sshfence
shell(/bin/true)
</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.private-key-files</name>
<value>/home/hadoop/.ssh/id_rsa</value>
</property>
<property>
<name>dfs.ha.fencing.ssh.connect-timeout</name>
<value>30000</value>
</property>
</configuration>
cp mapred-site.xml.template mapred-site.xml
vi mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
vi yarn-site.xml
<configuration>
<!-- Site specific YARN configuration properties -->
<property>
<name>yarn.resourcemanager.ha.enabled</name>
<value>true</value>
</property>
<property>
<name>yarn.resourcemanager.cluster-id</name>
<value>yrc</value>
</property>
<property>
<name>yarn.resourcemanager.ha.rm-ids</name>
<value>rm1,rm2</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm1</name>
<value>hadoop-0002</value>
</property>
<property>
<name>yarn.resourcemanager.hostname.rm2</name>
<value>hadoop-0003</value>
</property>
<property>
<name>yarn.resourcemanager.zk-address</name>
<value>hadoop-0004:2181,hadoop-0005:2181,hadoop-0006:2181</value>
</property>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
vi slaves
hadoop-0004 hadoop-0005 hadoop-0006
配置完毕之后，将hadoop-0000上的hadoop文件夹scp -r到其他6台机器
启动：
第一次配置启动务必按照以下步骤一步一步往下走，亲测不会出错。
启动zookeeper集群（分别在hadoop-0004, hadoop-0005, hadoop-0006上启动）
zkServer.sh start
查看状态：一个leader，两个follower
zkServer.sh status
启动journalnode（分别在hadoop-0004, hadoop-0005, hadoop-0006上启动）
hadoop-daemon.sh start journalnode
JPS，hadoop-0004, hadoop-0005, hadoop-0006上多了JournalNode进程
格式化HDFS（hadoop-0000上执行）
hdfs namenode -format
格式化后会在根据core-site.xml中的hadoop.tmp.dir配置生成个文件，这里配置的是/home/hadoop/app/hadoop-2.6.4/tmp，然后将/home/hadoop/app/hadoop-2.6.4/tmp拷贝到hadoop-0001的/home/hadoop/app/hadoop-2.6.4/下
scp -r tmp/ hadoop-0001:/home/hadoop/app/hadoop-2.6.4/
格式化ZKFC（hadoop-0000上执行）
hdfs zkfc -formatZK
启动HDFS（hadoop-0000上执行）
start-dfs.sh
启动YARN（hadoop-0002上执行）
start-yarn.sh
另外需要在hadoop-0003上执行
yarn-daemon.sh start resourcemanager
测试：
http://hadoop-0000:50070/
http://hadoop-0001:50070/
http://hadoop-0002:8088/
http://hadoop-0003:8088/
以后启动集群，可以先在hadoop-0000上start-dfs.sh
之后在hadoop-0002上start-yarn.sh
接着hadoop-0003上yarn-daemon.sh start resourcemanager
最后可以在hadoop-0004，hadoop-0005，hadoop-0006上JPS
