Apache Hadoop集群缩容实战案例

　　　　　　　　　　　　　Apache Hadoop集群缩容实战案例

　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　　作者：尹正杰

版权声明：原创作品，谢绝转载！否则将追究法律责任。

一.黑名单退役

1>.在所有节点未退役之前，查看NameNode的WebUI

2>.编写黑名单的配置文件

[[email protected] ~]# echo ${HADOOP_HOME}
/yinzhengjie/softwares/hadoop-2.10.0
[[email protected] ~]# 
[[email protected] ~]# vim ${HADOOP_HOME}/etc/hadoop/blacklist
[[email protected] ~]# 
[[email protected] ~]# cat ${HADOOP_HOME}/etc/hadoop/blacklist
hadoop104.yinzhengjie.org.cn
[[email protected] ~]# 

3>.进修改NameNode节点的hdfs-site.xml配置文件

[[email protected] ~]# vim ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
[[email protected] ~]# 
[[email protected] ~]# cat ${HADOOP_HOME}/etc/hadoop/hdfs-site.xml
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
  Licensed under the Apache License, Version 2.0 (the "License");
  you may not use this file except in compliance with the License.
  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License. See accompanying LICENSE file.
-->

<!-- Put site-specific property overrides in this file. -->

<configuration>

    <property>
        <name>dfs.replication</name>
        <value>3</value>
        <description>指定HDFS副本的数量</description>
    </property>

    <property>
        <name>dfs.namenode.secondary.http-address</name>
        <value>hadoop105.yinzhengjie.org.cn:50090</value>
        <description>指定Hadoop辅助名称节点主机配置</description>
    </property>

    <property>
        <name>dfs.hosts.exclude</name>
        <value>${HADOOP_HOME}/etc/hadoop/blacklist</value>
        <description>指定黑名单路径</description>
    </property>

</configuration>
[[email protected] ~]# 

4>.刷新NameNode和ResourceMangaer的配置

[[email protected] ~]# hdfs dfsadmin -refreshNodes
Refresh nodes successful
[[email protected] ~]# 

5>.

二.