伪分布式环境——一个服务器上运行多个进程
1.修改配置文件core-site.xml
[root@server ~]# cd hadoop/hadoop-2.7.3/
[root@server hadoop-2.7.3]# cd etc/hadoop/
[root@server hadoop]# ls
capacity-scheduler.xml kms-env.sh
configuration.xsl kms-log4j.properties
container-executor.cfg kms-site.xml
core-site.xml log4j.properties
hadoop-env.cmd mapred-env.cmd
hadoop-env.sh mapred-env.sh
hadoop-metrics2.properties mapred-queues.xml.template
hadoop-metrics.properties mapred-site.xml.template
hadoop-policy.xml slaves
hdfs-site.xml ssl-client.xml.example
httpfs-env.sh ssl-server.xml.example
httpfs-log4j.properties yarn-env.cmd
httpfs-signature.secret yarn-env.sh
httpfs-site.xml yarn-site.xml
kms-acls.xml
[root@server hadoop]# vim core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://server:8020</value>
</property>
</configuration>
说明：
# 修改主节点namenode
# 通过hdfs协议，后面填ip或者映射的主机名，还有指定端口
2.修改分布式存储配置文件hdfs-site.xml
[root@server hadoop]# vim hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
</configuration>
以上两步配置hdfs配置项配置完成，就可以启动HDFS了
3.启动HDFS
（1）格式化namenode
清空NameNode目录下的所有数据，生成目录结构，初始化一些信息到文件中
[root@server hadoop]# cd ../../
[root@server hadoop-2.7.3]# bin/hdfs namenode -format
18/12/17 09:34:34 INFO namenode.NameNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting NameNode
STARTUP_MSG: host = server/192.168.2.100
STARTUP_MSG: args = [-format]
STARTUP_MSG: version = 2.7.3
STARTUP_MSG: classpath = /root/hadoop/hadoop-2.7.3/etc/hadoop:/root/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/root/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/root/hadoop/hadoop-2.7.3/share/hadoop/common/lib/stax-api-1.0-2.jar:/root/hadoop/hadoop-2.7.3/share/hadoop/common/lib/activation-1.1.jar:/root/hadoop/hadoop-2.7.3/share/hadoop/common/l
/************************************************************
SHUTDOWN_MSG: Shutting down NameNode at server/192.168.2.100
************************************************************/
[root@server hadoop-2.7.3]#
格式化成功
（2）启动NameNode、DataNode、SecondaryNameNode
[root@server hadoop]# cd ../../
[root@server hadoop-2.7.3]# cd sbin/
[root@server sbin]# ls
distribute-exclude.sh start-all.cmd stop-balancer.sh
hadoop-daemon.sh start-all.sh stop-dfs.cmd
hadoop-daemons.sh start-balancer.sh stop-dfs.sh
hdfs-config.cmd start-dfs.cmd stop-secure-dns.sh
hdfs-config.sh start-dfs.sh stop-yarn.cmd
httpfs.sh start-secure-dns.sh stop-yarn.sh
kms.sh start-yarn.cmd yarn-daemon.sh
mr-jobhistory-daemon.sh start-yarn.sh yarn-daemons.sh
refresh-namenodes.sh stop-all.cmd
slaves.sh stop-all.sh
[root@server sbin]# ./start-dfs.sh
Java HotSpot(TM) Client VM warning: You have loaded library /root/hadoop/hadoop-2.7.3/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
18/12/17 12:12:24 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Starting namenodes on [server]
The authenticity of host 'server (192.168.2.100)' can't be established.
ECDSA key fingerprint is SHA256:lIAJ3lJqDQeb4KGF8pC+GE3ld051bKlR8qrMMNlCK8k.
ECDSA key fingerprint is MD5:24:aa:9c:60:93:03:22:63:ea:74:f4:84:c1:9a:38:8a.
Are you sure you want to continue connecting (yes/no)? yes
server: Warning: Permanently added 'server,192.168.2.100' (ECDSA) to the list of known hosts.
root@server's password:
server: Error: JAVA_HOME is not set and could not be found.
The authenticity of host 'localhost (::1)' can't be established.
ECDSA key fingerprint is SHA256:lIAJ3lJqDQeb4KGF8pC+GE3ld051bKlR8qrMMNlCK8k.
ECDSA key fingerprint is MD5:24:aa:9c:60:93:03:22:63:ea:74:f4:84:c1:9a:38:8a.
Are you sure you want to continue connecting (yes/no)? yes
localhost: Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.
root@localhost's password:
localhost: Error: JAVA_HOME is not set and could not be found.
启动报错。
原因：JAVA_HOME环境变量没有设置，去hadoop环境中设置
[root@server hadoop-2.7.3]# cd etc/hadoop/
[root@server hadoop]# ls
...
hadoop-env.cmd mapred-env.cmd
hadoop-env.sh mapred-env.sh
...
[root@server hadoop]# echo $JAVA_HOME
/root/tomcat/jdk1.8.0_191
[root@server hadoop]# vim hadoop-env.sh
# The java implementation to use.
export JAVA_HOME=/root/tomcat/jdk1.8.0_191
[root@server hadoop]#
将hadoop-env.sh下的JAVA_HOME路径替换成安装的jdk的路径
重新启动dfs启动项：
[root@server hadoop-2.7.3]# ./sbin/start-dfs.sh
Java HotSpot(TM) Client VM warning: You have loaded library /root/hadoop/hadoop-2.7.3/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
18/12/17 12:28:39 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@server hadoop-2.7.3]# jps #查看各个节点是否启动成功
22985 Jps
22555 NameNode
22684 DataNode
22862 SecondaryNameNode
[root@server hadoop-2.7.3]#
因为要启动多个进程，各个进程之间需要相互通信，所以要多次输入密码
各个节点启动成功
查看一下50070端口是否启动（50070是hdfs的http端口号）
[root@server hadoop-2.7.3]# netstat -nlt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 0.0.0.0:50090 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:6000 0.0.0.0:* LISTEN
tcp 0 0 192.168.2.100:8020 0.0.0.0:* LISTEN
tcp 0 0 192.168.122.1:53 0.0.0.0:* LISTEN
tcp 0 0 127.0.0.1:37238 0.0.0.0:* LISTEN
tcp 0 0 0.0.0.0:50070 0.0.0.0:* LISTEN
50070启动成功
注意：50070是http的端口号，上面配的8020是RPC tcp协议的端口号
4.关闭防火墙（或者将hadoop产生的端口全部开放）
[root@server hadoop-2.7.3]# systemctl stop firewalld
[root@server hadoop-2.7.3]#
5.浏览器访问主节点NameNode
首先需要在windows上做一下映射：
步骤：此电脑—>C:—> Windows—>System32—>drivers—>etc—>hosts
添加主机名映射：
# localhost name resolution is handled within DNS itself.
# 127.0.0.1 localhost
# ::1 localhost
192.168.2.100 server
浏览器访问：http://ip(主机名):50070
访问成功
6.停止HDFS各个节点
[root@server ~]# cd hadoop/hadoop-2.7.3/
[root@server hadoop-2.7.3]# ./sbin/stop-
stop-all.cmd stop-dfs.cmd stop-yarn.cmd
stop-all.sh stop-dfs.sh stop-yarn.sh
stop-balancer.sh stop-secure-dns.sh
[root@server hadoop-2.7.3]# ./sbin/stop-dfs.sh
Java HotSpot(TM) Client VM warning: You have loaded library /root/hadoop/hadoop-2.7.3/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
18/12/17 13:08:08 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Stopping namenodes on [server]
root@server's password:
server: stopping namenode
root@localhost's password:
localhost: stopping datanode
Stopping secondary namenodes [0.0.0.0]
[email protected]'s password:
0.0.0.0: stopping secondarynamenode
Java HotSpot(TM) Client VM warning: You have loaded library /root/hadoop/hadoop-2.7.3/lib/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
18/12/17 13:08:43 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[root@server hadoop-2.7.3]# jps
23943 Jps
[root@server hadoop-2.7.3]#
HDFS各个节点成功停止
