我终于用Eclipse成功连接上Hadoop了!!
啊啊啊啊啊啊!!来自少女的狂欢啊!!!!太快乐了!!!!
我遇到的情况如下:
然后就是还有下面这一大长串:(看着就头疼!!)
org.apache.hadoop.net.ConnectTimeoutException: Call From LAPTOP-JNDAQOOH/192.168.137.1 to 192.168.100.11:9000 failed on socket timeout exception: org.apache.hadoop.net.ConnectTimeoutException: 20000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=192.168.100.11/192.168.100.11:9000]; For more details see: http://wiki.apache.org/hadoop/SocketTimeout
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.hadoop.net.NetUtils.wrapWithMessage(NetUtils.java:792)
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:751)
at org.apache.hadoop.ipc.Client.call(Client.java:1480)
at org.apache.hadoop.ipc.Client.call(Client.java:1413)
at org.apache.hadoop.ipc.ProtobufRpcEngine
Proxy13.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.mkdirs(ClientNamenodeProtocolTranslatorPB.java:563)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:191)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy14.mkdirs(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.primitiveMkdir(DFSClient.java:3014)
at org.apache.hadoop.hdfs.DFSClient.mkdirs(DFSClient.java:2984)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1047)
at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1043)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirsInternal(DistributedFileSystem.java:1043)
at org.apache.hadoop.hdfs.DistributedFileSystem.mkdirs(DistributedFileSystem.java:1036)
at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:1880)
at com.hadoop.hdfs.hfs.test(hfs.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
at org.junit.runners.ParentRunner
Connection.setupConnection(Client.java:615)
at org.apache.hadoop.ipc.Client
Connection.access$2900(Client.java:376)
at org.apache.hadoop.ipc.Client.getConnection(Client.java:1529)
at org.apache.hadoop.ipc.Client.call(Client.java:1452)
… 43 more
被这个bug扰了我两天两夜啊!!
我尝试了很多方法都没有用,然后让我一直以为是我的hadoop-eclipse-plugin-2.7.7.jar出了问题,不过它确实是有一点问题:
看我的博客:编译hadoop-eclipse-2.7.7插件
里面最后Success的代码前是有提示警告,这就涉及到你的jdk版本了,一般1.8的比较好,也得跟你的hadoop集群一样。
接着就是关键问题所在了!!
- Hadoop集群防火墙是否关闭
我一直坚信我的防火墙是关闭的了。
特意看了我的集群搭建博客:Hadoop集群搭建(超详细)(伪分布式配置) 是关了的
你们千万千万不要跟我一样啊!!!赶紧的,不管它关没关,咋们再关它一次!!
有几个集群就给我关它几次
systemctl stop firewalld.service
systemctl disable firewalld.service
- 在cmd里检查下master node上的9000端口是否开启了,可以通过telnet的方式测试
telnet 192.168.100.11 9000
第一次基本上是无法连接的,因为一般win7或win10系统中telnet是禁用状态,需要手动启动它。
控制面板-- 系统与安全--左边列里的程序--程序与功能下有个管理员的标志--启用或关闭Windows功能-- √ Telnet Client
这样就打开了Telnet功能了,然后重新在cmd里执行就成功了
telnet 192.168.100.11 9000
成功连接后一般不会有信息,所以不要以为是有错误哦!
回到Eclipse后再尝试连接hadoop就成功了(记得hadoop先连上集群哦)。
集群查看就可以看到新建的文件夹 /dt/tmp1