2016-09-09 15:32:57,749 INFO [main-SendThread(madwx31:2181)]
zookeeper.ClientCnxn: Socket connection established to madwx31/192.168.32.31:2181, initiating session 2016-09-09 15:32:57,749 INFO [main-SendThread(madwx31:2181)] zookeeper.ClientCnxn: Unable to read additional data from server sessionid 0x0, likely server has closed socket, closing socket connection and attempting reconnect 2016-09-09 15:32:59,379 INFO [main-SendThread(madwx51:2181)] zookeeper.ClientCnxn: Opening socket connection to server madwx51/192.168.32.51:2181. Will not attempt to authenticate using SASL (unknown error) 2016-09-09 15:32:59,380 INFO [main-SendThread(madwx51:2181)] zookeeper.ClientCnxn: Socket connection established to madwx51/192.168.32.51:2181, initiating session 2016-09-09 15:32:59,380 WARN [main-SendThread(madwx51:2181)] zookeeper.ClientCnxn: Session 0x0 for server madwx51/192.168.32.51:2181, unexpected error, closing socket connection and attempting reconnect java.io.IOException: Connection reset by peer at sun.nio.ch.FileDispatcherImpl.read0(Native Method) at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39) at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223) at sun.nio.ch.IOUtil.read(IOUtil.java:192) at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379) at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68) at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081) 2016-09-09 15:32:59,481 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper create failed after 4 attempts 2016-09-09 15:32:59,481 ERROR [main] master.HMasterCommandLine: Master exiting java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster. at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2422) at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:232) at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:138) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126) at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2432) Caused by: org.apache.hadoop.hbase.ZooKeeperConnectionException: master:160000x0, quorum=madwx51:2181,madwx41:2181,madwx31:2181, baseZNode=/hbase Unexpected KeeperException creating base node at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:206) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:187) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:585) at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:379) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2415) ... 5 more Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase at org.apache.zookeeper.KeeperException.create(KeeperException.java:99) at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.createNonSequential(RecoverableZooKeeper.java:575) at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.create(RecoverableZooKeeper.java:554) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1204) at org.apache.hadoop.hbase.zookeeper.ZKUtil.createWithParents(ZKUtil.java:1182) at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:194) ... 13 more
一看，大概意思就是连不上zookeeper，但是检查zookeeper服务一切都正常，但是当查看到zookeeper的日志文件时候发现报too many connections from host - max is 10 的错误，明显是hbase连接zookeeper的连接数太多了，所以在zookeeper的zoo.cfg配置文件中加上如下配置：
maxClientCnxns=300
然后重启zookeeper，hbase问题解决。
maxClientCnxns=300
默认zookeeper给每个客户端IP使用的连接数为10个，经常会出现连接不够用的情况。修改连接数目前好像只支持zoo.cfg配置文件修改，所以需要zookeeper重启才能生效。
zoo.cfg:
maxClientCnxns=300
否则报错如下：2011-10-28 09:39:44,856 – WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:5858:NIOServerCnxn$Factory@253] – Too many connections from /192.168.*.* – max is 10