版权声明：本文为博主原创文章，未经博主允许不得转载。 https://blog.csdn.net/Lockey23/article/details/81571383
基础配置同上篇博客
配置教程请先参阅：厦门大学数据库实验室系列博客
Spark 2.0分布式集群环境搭建
需要注意的配置有两个
cd /usr/local/spark/
cp ./conf/slaves.template ./conf/slaves
#slaves文件设置Worker节点。编辑slaves内容,把默认内容localhost替换成如下内容：
slave1
slave2
配置spark-env.sh文件
将 spark-env.sh.template 拷贝到 spark-env.sh
cp ./conf/spark-env.sh.template ./conf/spark-env.sh
编辑spark-env.sh,添加如下内容：
export SPARK_DIST_CLASSPATH=$(/usr/local/hadoop/bin/hadoop classpath)
export HADOOP_CONF_DIR=/usr/local/hadoop/etc/hadoop
export SPARK_MASTER_IP=192.168.137.129
配置完成后启动master：
lockey@master:/usr/local$ ./spark/sbin/start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /usr/local/spark/logs/spark-lockey-org.apache.spark.deploy.master.Master-1-master.out
lockey@ubuntu-lockey:/usr/local/spark$ jps
16371 Master
16421 Jps
15063 SecondaryNameNode
14840 NameNode
15210 ResourceManager
然后再启动两个slave（此命令在master上执行）：
lockey@master:/usr/local$ ./spark/sbin/start-slaves.sh
slave1: starting org.apache.spark.deploy.worker.Worker, logging to /usr/local/spark/logs/spark-lockey-org.apache.spark.deploy.worker.Worker-1-slave1.out
slave2: starting org.apache.spark.deploy.worker.Worker, logging to /usr/local/spark/logs/spark-lockey-org.apache.spark.deploy.worker.Worker-1-slave2.out
lockey@slave1:/usr/local$ jps
1832 Jps
1578 NodeManager
1435 DataNode
1787 Worker
然后我们来验证以下web界面吧，master端口为8080：
启动slaves之后的界面会多出两个Worker
我们再来看一下worker的web界面：
好了，到这里我们的spark简版集群就搭建好了
关闭Spark集群
关闭Master节点
sbin/stop-master.sh
关闭Worker节点
sbin/stop-slaves.sh
关闭Hadoop集群
cd /usr/local/hadoop/
sbin/stop-all.sh