一 、kafka安装配置
1、kafka1.0.0 下载地址
http://kafka.apache.org/downloads.html
2、三台服务器名称
192.168.20.191 master
192.168.20.192 slave1
192.168.20.193 slave2
3、具体安装配置
1).在主节点192.168.20.191的/usr/tools/kafka路径下解压kafka压缩文件： tar -zxvf kafka_2.12-1.0.0.tgz
2).查看配置文件：/usr/tools/kafka/kafka_2.12-1.0.0/config
3).修改server.properties配置文件：vim server.properties
mkdir kafka-logs
三台集群server.properties配置如下：
192.168.20.191：
broker.id=0
port=9092
host.name=192.168.20.191
log.dirs=/usr/tools/kafka/kafka-logs
zookeeper.connect=192.168.20.191:2181,192.168.20.192:2181,192.168.20.193:2181
192.168.20.192：
broker.id=1
port=9092
host.name=192.168.20.192
log.dirs=/usr/tools/kafka/kafka-logs
zookeeper.connect=192.168.20.191:2181,192.168.20.192:2181,192.168.20.193:2181
192.168.20.193：
broker.id=2
port=9092
host.name=192.168.20.193
log.dirs=/usr/tools/kafka/kafka-logs
zookeeper.connect=192.168.20.191:2181,192.168.20.192:2181,192.168.20.193:2181
4).把整个kafka文件夹远程复制到slave1和slave2上：
scp -r /usr/tools/kafka root@slave1:/usr/tools/
scp -r /usr/tools/kafka root@slave2:/usr/tools/
5).复制过去要记得修改server.properties中的broker.id和host.name
6).集群启动和停止（三台分别执行）：/usr/tools/kafka/kafka_2.12-1.0.0/bin
启动：
./kafka-server-start.sh -daemon ../config/server.properties &
停止：
./kafka-server-stop.sh
7).配置环境变量(三台): vim /etc/profile
# KAFKA
export KAFKA_HOME=/usr/tools/kafka/kafka_2.12-1.0.0
export PATH=$PATH:$KAFKA_HOME/bin
使其生效:source /etc/profile
二 、kafka集群启动与停止脚本及其注意事项
1、新建sh脚本
在 /usr/tools/kafka/kafka_2.12-1.0.0/bin目录下新建：
touch start-all-kafka.sh
#!/bin/bash
BROKERS="master slave1 slave2"
APPHOME="/usr/tools/kafka/kafka_2.12-1.0.0"
APP_NAME="kafka_2.12-1.0.0"
for i in $BROKERS
do
echo "Starting ${APP_NAME} on ${i} "
ssh ${i} "source /etc/profile; nohup sh ${APPHOME}/bin/kafka-server-start.sh ${APPHOME}/config/server.properties > /dev/null 2>&1 &"
if [[ $? -ne 0 ]]; then
echo "Starting ${APP_NAME} on ${i} is ok"
fi
done
echo All $APP_NAME are started
exit 0
touch stop-all-kafka.sh
#!/bin/bash
BROKERS="master slave1 slave2"
APPHOME="/usr/tools/kafka/kafka_2.12-1.0.0"
APP_NAME="kafka_2.12-1.0.0"
for i in $BROKERS
do
echo "Stopping ${APP_NAME} on ${i} "
ssh ${i} "source /etc/profile;bash ${APPHOME}/bin/kafka-server-stop.sh"
if [[ $? -ne 0 ]]; then
echo "Stopping ${APP_NAME} on ${i} is down"
fi
done
echo All $APP_NAME are stopped
exit 0
2、授权
chmod +x *.sh
3、启动或停止（./start-all-kafka.sh和./stop-all-kafka.sh）
4、创建和查看topic（此处创建的是名称为“my-topic”的topic）
191上创建：
./kafka-topics.sh --create --zookeeper master:2181 --replication-factor 1 --partitions 1 --topic my-topic
191上查看：
./kafka-topics.sh --list --zookeeper master:2181
5、发送消息
191上生产：
./kafka-console-producer.sh --broker-list master:9092 --topic my-topic
6、启动consumer
192或193上读取消息并输出：
./kafka-console-consumer.sh --zookeeper master:2181 --topic my-topic --from-beginning
