最全的手把手单机 部署伪集群zookeeper和kafka

注意，下面所有ip为127.0.0.1需替换成本机ip，否则后续客户端通过ip连接则会连接不成功

部署伪集群zookeeper

创建目录，上传包

# 新建目录
mkdir /opt/zookeeper
cd /opt/zookeeper
# 上传apache-zookeeper-3.6.4-bin.tar.gz包 
tar zxf apache-zookeeper-3.6.4-bin.tar.gz
mv apache-zookeeper-3.6.4-bin zookeeper-1

编辑zk节点1配置文件

cd zookeeper-1/conf/
mv zoo_sample.cfg zoo.cfg
vim zoo.cfg

主要修改如下几个地方，端口都要修改，日志目录每个节点都不一样才可以

clientPort=2181
dataLogDir=/opt/zookeeper/zookeeper-1/logs
dataDir=/opt/zookeeper/zookeeper-1/data
server.1=127.0.0.1:2888:3888
server.2=127.0.0.1:2889:3889
server.3=127.0.0.1:2890:3890

创建节点id文件

mkdir /opt/zookeeper/zookeeper-1/data
echo "1" > /opt/zookeeper/zookeeper-1/data/myid 

节点1配置结束

------------------------------------------------------------------------------------------------------------------------

复制节点2和节点3

cd /opt/zookeeper
cp -r zookeeper-1 zookeeper-2
cp -r zookeeper-1 zookeeper-3

配置节点2

cd zookeeper-2
vi conf/zoo.cfg

修改如下

clientPort=2182
dataLogDir=/opt/zookeeper/zookeeper-2/logs
dataDir=/opt/zookeeper/zookeeper-2/data

创建节点id文件

echo "2" > /opt/zookeeper/zookeeper-2/data/myid 

配置节点3

cd /opt/zookeeper/zookeeper-3
vi conf/zoo.cfg

修改如下

clientPort=2183
dataLogDir=/opt/zookeeper/zookeeper-3/logs
dataDir=/opt/zookeeper/zookeeper-3/data

创建节点id文件

echo "3" > /opt/zookeeper/zookeeper-3/data/myid 

启动zk

/opt/zookeeper/zookeeper-1/bin/zkServer.sh start
/opt/zookeeper/zookeeper-2/bin/zkServer.sh start
/opt/zookeeper/zookeeper-3/bin/zkServer.sh start

查看zk节点状态

/opt/zookeeper/zookeeper-1/bin/zkServer.sh status
/opt/zookeeper/zookeeper-2/bin/zkServer.sh status
/opt/zookeeper/zookeeper-3/bin/zkServer.sh status

部署伪集群kafka

创建目录上传包

mkdir /opt/kafka
cd /opt/kafka
#上传 kafka_2.13-2.8.2.tgz
tar zxf kafka_2.13-2.8.2.tgz
mv kafka_2.13-2.8.2 kafka-1

编辑kafka配置文件

vim kafka-1/config/server.properties

主要修改如下

broker.id=0 # 节点1
listeners=PLAINTEXT://127.0.0.1:9092  # 这个默认配置没有新加
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183   # 这个默认配置有，注释原配置新增
log.dirs=/opt/kafka/kafka-1/kafka-logs   # 这个默认配置有，注释原配置新增

复制kafka-2，3

cp -r kafka-1 kafka-2
cp -r kafka-1 kafka-3

配置kafka-2

vim kafka-2/config/server.properties

broker.id=1 # 节点2
listeners=PLAINTEXT://127.0.0.1:9093
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/opt/kafka/kafka-2/kafka-logs

配置kafka-3

vim kafka-3/config/server.properties

broker.id=2 # 节点3
listeners=PLAINTEXT://127.0.0.1:9094
zookeeper.connect=127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
log.dirs=/opt/kafka/kafka-3/kafka-logs

启动kafka

# 启动kafka
/opt/kafka/kafka-1/bin/kafka-server-start.sh -daemon /opt/kafka/kafka-1/config/server.properties
/opt/kafka/kafka-2/bin/kafka-server-start.sh -daemon /opt/kafka/kafka-2/config/server.properties
/opt/kafka/kafka-3/bin/kafka-server-start.sh -daemon /opt/kafka/kafka-3/config/server.properties

下面命令创建和查看运行是否正常

# 创建名为test的topic
/opt/kafka/kafka-3/bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183 --replication-factor 3 --partitions 3 --topic test
# 列出已创建的topic
/opt/kafka/kafka-3/bin/kafka-topics.sh --list --zookeeper 127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183
# 模拟生产者
/opt/kafka/kafka-3/bin/kafka-console-producer.sh --broker-list 172.25.25.3:9092,172.25.25.4:9092,172.25.25.5:9092 --topic test
# 模拟消费者
/opt/kafka/kafka-3/bin/kafka-console-consumer.sh --bootstrap-server 172.25.25.3:9092,172.25.25.4:9092,172.25.25.5:9092 --from-beginning --topic test
# 查看指定主题
/opt/kafka/kafka-3/bin/kafka-topics.sh --describe --zookeeper 127.0.0.1:2181,127.0.0.1:2182,127.0.0.1:2183 --topic test