1.Kafka概述
和消息系统类似
消息中间件:流水线中的缓冲区
可进行集群部署
2.Kafka架构
producer:生产者
consumer:消费者
broker:缓存区
topic:给产品打标签,分配给指定的consumer
3.Kafka部署
/home/liaomj/app/kafka-0.9.0.0-src/config/server
.properties
broker.id=0
listeners=PLAINTEXT://hadoop000:9092
host.name=hadoop000
log.dirs=/home/liaomj/app/tmp/kafka-logs
zookeeper.connect=hadoop000:2181
启动
bin
/kafka-server-start
.sh config
/server
.properties
bin/kafka-server-start.sh -daemon config/server.properties --后台
创建topic
bin/kafka-topics.sh --create --zookeeper hadoop000:2181 --replication-factor 1 --partitions 1 --topic test
查看所有topic
bin/kafka-topics.sh --list --zookeeper hadoop000:2181
发送消息
bin/kafka-console-producer.sh --broker-list hadoop000:9092 --topic test
消费消息
bin
/kafka-console-consumer
.sh --zookeeper hadoop000:2181 --topic
test
--from-beginning
配置多个broker
Setting up a multi-broker cluster