Author | rhino keepers
Zebian | Xu Veyron
Cover pay to download the visual China
This article lists some of the common tools Kafka, and gave some examples to help understand. There is a need of a small partner, Mark can look up.
surroundings
The following operations are based kafka_2.11-2.2.0
tool
New topic
bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 2 --topic spring-kafka-demo2
replication-factor: Specifies the number of copies
partitions: specify the partition
View topic list
/bin/kafka-topics.sh --zookeeper localhost:2181 --list
To remove a topic
/bin/kafka-topics.sh --zookeeper localhost:2181 --delete -topic spring-kafka-demo
See what consumer groups
./bin/kafka-consumer-groups.sh --bootstrap-server 127.0.0.1:9092 --list
The old version is designated zk address, like this:
kafka-consumer-groups.sh --zookeeper 127.0.0.1:2181 --list
The new version uses bootstrap, this is the difference.
View details of a consumer group
kafka-consumer-groups.sh --new-consumer --bootstrap-server 127.0.0.1:9092 --group test-consumer-group --describe
This is the view the group named test-consumer-group case of consumer groups.
Simple explanation of each column:
TOPIC：
Consumer topic name
PARTITION：
Name of district
CURRENT-OFFSET：
The last time consumer group offset submitted
LOG-END-OFFSET：
Production last message submitted by offset
LAG
The difference between consumption and production offset offset
CONSUMER-ID
Consumer ID number, we know the consumer group which may have at least have a consumer, of course, can have multiple consumers.
HOST
Consumers host IP address.
CLIENT-ID
Linked ID number.
About offset add some knowledge points.
kafka has a common setting is auto.offset.reset , its meaning is
This attribute specifies a read at the consumer without offset or shift the invalid amount partition case (as consumers long time has expired, record including the offset well is obsolete deleted) the process for what. Its default value is the latest, meaning that, under the invalid offset, consumers will start reading the latest recording data (generated after consumers start recording). Earliest is another value, meaning that, in the case where the offset is invalid, the consumer reads the recording start position from the partition.
This property has the following values,
Earliest : When offset has been submitted under the district, from start spending offset submitted; no offset when submitted, scratch consumption
latest data submitted at the time of offset-free, consumption of the newly created partition; when there is offset submitted under the district, from start spending offset submitted:
none : Topic district when there is offset submitted, after the offset from the beginning of the consumer; as long as there is a partition offset submitted does not exist, an exception is thrown
I want to emphasize is that this setting is only our consumers (or consumer groups) to take effect at the time of partition No valid offset.
for example.
We use java kafka client to operate kafak. For example, we in the consumer group group1 there is a consumer, then the message consumed five nodes hung up.
Then we restart the consumer node, then I ask you, the consumer will start spending from where?
If you answer according to auto.offset.reset configured to determine that shows that you do not understand what I said above.
The correct answer is that consumers will continue to spend there continue to hang from the last offset (kafka broker save), simply ignore auto.offset.reset .
Another example.
Producers in the production of a topic some news, then we start a consumer group group2 , which has a consumer.
If this time is not the topic kafka message offset information, then auto.offset.reset value decided where to spend.
"Force plan [the second quarter] - learning ability Challenge" started! From now until March 21, ten million flow to support the original author! Exclusive [more] medal waiting for you to challenge
推荐阅读：云原生的漏洞与威胁有哪些？云原生安全性如何？这里有你想知道的一切！
放心！没人在意你使用的是命令式编程还是声明式编程
如何基于本地语音识别，搭建一款智能聊天机器人？
编程小白模拟简易比特币系统，手把手带你写一波！（附代码） | 博文精选
3 万字 51 张图教你 CPU、内存、操作系统硬核知识！
前沿技术探秘：知识图谱构建流程及方法
真香，朕在看了！
