kafka is a distributed publish-subscribe messaging system, also called the MQ systems MQ is Message Queue, the message queue.
Popular point, the producers to write a message queue, consumers read from the queue. Professional point, Producer send a message to the cluster via TCP protocol Kafka, Kafka cluster and then provide these messages to the Consumer.
Message is composed of a key, value, time stamps, classified by topic. Work often heard topic, it is this topic, used to classify messages.
Cheng Ruguan entire flow shown in FIG net painting.
kafka the concept of a Broker, refers to a cluster of servers, each server is a proxy (Broker).
Consumers often demand into a plurality of groups, it referred to the consumer group (consumer group).
kafka asynchronous transfer messages to solve the problem caused by parallel partition topic of consumption.
Each partition has a leader, zero or more follower. All read and write requests Leader process of this partition, and a passive follower copy data. If the Leader down the enemy, generate new Leader of the follower. Broker may also be a partition of a leader, follower of another partition.
Producers hair which partition the topic made it? The easiest way is to select alternate partition from the list. You can also press a particular algorithm selected partition.
Message has two modes, the queue type and publish - subscribe. Queue, a message that only a consumer handling. Publish - subscribe, broadcasts a message to all consumers, since the consumer receives the message can handle.
Partitions are divided a message sequence number, called an offset (offset).
Consumer data is held by the offset. This is set by the consumers themselves, so you can read the history of the message.
At most once --- messages may be lost, but will not be re-issued.
At least once --- no messages will be lost, but it is possible to re-send.
--- exactly once for each message delivery once and only once.
kafka default is "at least once." By prohibiting the producers of retries to achieve "more than once." To achieve "exactly once" by adding the offset.
kafka does not delete messages consumed immediately, exactly when deleted by the Broker configuration decisions.
Copyright Notice: This article blogger original article, reproduced please retain the original link and author.