Java easiest to use operating Kafka--

 

java kafka very simple operation, then kafka also offers many defaults, under normal circumstances we do not need to modify too many parameters you can use. Now I posted the code.

pom.xml

<dependency>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
    <version>0.10.2.0</version>
</dependency>

Producers

package cn.duanjt;
import java.util.Properties;
import java.util.Random;
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

public class Producer {
    public static String topic = "duanjt_test";//定义主题
    public static void main(String[] args) throws InterruptedException {
        Properties p = new Properties();
        p.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.23.76:9092,192.168.23.77:9092");//kafka地址，多个地址用逗号分割
        p.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        p.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        KafkaProducer<String, String> kafkaProducer = new KafkaProducer<>(p);
        try {
            while (true) {
                String msg = "Hello," + new Random().nextInt(100);
                ProducerRecord<String, String> record = new ProducerRecord<String, String>(topic, msg);
                kafkaProducer.send(record);
                System.out.println("消息发送成功:" + msg);
                Thread.sleep(500);
            }
        } finally {
            kafkaProducer.close();
        }

    }
}

note:

1.kafka If the cluster, a plurality of addresses separated by commas (,)

2.Properties put the method, the first parameter can be a string, such as: p.put ( "bootstrap.servers", "192.168.23.76:9092")

3.kafkaProducer.send (record) can determine whether to send kafka, enhance the reliability of the message by returning Future. At the same time the second parameter may be used to send the callback, the callback determines whether to transmit successfully.

Full path setup sequence class, the class can be written; 4.p.put (ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class)

consumer

package cn.duanjt;
import java.util.Collections;
import java.util.Properties;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;
public class Consumer {
    public static void main(String[] args) {
        Properties p = new Properties();
        p.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.23.76:9092");
        p.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        p.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        p.put(ConsumerConfig.GROUP_ID_CONFIG, "duanjt_test");
        KafkaConsumer<String, String> kafkaConsumer = new KafkaConsumer<String, String>(p);
        kafkaConsumer.subscribe(Collections.singletonList(Producer.topic));// 订阅消息
        while (true) {
            ConsumerRecords<String, String> records = kafkaConsumer.poll(100);
            for (ConsumerRecord<String, String> record : records) {
                System.out.println(String.format("topic:%s,offset:%d,消息:%s", //
                        record.topic(), record.offset(), record.value()));
            }
        }
    }
}

note:

1. Subscribe to news can subscribe to multiple topics

2.ConsumerConfig.GROUP_ID_CONFIG indicates a packet of consumers, kafka judgment based on the group name is not the same group of consumers, the same group of consumers to spend a topic of data when the data in this group of consumers polled above.

3. The theme involves the concept of partition, the number of the same group of consumers can not be greater than the number of partitions. Because: a partition can only be consumed by a consumer in the same group. Partition is less than the number of consumers they occur, you can dynamically increase partition.

4. Note that comparison and producers, Properties of key and value are deserialized, and the producers are serialized.