Kafka安装，SpringBoot整合Kafka，SpringBoot整合Kafka实现日志收集

一、Kafka安装

1. 点击下载地址
   
2. 解压，进入windows目录，启动文件都在这个目录下。
   

二、启动服务

没有java环境先安装java。

1、启动ZooKeeper

进入D:\my_software\kafka_2.13-2.4.1\bin\windows目录，右键打开PowerShell，输入命令

.\zookeeper-server-start.bat ..\..\config\zookeeper.properties

2、启动Kafka

进入D:\my_software\kafka_2.13-2.4.1\bin\windows目录，右键打开PowerShell，输入命令

.\kafka-server-start.bat ..\..\config\server.properties

Kafka默认连接暴露端口：9092

三、SpringBoot整合Kafka

1、引入pom文件

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.6.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>
    <groupId>per.cyl</groupId>
    <artifactId>springboot-kafka-log</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <name>springboot-kafka-log</name>
    <description>Demo project for springboot-kafka-log</description>

    <properties>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>

</project>

2、配置文件

spring:
  kafka:
    # kafka服务器地址，多个集群用逗号分隔
    bootstrap-servers: 127.0.0.1:9092
    producer:
      key-serializer: org.apache.kafka.common.serialization.StringSerializer
      value-serializer: org.apache.kafka.common.serialization.StringSerializer
    consumer:
      group-id: default_consumer_group
      enable-auto-commit: true
      auto-commit-interval: 1000
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer

3、模拟请求产生消息

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * 模拟请求产生消息
 */
@RestController
public class MessageProducerController {
    @Autowired
    private KafkaTemplate<String,Object> kafkaTemplate;
    @RequestMapping("/hello/{name}")
    public String sayHello(@PathVariable(value = "name") String name){
        //使用kafka模板发送信息
        //topic_hello为消息订阅主题
        kafkaTemplate.send("topic_hello", "你好！"+name);
        return name;
    }
}

4、监听消息

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;

/**
 * 监听topic_hello主题产生的消息
 */
@Component
public class MessageConsumer {
    @Autowired
    private KafkaTemplate<String,Object> kafkaTemplate;

    /**
     * 启动spring容器时注册主题
     */
    @Bean
    public void configTopic(){
        kafkaTemplate.send("topic_hello", "topic_hello registered ！");
    }

    @KafkaListener(topics = "topic_hello")
    public void listen (ConsumerRecord<?, ?> record){
        System.out.println("接收到的消息为："+record.value());
    }
}

启动容器时需要使用KafkaTemplate.send()方法将主题注册到Kafka

5、启动项目测试

输入：http://localhost:8080/hello/孙悟空，控制到打印出消息

未完。。。