1.搭建kafka运行环境 ,下载地址:http://kafka.apache.org/downloads
- 下载完修改安装文件config里面的server.properties文件,将listeners=PLAINTEXT://IP:9092改成服务器的IP,修改zookeeper注册地址
- .\bin\windows\kafka-server-start.bat .\config\server.properties使用该命令启动kafka
- 测试一下是否可以发送消息
bin\windows\kafka-console-producer.bat –broker-list IP9:092 –topic pipi
bin\windows\kafka-console-consumer.bat –bootstrap-server IP:9092 –topic
pipi –from- beginning
2.搭建生产者环境
- 配置producer.xml文件
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- 定义producer属性-->
<bean id="kafakaproducer" class="java.util.HashMap">
<constructor-arg>
<map>
<entry key="bootstrap.servers" value="10.168.114.91:9092"/>
<entry key="group.id" value="0"/>
<entry key="retries" value="10"/>
<entry key="batch.size" value="16384"/>
<entry key="linger.ms" value="1"/>
<entry key="key.serializer" value="org.apache.kafka.common.serialization.IntegerSerializer"/>
<entry key="value.serializer" value="org.apache.kafka.common.serialization.StringSerializer"/>
</map>
</constructor-arg>
</bean>
<!--配置kafka的工厂 -->
<bean id="producerFactory" class="org.springframework.kafka.core.DefaultKafkaProducerFactory">
<constructor-arg>
<ref bean="kafakaproducer"/>
</constructor-arg>
</bean>
<bean id="kafkaTemplate" class="org.springframework.kafka.core.KafkaTemplate">
<constructor-arg>
<ref bean="producerFactory"/>
</constructor-arg>
<property name="defaultTopic" value="mhb-test"/>
</bean>
</beans>
- 测试类
public class SpringKafkaTest {
public static void main(String[] args) {
ApplicationContext ctx = new ClassPathXmlApplicationContext("classpath:config/spring-kafka.xml");
KafkaTemplate kafkaTemplate = ctx.getBean("kafkaTemplate", KafkaTemplate.class);
for(int i=0;i<1000000000;i++){
String message="第"+i+"条信息";
kafkaTemplate.send("pipi",message);
System.out.println("message="+message);
}
}
}
3 . 搭建消费者环境
- 配置consumer.xml文件
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<!-- 定义消费者属性 -->
<bean id="kafkaConsumerProperites" class="java.util.HashMap">
<constructor-arg>
<map>
<entry key="bootstrap.servers" value="10.168.114.91:9092" />
<entry key="group.id" value="0"/>
<entry key="enable.auto.commit" value="true"/>
<entry key="auto.commit.interval.ms" value="1000"/>
<entry key="session.timeout.ms" value="15000"/>
<entry key="key.deserializer" value="org.apache.kafka.common.serialization.IntegerDeserializer"/>
<entry key="value.deserializer" value="org.apache.kafka.common.serialization.StringDeserializer"/>
</map>
</constructor-arg>
</bean>
<!--配置kafka的工厂 -->
<bean id="kafkaConsumerFactory"
class="org.springframework.kafka.core.DefaultKafkaConsumerFactory">
<constructor-arg>
<ref bean="kafkaConsumerProperites" />
</constructor-arg>
</bean>
<!--消费者具体实体类-->
<bean id="KafkaConsumerListener" class="service.KafkaConsumerListener" />
<!--配置容器属性-->
<bean id="consumerContainerProperties"
class="org.springframework.kafka.listener.config.ContainerProperties">
<constructor-arg value="pipi" />
<property name="messageListener" ref="KafkaConsumerListener" />
</bean>
<!--配置消费者容器-->
<bean id="conusmerContainer"class="org.springframework.kafka.listener.KafkaMessageListenerContainer " init-method="doStart">
<constructor-arg ref="kafkaConsumerFactory" />
<constructor-arg ref="consumerContainerProperties" />
</bean>
</beans>
- 配置监听器
@Component("kafkaConsumerListener")
public class KafkaConsumerListener implements MessageListener<String, String>{
@Override
public void onMessage(ConsumerRecord<String, String> record) {
System.out.printf("接收到消息="+ record.value());
}
}
- 消费者是web项目,启动后监听器就开始工作了,然后跑生产者的测试类就可以在消费者中控制台中看到日志了、
一个kafka消费者的管理工具kafkamonitor 在界面中就能看到消息模式和消费者
- kafkamonitor是一个jar,只需把它放在kafka安装目录windows下,编写一个脚本并运行即可启动项目
java -cp KafkaOffsetMonitor-assembly-0.2.0.jar com.quantifind.kafka.offsetapp.OffsetGetterWeb --zk 10.168.114.91:2181 --port 8089 --refresh 10.seconds --retain 1.days