hermes 是一个不错的基于kafaka 的event broker，基于push模型（webhook）
测试环境使用docker-compose 运行
环境准备
- docker-compose
version: '3'
services:
graphite:
image: nickstenning/graphite
ports:
- "80:80"
- "2003:2003"
frontend:
image: allegro/hermes-frontend
ports:
- "8080:8080"
depends_on:
- zk
- kafka
- graphite
zk:
image: wurstmeister/zookeeper:3.4.6
ports:
- "2181:2181"
kafka:
image: wurstmeister/kafka:1.1.0
ports:
- "9092:9092"
depends_on:
- zk
environment:
KAFKA_ADVERTISED_HOST_NAME: kafka
KAFKA_ADVERTISED_PORT: 9092
KAFKA_ZOOKEEPER_CONNECT: zk:2181
KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'false'
KAFKA_DELETE_TOPIC_ENABLE: 'true'
KAFKA_BROKER_ID: 0
volumes:
- /var/run/docker.sock:/var/run/docker.sock
consumers:
image: allegro/hermes-consumers
depends_on:
- zk
- kafka
- graphite
benthos:
image: jeffail/benthos
volumes:
- "./conf/webhook.yaml:/benthos.yaml"
ports:
- "4195:4195"
management:
image: allegro/hermes-management
ports:
- "8090:8090"
depends_on:
- zk
- kafka
- graphite
- 运行
docker-compose up -d
- webhook 服务
webhook 服务使用benthos,配置如下
input:
type: broker
broker:
inputs:
- type: http_server
http_server:
path: /
processors:
- type: text
text:
operator: prepend
value: "get message: "
output:
type: stdout
- 参考架构图
- 组件说明
frontend 进行消息发送
consumers 消息订阅
manger 管理主题以及订阅
message store 存储以及路由消息当前支持的是kafka
medata store 共享状态，当前使用的是zk
metrics store 可选的，存储metrics 信息，当前是graphite
tracking store 存储追踪信息，当前是es
测试
- 创建group && topic
注意应该使用management 的rest api 地址，对应docker 环境为8090 端口,注意topic的格式
应该为 groupname.topicname
group:
curl -X POST -H "Content-Type: application/json" http://localhost:8090/groups -d '{"groupName": "dalong"}'
topic:
curl -X POST -H "Content-Type: application/json" http://localhost:8090/topics -d '{
"name": "dalong.userlogin",
"description": "This is userlogin topic",
"contentType": "JSON",
"retentionTime": {
"duration": 1
},
"owner": {
"source": "Plaintext",
"id": "dalong"
}
}'
- 创建订阅
注意应该使用management 的rest api 地址，对应docker 环境为8090 端口，注意topic的格式
应该为 groupname.topicname
curl -X POST -H "Content-Type: application/json" http://localhost:8090/topics/dalong.userlogin/subscriptions -d '{
"topicName": "dalong.userlogin",
"name": "userlogin",
"description": "This is my subscription",
"endpoint": "http://benthos:4195/",
"owner": {
"source": "Plaintext",
"id": "dalong"
}
}'
- 查看订阅状态
地址格式： http://localhost:8090/topics/{topicName}/subscriptions/{subscriptionName}/state，比如：
http://localhost:8090/topics/dalong.userlogin/subscriptions/userlogin/state
应该为：ACTIVE
支持的状态有：SUSPENDED，ACTIVE
- 发送
curl -X POST -H "Content-Type: application/json" http://localhost:8080/topics/dalong.userlogin -d '{"message": "Hello world!"}'
- 查看发送消息状态
docker-compose logs -f frontend benthos
效果
Attaching to hermes-project_frontend_1, hermes-project_benthos_1
benthos_1 | {"@timestamp":"2018-12-16T03:28:15Z","@service":"benthos","level":"INFO","component":"benthos","message":"Launching a benthos instance, use CTRL+C to close."}
benthos_1 | {"@timestamp":"2018-12-16T03:28:15Z","@service":"benthos","level":"INFO","component":"benthos","message":"Listening for HTTP requests at: http://0.0.0.0:4195\n"}
benthos_1 | get message: {"message": "Hello world!"}
说明
hermes 使用起来还是比较方便的，但是官方的docker-compose 对于kafaka 的配置有点问题
参考资料
https://hermes-pubsub.readthedocs.io/en/latest/
https://github.com/rongfengliang/hermes-docker-compose