Install Php extension:
pecl install rdkafa If an error like re2c-1.0.2 is reported, please install re2c-1.0.2.tar
PHP example: You can also refer to the official document, https://arnaud.le-blanc.net/php-rdkafka-doc/phpdoc/rdkafka.examples-high-level-consumer.html
//生产例子
$conf = new RdKafka\Conf();
$conf->set('metadata.broker.list', '192.168.0.33:9092,192.168.0.185:9092,192.168.0.193:9092');
$objRdKafka = new RdKafka\Producer($conf);
$topic = $objRdKafka->newTopic("test");
//$oObjTopic = $objRdKafka->newTopic("test");
// 发送消息
$sMsg = json_encode(["code" => 200, "data" => "sfsfsfs"]);
$topic->produce(RD_KAFKA_PARTITION_UA, 0, $sMsg);
$objRdKafka->poll(0);
$result = $objRdKafka->flush(10000);
var_dump($result);
//消息例子
$conf = new RdKafka\Conf();
// Set a rebalance callback to log partition assignments (optional)
$conf->setRebalanceCb(function (RdKafka\KafkaConsumer $kafka, $err, array $partitions = null) {
switch ($err) {
case RD_KAFKA_RESP_ERR__ASSIGN_PARTITIONS:
echo "Assign: ";
var_dump($partitions);
$kafka->assign($partitions);
break;
case RD_KAFKA_RESP_ERR__REVOKE_PARTITIONS:
echo "Revoke: ";
var_dump($partitions);
$kafka->assign(NULL);
break;
default:
throw new \Exception($err);
}
});
// Configure the group.id. All consumer with the same group.id will consume
// different partitions.
$conf->set('group.id', 'myConsumerGroup');
// Initial list of Kafka brokers
$conf->set('metadata.broker.list', '192.168.0.33:9092,192.168.0.185:9092,192.168.0.193:9092');
// Set where to start consuming messages when there is no initial offset in
// offset store or the desired offset is out of range.
// 'smallest': start from the beginning
$conf->set('auto.offset.reset', 'smallest');
$consumer = new RdKafka\KafkaConsumer($conf);
// Subscribe to topic 'test'
$consumer->subscribe(['test']);
echo "Waiting for partition assignment... (make take some time when\n";
echo "quickly re-joining the group after leaving it.)\n";
while (true) {
$message = $consumer->consume(120 * 1000);
switch ($message->err) {
case RD_KAFKA_RESP_ERR_NO_ERROR:
var_dump($message);
break;
case RD_KAFKA_RESP_ERR__PARTITION_EOF:
echo "No more messages; will wait for more\n";
break;
case RD_KAFKA_RESP_ERR__TIMED_OUT:
echo "Timed out\n";
break;
default:
throw new Exception($message->errstr(), $message->err);
break;
}
}
For the installation of Kafka cluster, see another blog of mine. Common commands are:
Note: Whether kafka is started successfully, if not, you need to find a solution based on the error.
启动服务器
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-server-start.sh -daemon ../config/server.properties
关闭服务器
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-server-stop.sh
创建Topic
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-topics.sh --create --zookeeper 127.0.0.1:2181 --replication-factor 1 --partitions 1 --topic test
列出所有Topic:
bin/kafka-topics.sh -list -zookeeper 192.168.0.185:2181
启动Producer并发送消息:
/usr/local/kafka/kafka_2.12-0.11.0.0/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic test
启动Consumer并接收消息:
kafka-console-consumer.sh --bootstrap-server 192.168.0.185:9092,192.168.0.33:9092,192.168.0.193:9092 --topic test --from-beginning