RocketMQ 编码快速入门

本文导读

• 本文承接《RocketMQ 安装详细说明》，在成功安装了 RocketMQ 之后就可以自己编码通信了

• Name Server 是一个几乎无状态节点，可集群部署，节点之间无任何信息同步。
• Broker 部署相对复杂，Broker 分为 Master 与 Slave，一个 Master（主人） 可以对应多个 Slave(奴隶)，但是一个Slave 只能对应一个 Master，Master与Slave的对应关系通过指定相同的 BrokerName，不同的BrokerId来定义，BrokerId为0表示Master，非0表示Slave。Master也可以部署多个，每个Broker与Name Server集群中的所有节点建立长连接，定时注册Topic 信息到所有 Name Server。
• Producer 与 Name Server 集群中的其中一个节点（随机选择）建立长连接，定期从Name Server取Topic路由信息，并向提供 Topic 服务的 Master 建立长连接，且定时向 Master 发送心跳。Producer完全无状态，可集群部署。
• Consumer 与 Name Server 集群中的其中一个节点（随机选择）建立长连接，定期从Name Server取Topic路由信息，并向提供 Topic 服务的 Master、Slave 建立长连接，且定时向 Master、Slave发送心跳。Consumer既可以从Master订阅消息，也可以从Slave订阅消息，订阅规则由Broker配置决定。

特别提醒

• 本文虽然一起合成，但是实际第一次操作时出现了很多问题，然后慢慢的一一解决了，在此所以先做提醒
• 1）启动 broker 时指定的端口，应该在防火墙中开放，否则客户端连接不上
• 2）如果使用的 虚拟机 这种配置不高的 Linux 系统，建议直接关闭 Linux 防火墙进行学习 RocketMQ ，否则即使 防火墙放开了 broker 指定是的端口，也很可能因为网络问题，而导致客户端连接失败，或者消息发送失败
• 可以参考《RocketMQ 常见异常处理》

新建项目

• 主要以学习 RocketMQ API 为主，所以这里新建传统的 Java SE 项目，不使用 Maven 以及 Spring Boot 等

获取 RocketMQ 开发包

• 不采用 Maven 工程，而是直接手动导包。进入官方的快速入门页面：https://rocketmq.apache.org/docs/quick-start/

导入 RocketMQ 开发包

RockatMQ Hello World

• 导入的 jar 中有一个 rcokermq-example-4.3.0.jar，里面全是官方提供的例子，这里选择里面那个快速入门的例子来进行学习

开启 nameServer

• 对启动指令不熟悉时，可以参考《 RocketMQ 安装详细说明》，如下所示 nameServer 启动成功

Last login: Sat Aug 11 23:21:59 2018
[root@localhost ~]# cd /usr/local/rocketmq-all-4.3.0/distribution/target/apache-rocketmq
[root@localhost apache-rocketmq]# nohup sh bin/mqnamesrv &
[1] 3033
[root@localhost apache-rocketmq]# nohup: 忽略输入并把输出追加到"nohup.out"
tail -f ~/logs/rocketmqlogs/namesrv.log
2018-08-11 23:43:19 INFO main - tls.client.keyPath = null
2018-08-11 23:43:19 INFO main - tls.client.keyPassword = null
2018-08-11 23:43:19 INFO main - tls.client.certPath = null
2018-08-11 23:43:19 INFO main - tls.client.authServer = false
2018-08-11 23:43:19 INFO main - tls.client.trustCertPath = null
2018-08-11 23:43:20 INFO main - Using OpenSSL provider
2018-08-11 23:43:22 INFO main - SSLContext created for server
2018-08-11 23:43:22 INFO main - The Name Server boot success. serializeType=JSON
2018-08-11 23:43:22 INFO NettyEventExecutor - NettyEventExecutor service started
2018-08-11 23:43:22 INFO FileWatchService - FileWatchService service started

开启 broker

• 如下所示 broker 启动成功，注册到 nameServer 没有问题

[root@localhost apache-rocketmq]# nohup sh bin/mqbroker -n localhost:9876 &
[2] 3256
[1]   退出 143              nohup sh bin/mqbroker -n localhost:9876
[root@localhost apache-rocketmq]# nohup: 忽略输入并把输出追加到"nohup.out"
tail -f ~/logs/rocketmqlogs/broker.log 
2018-08-11 23:48:26 INFO main - load exist subscription group, SubscriptionGroupConfig [groupName=CID_ONSAPI_OWNER, consumeEnable=true, consumeFromMinEnable=true, consumeBroadcastEnable=true, retryQueueNums=1, retryMaxTimes=16, brokerId=
2018-08-11 23:48:31 WARN main - Load default discard message hook service: DefaultTransactionalMessageCheckListener
2018-08-11 23:48:31 INFO FileWatchService - FileWatchService service started
2018-08-11 23:48:31 INFO PullRequestHoldService - PullRequestHoldService service started
2018-08-11 23:48:31 INFO brokerOutApi_thread_1 - register broker to name server localhost:9876 OK
2018-08-11 23:48:31 INFO main - Start transaction service!
2018-08-11 23:48:31 INFO main - The broker[localhost.localdomain, 192.168.58.129:10911] boot success. serializeType=JSON and name server is localhost:9876
2018-08-11 23:48:41 INFO BrokerControllerScheduledThread1 - dispatch behind commit log 0 bytes
2018-08-11 23:48:41 INFO BrokerControllerScheduledThread1 - Slave fall behind master: 906412 bytes
2018-08-11 23:48:41 INFO brokerOutApi_thread_2 - register broker to name server localhost:9876 OK

Producer（生产者）

package com.lct.quickstart;

import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;

/**
 * Created by Administrator on 2018/8/11 0011.
 * 生产者  发送消息
 */

public class Producer {
    public Producer() {
    }

    public static void main(String[] args) throws MQClientException, InterruptedException {
        /**
         * 声明并初始化一个 producer,同时指定 Producer Group 的名称
         * 一个应用创建一个 Producer，由应用来维护此对象，可以设置为全局对象或者单例
         *  Producer Group 的名称 "producerGroupName" 需要由应用来保证唯一性
         * ProducerGroup 这个概念发送普通的消息时，作用不大，但是发送分布式事务消息时，比较关键，
         * 因为服务器会回查这个 Group 下的任意一个 Producer
         */
        DefaultMQProducer producer = new DefaultMQProducer("producerGroupName");
        /**
         * 指定 Producer 连接的 nameServer 服务器所在地址以及端口
         * 如果是分布式部署的多个，则用分号隔开，如：
         * setNamesrvAddr("172.16.235.77:9876;172.16.235.78:9876");
         * 这里只是为了方便才将地址与端口写死，实际中应该至少放在配置文件中去
         */
        producer.setNamesrvAddr("192.168.58.129:9876");
        /**
         * 指定自己的在 Producer Group 中的 名称
         */
        producer.setInstanceName("producer1");

        /**
         * Producer 对象在使用之前必须要调用 start 进行启动初始化
         * 初始化一次即可，切忌不可每次发送消息时，都调用start方法
         */
        producer.start();

        /**
         * 一个 Producer 对象可以发送多个 topic（主题），多个 tag 的消息
         * 本实例 send 方法采用同步调用，只要不抛异常就标识成功
         */
        for (int i = 0; i < 10; ++i) {
            try {
                Message message = null;
                if (i < 5) {
                    message = new Message("TopicTest1", "TagA", ("Hello RocketMQ " + i).getBytes("UTF-8"));
                } else {
                    message = new Message("TopicTest2", "TagB", ("我爱你,中国 " + i).getBytes("UTF-8"));
                }
                SendResult sendResult = producer.send(message);
                System.out.printf("%s%n", new Object[]{sendResult});
            } catch (Exception var5) {
                var5.printStackTrace();
                Thread.sleep(1000L);
            }
        }
        /**
         * 应用退出时，调用 shutdown 关闭网络连接，清理资源，从 MocketMQ 服务器上注销自己
         * 建议应用在 JBOSS、Tomcat 等容器的退出钩子里调用 shutdown 方法
         */
        producer.shutdown();
    }
}

Consumer（消费者）

package com.lct.quickstart;

/**
 * Created by Administrator on 2018/8/11 0011.
 */

import org.apache.rocketmq.client.consumer.DefaultMQPushConsumer;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import org.apache.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import org.apache.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.common.consumer.ConsumeFromWhere;
import org.apache.rocketmq.common.message.MessageExt;

import java.util.List;

public class Consumer {
    public Consumer() {
    }

    /**
     * 当前例子是 PushConsumer 用法，给用户感觉是消息从 RocketMQ 服务器推到了应用客户端。
     * 而实际 PushConsumer 内部是使用长轮询 Pull(拉取) 方式从 MetaQ 服务器拉消息，然后再回调用户 Listener方法
     *
     * @param args
     * @throws InterruptedException
     * @throws MQClientException
     */
    public static void main(String[] args) throws InterruptedException, MQClientException {

        /**
         * 声明并初始化 一个 consumer
         * Consumer Group 组名，多个 Consumer 如果属于一个应用，订阅同样的消息，且消费逻辑一致，则应该将它们归为同一组
         * 一个应用创建一个 Consumer，由应用来维护此对象，可以设置为全局对象或者单例
         * ConsumerGroupName 需要由应用来保证唯一
         */
        DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumerGroupName");
        /**
         * 指定 NameServer 的地址 与端口
         * 指定自己在 Consumer Group 组中的名称
         */
        consumer.setNamesrvAddr("192.168.58.129:9876");
        consumer.setInstanceName("consumber");
        /**
         * 设置 consumer 的消费策略
         * CONSUME_FROM_LAST_OFFSET 默认策略，从该队列最尾开始消费，即跳过历史消息
         * CONSUME_FROM_FIRST_OFFSET 从队列最开始开始消费，即历史消息（还储存在broker的）全部消费一遍
         * CONSUME_FROM_TIMESTAMP 从某个时间点开始消费，和 setConsumeTimestamp() 配合使用，默认是半个小时以前
         */
        consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET);

        /**
         * 消费者订阅消息，如下所示订阅 topic 为 TopicTest2 下的所有 tag 类型 的消息
         * 订阅指定 topic下 tags 分别等于TagA或TagB或TagC，则为：consumer.subscribe("TopicTest", "TagA || TagB || TagC");
         * 一个 consumer 对象可以订阅多个 topic
         */
        consumer.subscribe("TopicTest2", "*");

        /**
         * 注册消息监听器，如果有订阅的消息就会响应
         */
        consumer.registerMessageListener(new MessageListenerConcurrently() {
            /**
             * 默认 msgs 里只有一条消息，可以通过设置 consumeMessageBatchMaxSize 参数来批量接收消息
             * consumeThreadMin:消费线程池数量 默认最小值10
             * consumeThreadMax:消费线程池数量 默认最大值20
             */
            public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs, ConsumeConcurrentlyContext context) {
                System.out.println(Thread.currentThread().getName() + " Receive New Messages: " + msgs.size());
                MessageExt msg = msgs.get(0);
                System.out.printf("%s Receive New Messages: %s %n", new Object[]{Thread.currentThread().getName(), msgs});
                System.out.println("----------:" + new String(msg.getBody()));
                /**
                 * 返回消费状态
                 * CONSUME_SUCCESS 消费成功
                 * RECONSUME_LATER 消费失败，需要稍后重新消费
                 */
                return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
            }
        });
        /**
         * Consumer对象在使用之前必须要调用start初始化，初始化一次即可<br>
         */
        consumer.start();
        System.out.printf("Consumer Started.%n", new Object[0]);
    }
}

运行测试

• 运行 Producer 输出信息如下，即发送消息成功。

17:00:09.904 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597690000, offsetMsgId=C0A83A8100002A9F00000000000DFBE2, messageQueue=MessageQueue [topic=TopicTest1, brokerName=localhost.localdomain, queueId=3], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597750001, offsetMsgId=C0A83A8100002A9F00000000000DFC95, messageQueue=MessageQueue [topic=TopicTest1, brokerName=localhost.localdomain, queueId=0], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597780002, offsetMsgId=C0A83A8100002A9F00000000000DFD48, messageQueue=MessageQueue [topic=TopicTest1, brokerName=localhost.localdomain, queueId=1], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F03725977C0003, offsetMsgId=C0A83A8100002A9F00000000000DFDFB, messageQueue=MessageQueue [topic=TopicTest1, brokerName=localhost.localdomain, queueId=2], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F03725977F0004, offsetMsgId=C0A83A8100002A9F00000000000DFEAE, messageQueue=MessageQueue [topic=TopicTest1, brokerName=localhost.localdomain, queueId=3], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597830005, offsetMsgId=C0A83A8100002A9F00000000000DFF61, messageQueue=MessageQueue [topic=TopicTest2, brokerName=localhost.localdomain, queueId=1], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597880006, offsetMsgId=C0A83A8100002A9F00000000000E0016, messageQueue=MessageQueue [topic=TopicTest2, brokerName=localhost.localdomain, queueId=2], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F03725978B0007, offsetMsgId=C0A83A8100002A9F00000000000E00CB, messageQueue=MessageQueue [topic=TopicTest2, brokerName=localhost.localdomain, queueId=3], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597900008, offsetMsgId=C0A83A8100002A9F00000000000E0180, messageQueue=MessageQueue [topic=TopicTest2, brokerName=localhost.localdomain, queueId=0], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=C0A8011400E827C170F0372597930009, offsetMsgId=C0A83A8100002A9F00000000000E0235, messageQueue=MessageQueue [topic=TopicTest2, brokerName=localhost.localdomain, queueId=1], queueOffset=5]
17:00:10.542 [NettyClientSelector_1] INFO  RocketmqRemoting - closeChannel: close the connection to remote address[192.168.58.129:9876] result: true
17:00:10.542 [NettyClientSelector_1] INFO  RocketmqRemoting - closeChannel: close the connection to remote address[192.168.58.129:10911] result: true
17:00:10.644 [NettyClientSelector_1] INFO  RocketmqRemoting - closeChannel: close the connection to remote address[192.168.58.129:10909] result: true

Process finished with exit code 0

• 运行 Consumer 输出信息如下，成功的接收到了 生产者发送的消息，因为订阅的是 TopicTest2 的消息，所以输出的都是它的

17:00:06.324 [main] DEBUG i.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
Consumer Started.
ConsumeMessageThread_1 Receive New Messages: 1
ConsumeMessageThread_1 Receive New Messages: [MessageExt [queueId=2, storeSize=181, queueOffset=4, sysFlag=0, bornTimestamp=1533978010504, bornHost=/192.168.58.1:10966, storeTimestamp=1534006370763, storeHost=/192.168.58.129:10911, msgId=C0A83A8100002A9F00000000000E0016, commitLogOffset=917526, bodyCRC=195062740, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest2', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=5, CONSUME_START_TIME=1533978010529, UNIQ_KEY=C0A8011400E827C170F0372597880006, WAIT=true, TAGS=TagB}, body=[-26, -120, -111, -25, -120, -79, -28, -67, -96, 44, -28, -72, -83, -27, -101, -67, 32, 54], transactionId='null'}]] 
ConsumeMessageThread_4 Receive New Messages: 1
ConsumeMessageThread_4 Receive New Messages: [MessageExt [queueId=3, storeSize=181, queueOffset=3, sysFlag=0, bornTimestamp=1533978010507, bornHost=/192.168.58.1:10966, storeTimestamp=1534006370767, storeHost=/192.168.58.129:10911, msgId=C0A83A8100002A9F00000000000E00CB, commitLogOffset=917707, bodyCRC=2091342658, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest2', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=4, CONSUME_START_TIME=1533978010531, UNIQ_KEY=C0A8011400E827C170F03725978B0007, WAIT=true, TAGS=TagB}, body=[-26, -120, -111, -25, -120, -79, -28, -67, -96, 44, -28, -72, -83, -27, -101, -67, 32, 55], transactionId='null'}]] 
----------:我爱你,中国 7
ConsumeMessageThread_3 Receive New Messages: 1
ConsumeMessageThread_3 Receive New Messages: [MessageExt [queueId=0, storeSize=181, queueOffset=4, sysFlag=0, bornTimestamp=1533978010512, bornHost=/192.168.58.1:10966, storeTimestamp=1534006370770, storeHost=/192.168.58.129:10911, msgId=C0A83A8100002A9F00000000000E0180, commitLogOffset=917888, bodyCRC=1813530323, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest2', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=5, CONSUME_START_TIME=1533978010531, UNIQ_KEY=C0A8011400E827C170F0372597900008, WAIT=true, TAGS=TagB}, body=[-26, -120, -111, -25, -120, -79, -28, -67, -96, 44, -28, -72, -83, -27, -101, -67, 32, 56], transactionId='null'}]] 
----------:我爱你,中国 8
----------:我爱你,中国 6
ConsumeMessageThread_2 Receive New Messages: 1
ConsumeMessageThread_2 Receive New Messages: [MessageExt [queueId=1, storeSize=181, queueOffset=4, sysFlag=0, bornTimestamp=1533978010499, bornHost=/192.168.58.1:10966, storeTimestamp=1534006370757, storeHost=/192.168.58.129:10911, msgId=C0A83A8100002A9F00000000000DFF61, commitLogOffset=917345, bodyCRC=313080430, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest2', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=5, CONSUME_START_TIME=1533978010531, UNIQ_KEY=C0A8011400E827C170F0372597830005, WAIT=true, TAGS=TagB}, body=[-26, -120, -111, -25, -120, -79, -28, -67, -96, 44, -28, -72, -83, -27, -101, -67, 32, 53], transactionId='null'}]] 
----------:我爱你,中国 5
ConsumeMessageThread_5 Receive New Messages: 1
ConsumeMessageThread_5 Receive New Messages: [MessageExt [queueId=1, storeSize=181, queueOffset=5, sysFlag=0, bornTimestamp=1533978010515, bornHost=/192.168.58.1:10966, storeTimestamp=1534006370773, storeHost=/192.168.58.129:10911, msgId=C0A83A8100002A9F00000000000E0235, commitLogOffset=918069, bodyCRC=455046725, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message{topic='TopicTest2', flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=6, CONSUME_START_TIME=1533978010537, UNIQ_KEY=C0A8011400E827C170F0372597930009, WAIT=true, TAGS=TagB}, body=[-26, -120, -111, -25, -120, -79, -28, -67, -96, 44, -28, -72, -83, -27, -101, -67, 32, 57], transactionId='null'}]] 
----------:我爱你,中国 9

扫描二维码关注公众号，回复： 2750054 查看本文章