rocketmq Learning (II) rocketmq cluster deployment and graphical console installation

1.rocketmq graphical console installation

　　Although rocketmq provided using the command line management topics, consumer groups and broker functions configured for the user, but for unskilled non operation and maintenance personnel, the command line management interface is difficult to use. To this end, we can use the graphical management interface to simplify management operations.

　　rocketmq official recommended graphical console is still in an immature stage of incubation. Warehouse addresses ( https://github.com/apache/rocketmq-externals ), which contains rocketmq related expansion of projects falling within the incubation period. After downloading the source code, find rocketmq-console folder, which is the official recommended rocketmq graphical console project, based on springboot and angularJS.

　　Open application.properties, to see some important configuration parameters, such as port, nameServer address, login access control and so on. For startup setting parameters, you can choose to modify the configuration file directly; may also be specified by the command line when starting the project.

　　

　　For the deployment of projects, execute maven package command (mvn clean package), generate jar package.

　　Then execute java -jar rocketmq-console-ng-1.0.1.jar --rocketmq.config.namesrvAddr = localhost: 9876 (nameServer address) --server.port = 8080 (priming port).

　　Launched through a browser to access the project ip / port, you can see the following Management Interface (upper right corner you can switch in English).

 　　So far, rocketmq graphical console installed successfully.

2.rocketmq cluster deployment

　　rocketmq convenient stand-alone deployment is simple, but there is a single point of failure. By deploying nameServer and broker clusters can achieve high availability rocketmq server-side.

　　Here rocketmq cluster deployed to build a double-double main high availability cluster from rocketmq on two machines as an example, the two machines (linux environment) of the IP addresses are 192.168.32.130, and 192.168.32.131.

2.1 nameServer cluster deployment

　　Since the communication between the cluster nodes and do not nameServer, so no additional configuration.

　　Performed in the root path of the two machines are mounted rocketmq "SH bin / mqnamesrv" , each of a start nameServer, using its default port 9876.

　　Now, 192.168.32.130: and on the 9876 192.168.32.131:9876 are each running a nameServer service.

2.2 broker cluster deployment

　　As the core rocketmq broker, it is vital that the operational stability. Double dual master from the aforementioned ( 2-2-Master-Slave ) actually refer to the operating mode of the cluster broker, the broker from the broker as a backup master, the master and is responsible for keeping the data broker, not writable read.

　　rocketmq to distinguish the different roles of broker by giving the broker name. We present two roles of broker named broker-a and broker-b.

　　192.168.32.130 deployed from the primary broker-a broker-b, and from the broker-b 192.168.32.131 deployment of the primary broker-a, respectively from the same role main deployed on different machines. Thus, even if any machine hang up, due to the presence from the broker, broker-a and broker-b can still provide services.

　　When you start broker, to broker a series of parameters can be set by specifying configuration files. Different roles broker, the main profile different from the parameters between the content broker.

broker configuration file parameters introduced:

　　brokerName: Broker name, broker name mainly from the mutual consistency.

　　namesrvAddr: nameserver address associated with a plurality of ";" separated.

　　listenPort: broker listen port, the same machine can not deploy multiple broker listen port can not be the same, to avoid conflict

　　storePathRootDir: the root broker storing data

　　brokerClusterName: Broker cluster name, to the same cluster master can recognize each other

　　brokerId: 0 representative of master, is greater than 0 representing different slave-broker

　　deleteWhen: delete obsolete messages of the time, 04 representatives 4:00

　　fileReservedTime: When you save the data off the disk file length, in hours

　　brokerRole: brokerRole There are three types, SYNC_MASTER , ASYNC_MASTER and SLAVE , SYNC_M and ASYNC_M represent the main broker, the main difference is that the data from the different between a synchronous manner. Representative from the SYNC master data synchronization is complete, it returns to the client a result message has been sent successfully; ASYNC contrary represents the master database sends back a successful result message immediately after receiving the message.

　　We can see, more efficient ASYNC_MASTER, but when MASTER fails, the message loss problems may arise. It requires the user to trade-off between efficiency and reliability.

　　flushDiskType: flushDiskType There are two types, SYNC_FLUSH (synchronous brush disc) and ASYNC_FLUSH (asynchronous brush plate) , specifies the broker upon receiving the message, the return message data off the disk and transmits the result of processing strategies. When you select synchronous brush set, only when the message data is actually written to disk persistence, it returns the message sent successfully. When the brush plate select asynchronous, message data written to the local virtual memory mapping, direct return.

　　broker local disk off the main selection strategy and policy synchronization from a similar, need to be a trade-off between efficiency and reliability, consistency.

　　Compare the recommended one configuration is selected SYNC_MASTER master-slave synchronization strategy and the local order placing policy choice SYNC_FLUSH. From a reliability point of view, as long as the master while the broker does not hang (avoiding the single point of failure), the message will not be lost; From an efficiency standpoint, since the main plate from falling broker are asynchronous, there efficiency certain degree of protection, is an excellent compromise.

broker configuration file details:

broker-a primary broker configuration file (broker-a.properties):

brokerName=broker-a
namesrvAddr=192.168.32.130:9876;192.168.32.131:9876
listenPort=10911
storePathRootDir=/root/rocketmq/data/store/store-a
brokerClusterName=DefaultCluster
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH

broker-b primary broker configuration file (broker-b.properties):

brokerName=broker-b
namesrvAddr=192.168.32.130:9876;192.168.32.131:9876
listenPort=10911
storePathRootDir=/root/rocketmq/data/store/store-b
brokerClusterName=DefaultCluster
brokerId=0
deleteWhen=04
fileReservedTime=48
brokerRole=SYNC_MASTER
flushDiskType=ASYNC_FLUSH

broker-a profile (broker-as.properties) from the broker:

brokerName=broker-a
namesrvAddr=192.168.32.130:9876;192.168.32.131:9876
listenPort=11011
storePathRootDir=/root/rocketmq/data/store/store-a
brokerClusterName=DefaultCluster
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

broker-b profiles (broker-bs.properties) from the broker:

brokerName=broker-b
namesrvAddr=192.168.32.130:9876;192.168.32.131:9876
listenPort=11011
storePathRootDir=/root/rocketmq/data/store/store-b
brokerClusterName=DefaultCluster
brokerId=1
deleteWhen=04
fileReservedTime=48
brokerRole=SLAVE
flushDiskType=ASYNC_FLUSH

 　　Sequentially executes the command corresponding machine, in turn starts Broker (Broker to start the main, after starting the broker).

　　192.168.32.130 execute: sh bin / mqbroker -c [Profile Path eg: rocketmq / data / conf / broker-a.properties]

　　192.168.32.131 execute: sh bin / mqbroker -c [Profile Path eg: rocketmq / data / conf / broker-b.properties]

　　192.168.32.131 execute: sh bin / mqbroker -c [Profile Path eg: rocketmq / data / conf / broker-as.properties]

　　192.168.32.130 execute: sh bin / mqbroker -c [Profile Path eg: rocketmq / data / conf / broker-bs.properties]

　　At this time, rocketmq double dual master cluster from the broker is now fully set. Enabling a graphical console, specify a command line parameter namesrvAddr = 192.168.32.130: 9876; 192.168.32.131: 9876, see the following information:

3. java messaging client

　　Earlier we tested rocketmq send and receive messages via the command line. However, in actual use, or you need sdk client to send and receive messages. Here, we use the java sdk rocketmq provided to carry out experiments rocketmq of messaging.

　　Create a topic for testing through the graphical console, a subject name of "TopicTest" (taken lightly).

　　Then start a java project, adding rely rocketmq-client of.

maven coordinates:

<!-- 原生rocketmq client -->
<dependency>
     <groupId>org.apache.rocketmq</groupId>
     <artifactId>rocketmq-client</artifactId>
     <version>4.4.0</version>
</dependency>

Sample Code Manufacturer producer:

public class Producer {
    public static void main(String[] args) throws Exception {
        final DefaultMQProducer producer = new DefaultMQProducer("test_producer_group");
        // 设置nameServer地址
        producer.setNamesrvAddr("192.168.32.130:9876;192.168.32.131:9876");
        producer.start();
        for (int i = 0; i < 1000; i++) {
            try {
                // 构造消息对象，topic=TopicTest，tag=TagA
                Message msg = new Message("TopicTest", "TagA" ,
                    ("Hello RocketMQ TopicTest" + i).getBytes(RemotingHelper.DEFAULT_CHARSET));
                // 发送消息
                SendResult sendResult = producer.send(msg);
                System.out.printf("%s%n", sendResult);
            } catch (Exception e) {
                e.printStackTrace();
                Thread.sleep(1000);
            }
        }
        producer.shutdown();
    }
}

Consumers consumer sample code:

public  class Consumer {
     public  static  void main (String [] args) throws Exception { 
        DefaultMQPushConsumer Consumer = new new DefaultMQPushConsumer ( "test_consumer_group" );
         // set nameServer address 
        consumer.setNamesrvAddr ( "192.168.32.130:9876;192.168.32.131:9876" ); 
        consumer.setConsumeFromWhere (ConsumeFromWhere.CONSUME_FROM_FIRST_OFFSET); 

        // subscribe to a topic tag = "*" subscription on behalf of all subtopics under TopicTest messages theme 
        Consumer.subscribe ( "TopicTest", "*" );
         // Register the message listener callback function
        consumer.registerMessageListener((MessageListenerConcurrently)(msgs, context) -> {
            for(MessageExt messageExt : msgs){
                String strMsg = new String(messageExt.getBody());
                System.out.printf("%s Receive New Messages: %s %n", Thread.currentThread().getName(), strMsg);
            }
            return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
        });
        consumer.start();
        System.out.printf("Consumer Started.%n");
    }
}

　　Producers and consumers are simply main way to start, start producer sends a message, and then start consumer acceptance message, you will see a message to accept the log on the console. Can take the initiative to try to close a master-broker, look at the messaging broker cluster is normal.

　　Thus, by using the java client rocketmq tests come to an end.

to sum up

　　This blog describes the cluster deployment, graphical interface installation rocketmq and interact with rocketmq how to use the java client. rocketmq There are many easy to use, powerful features, combined with the subsequent blog rocketmq source to introduce them.

　　To read and understand the source code, you can be able to look a little deeper in solving problems and farther. Rocketmq by reading the source code, in addition to better grasp rocketmq, but also to learn from the source code to a number of architectural design and programming knowledge related to practice.

　　If understood not in place, the exhibitions.