RabbitMQ Technology - Primary

1. Getting to know RabbitMQ first

1. Basic structure

Some roles in RabbitMQ:

• publisher: producer
• consumer: consumer
• exchange: switch, responsible for message routing
• queue: queue, store messages
• virtualHost: virtual host, which isolates the exchange, queue, and message isolation of different tenants

2. Message model

RabbitMQ officially provides 5 different Demo examples, corresponding to different message models:

3. Project creation

Create the project structure as follows:

Consists of three parts:

• mq-demo: parent project, manage project dependencies
• publisher: the sender of the message
• consumer: the consumer of the message

4. Getting Started Case

Model diagram of the simple queue mode:

The official HelloWorld is implemented based on the most basic message queue model, including only three roles:

• publisher: message publisher, send the message to the queue queue
• queue: message queue, responsible for accepting and caching messages
• consumer: Subscribe to the queue and process messages in the queue

4.1. publisher implementation

Ideas:

• establish connection
• Create Channels
• declare queue
• Send a message
• Close connections and channels

Code:

package cn.zqd.mq.helloworld;

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;
import org.junit.Test;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class PublisherTest {
    
    
    @Test
    public void testSendMessage() throws IOException, TimeoutException {
    
    
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数，分别是：主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.150.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("zqd");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.发送消息
        String message = "hello, rabbitmq!";
        channel.basicPublish("", queueName, null, message.getBytes());
        System.out.println("发送消息成功：【" + message + "】");

        // 5.关闭通道和连接
        channel.close();
        connection.close();

    }
}

4.2.consumer implementation

Code ideas:

• establish connection
• Create Channels
• declare queue
• subscribe news

Code:

package cn.zqd.mq.helloworld;

import com.rabbitmq.client.*;

import java.io.IOException;
import java.util.concurrent.TimeoutException;

public class ConsumerTest {
    
    

    public static void main(String[] args) throws IOException, TimeoutException {
    
    
        // 1.建立连接
        ConnectionFactory factory = new ConnectionFactory();
        // 1.1.设置连接参数，分别是：主机名、端口号、vhost、用户名、密码
        factory.setHost("192.168.150.101");
        factory.setPort(5672);
        factory.setVirtualHost("/");
        factory.setUsername("zqd");
        factory.setPassword("123321");
        // 1.2.建立连接
        Connection connection = factory.newConnection();

        // 2.创建通道Channel
        Channel channel = connection.createChannel();

        // 3.创建队列
        String queueName = "simple.queue";
        channel.queueDeclare(queueName, false, false, false, null);

        // 4.订阅消息
        channel.basicConsume(queueName, true, new DefaultConsumer(channel){
    
    
            @Override
            public void handleDelivery(String consumerTag, Envelope envelope,
                                       AMQP.BasicProperties properties, byte[] body) throws IOException {
    
    
                // 5.处理消息
                String message = new String(body);
                System.out.println("接收到消息：【" + message + "】");
            }
        });
        System.out.println("等待接收消息。。。。");
    }
}

4.3. Summary

The message sending process of the basic message queue:

1. create connection

2. Create channels

3. Use channel to declare queue

4. Use the channel to send messages to the queue

The message receiving process of the basic message queue:

1. create connection

2. Create channels

3. Use channel to declare queue

4. Define consumer's consumption behavior handleDelivery()

5. Use channels to bind consumers to queues

2. Spring AMQP

SpringAMQP is a set of templates based on RabbitMQ package, and it also uses SpringBoot to realize automatic assembly, which is very convenient to use.

Official address of SpringAmqp: https://spring.io/projects/spring-amqp

Spring AMQP provides three functions:

• Automatic declaration of queues, exchanges and their bindings
• Annotation-based listener mode to receive messages asynchronously
• Encapsulates the RabbitTemplate tool for sending messages

1.Basic Queue simple queue model

Introduce dependencies in the parent project mq-demo

<!--AMQP依赖，包含RabbitMQ-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

1.1. Message sending

First configure the MQ address, and add the configuration to the application.yml of the publisher service:

spring:
  rabbitmq:
    host: 192.168.150.101 # 主机名
    port: 5672 # 端口
    virtual-host: / # 虚拟主机
    username: zqd# 用户名
    password: 123321 # 密码

Then write the test class SpringAmqpTest in the publisher service, and use RabbitTemplate to send messages:

package cn.zqd.mq.spring;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest
public class SpringAmqpTest {
    
    

    @Autowired
    private RabbitTemplate rabbitTemplate;

    @Test
    public void testSimpleQueue() {
    
    
        // 队列名称
        String queueName = "simple.queue";
        // 消息
        String message = "hello, spring amqp!";
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message);
    }
}

1.2. Message reception

First configure the MQ address, and add the configuration to the application.yml of the consumer service:

spring:
  rabbitmq:
    host: 192.168.150.101 # 主机名
    port: 5672 # 端口
    virtual-host: / # 虚拟主机
    username: zqd# 用户名
    password: 123321 # 密码

cn.zqd.mq.listenerThen create a new class SpringRabbitListener in the consumer service package, the code is as follows:

package cn.zqd.mq.listener;

import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

@Component
public class SpringRabbitListener {
    
    

    @RabbitListener(queues = "simple.queue")
    public void listenSimpleQueueMessage(String msg) throws InterruptedException {
    
    
        System.out.println("spring 消费者接收到消息：【" + msg + "】");
    }
}

1.3. Testing

Start the consumer service, then run the test code in the publisher service, and send MQ messages

2.WorkQueue

Work queues, also known as (Task queues), task model. Simply put, it is to let multiple consumers bind to a queue and consume messages in the queue together .

When message processing is time-consuming, the speed of message production may be far greater than the speed of message consumption. If things go on like this, more and more messages will pile up and cannot be processed in time.

At this time, the work model can be used, and multiple consumers can jointly process message processing, and the speed can be greatly improved.

2.1. Message sending

This time we send in a loop to simulate the accumulation of a large number of messages.

Add a test method to the SpringAmqpTest class in the publisher service:

/**
     * workQueue
     * 向队列中不停发送消息，模拟消息堆积。
     */
@Test
public void testWorkQueue() throws InterruptedException {
    
    
    // 队列名称
    String queueName = "simple.queue";
    // 消息
    String message = "hello, message_";
    for (int i = 0; i < 50; i++) {
    
    
        // 发送消息
        rabbitTemplate.convertAndSend(queueName, message + i);
        Thread.sleep(20);
    }
}

2.2. Message reception

To simulate multiple consumers binding to the same queue, we add two new methods to the SpringRabbitListener of the consumer service:

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue1(String msg) throws InterruptedException {
    
    
    System.out.println("消费者1接收到消息：【" + msg + "】" + LocalTime.now());
    Thread.sleep(20);
}

@RabbitListener(queues = "simple.queue")
public void listenWorkQueue2(String msg) throws InterruptedException {
    
    
    System.err.println("消费者2........接收到消息：【" + msg + "】" + LocalTime.now());
    Thread.sleep(200);
}

Note that the consumer sleeps for 1000 seconds, and the simulation task takes time.

2.3. Testing

After starting the ConsumerApplication, execute the sending test method testWorkQueue just written in the publisher service.

It can be seen that consumer 1 quickly completed its 25 messages. Consumer 2 is slowly processing its own 25 messages.

That is to say, the message is evenly distributed to each consumer, without taking into account the processing power of the consumer. This is obviously problematic.

2.4. Those who can do more work

There is a simple configuration in spring that can solve this problem. We modify the application.yml file of the consumer service and add the configuration:

spring:
  rabbitmq:
    listener:
      simple:
        prefetch: 1 # 每次只能获取一条消息，处理完成才能获取下一个消息

2.5. Summary

Use of the Work model:

• Multiple consumers are bound to a queue, and the same message will only be processed by one consumer
• Control the number of messages prefetched by consumers by setting prefetch

3. Publish/Subscribe

The publish-subscribe model is shown in the figure:

As you can see, in the subscription model, there is an additional exchange role, and the process has changed slightly:

• Publisher: The producer, that is, the program to send the message, but no longer sent to the queue, but to X (exchange)
• Exchange: switch, X in the figure. On the one hand, receive messages sent by producers. On the other hand, knowing how to process the message, such as delivering it to a particular queue, delivering it to all queues, or discarding the message. How it works depends on the type of Exchange. Exchange has the following 3 types:
  • Fanout: Broadcast, hand over the message to all queues bound to the exchange
  • Direct: directional, deliver the message to the queue that matches the specified routing key
  • Topic: wildcard, send the message to the queue that matches the routing pattern (routing pattern)
• Consumer: The consumer, as before, subscribes to the queue, no change
• Queue: The message queue is the same as before, receiving messages and buffering messages.

Exchange (exchange) is only responsible for forwarding messages, and does not have the ability to store messages , so if there is no queue bound to Exchange, or there is no queue that meets the routing rules, then the message will be lost!

4.Fanout

Fanout, the English translation is fan out, I think it is more appropriate to call it broadcast in MQ.

In broadcast mode, the message sending process is as follows:

• 1) There can be multiple queues
• 2) Each queue must be bound to Exchange (exchange)
• 3) The message sent by the producer can only be sent to the switch, and the switch decides which queue to send to, and the producer cannot decide
• 4) The switch sends the message to all bound queues
• 5) Consumers who subscribe to the queue can get the message

The plan is as follows:

• Create a switch zqd.fanout, the type is Fanout
• Create two queues fanout.queue1 and fanout.queue2, bound to the switch zqd.fanout

4.1. Declare queues and exchanges

Spring provides an interface Exchange to represent all different types of exchanges:

Create a class in consumer to declare queues and switches:

package cn.zqd.mq.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.FanoutExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class FanoutConfig {
    
    
    /**
     * 声明交换机
     * @return Fanout类型交换机
     */
    @Bean
    public FanoutExchange fanoutExchange(){
    
    
        return new FanoutExchange("zqd.fanout");
    }

    /**
     * 第1个队列
     */
    @Bean
    public Queue fanoutQueue1(){
    
    
        return new Queue("fanout.queue1");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue1(Queue fanoutQueue1, FanoutExchange fanoutExchange){
    
    
        return BindingBuilder.bind(fanoutQueue1).to(fanoutExchange);
    }

    /**
     * 第2个队列
     */
    @Bean
    public Queue fanoutQueue2(){
    
    
        return new Queue("fanout.queue2");
    }

    /**
     * 绑定队列和交换机
     */
    @Bean
    public Binding bindingQueue2(Queue fanoutQueue2, FanoutExchange fanoutExchange){
    
    
        return BindingBuilder.bind(fanoutQueue2).to(fanoutExchange);
    }
}

4.2. Message sending

Add a test method to the SpringAmqpTest class of the publisher service:

@Test
public void testFanoutExchange() {
    
    
    // 队列名称
    String exchangeName = "zqd.fanout";
    // 消息
    String message = "hello, everyone!";
    rabbitTemplate.convertAndSend(exchangeName, "", message);
}

4.3. Message reception

Add two methods to the SpringRabbitListener of the consumer service as a consumer:

@RabbitListener(queues = "fanout.queue1")
public void listenFanoutQueue1(String msg) {
    
    
    System.out.println("消费者1接收到Fanout消息：【" + msg + "】");
}

@RabbitListener(queues = "fanout.queue2")
public void listenFanoutQueue2(String msg) {
    
    
    System.out.println("消费者2接收到Fanout消息：【" + msg + "】");
}

4.4. Summary

What is the role of the switch?

• Receive messages sent by the publisher
• Route the message to the queue bound to it according to the rules
• Messages cannot be cached, routing fails, messages are lost
• FanoutExchange will route messages to each bound queue

What are the beans that declare queues, switches, and binding relationships?

• Queue
• FanoutExchange
• Binding

5.Direct

In Fanout mode, a message will be consumed by all subscribed queues. However, in some scenarios, we want different messages to be consumed by different queues. At this time, the Direct type of Exchange will be used.

Under the Direct model:

• The binding between the queue and the switch cannot be arbitrary, but a RoutingKey(routing key) must be specified
• The sender of the message must also specify the message ID when sending the message to Exchange RoutingKey.
• Exchange no longer delivers messages to each bound queue, but Routing Keyjudges based on the message. Only when the queue is completelyRoutingkey consistent with the message will the message be received.Routing key

The case requirements are as follows :

1. Use @RabbitListener to declare Exchange, Queue, RoutingKey

2. In the consumer service, write two consumer methods to listen to direct.queue1 and direct.queue2 respectively

3. Write a test method in the publisher and send a message to zqd.direct

5.1. Declare queues and switches based on annotations

It is cumbersome to declare queues and switches based on @Bean. Spring also provides annotation-based declarations.

Add two consumers to the consumer's SpringRabbitListener, and declare queues and switches based on annotations:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue1"),
    exchange = @Exchange(name = "zqd.direct", type = ExchangeTypes.DIRECT),
    key = {
    
    "red", "blue"}
))
public void listenDirectQueue1(String msg){
    
    
    System.out.println("消费者接收到direct.queue1的消息：【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "direct.queue2"),
    exchange = @Exchange(name = "zqd.direct", type = ExchangeTypes.DIRECT),
    key = {
    
    "red", "yellow"}
))
public void listenDirectQueue2(String msg){
    
    
    System.out.println("消费者接收到direct.queue2的消息：【" + msg + "】");
}

5.2. Message sending

Add a test method to the SpringAmqpTest class of the publisher service:

@Test
public void testSendDirectExchange() {
    
    
    // 交换机名称
    String exchangeName = "zqd.direct";
    // 消息
    String message = "红色警报！日本乱排核废水，导致海洋生物变异，惊现哥斯拉！";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "red", message);
}

5.3. Summary

Describe the difference between a Direct switch and a Fanout switch?

• Fanout exchange routes messages to each queue bound to it
• The Direct switch determines which queue to route to according to the RoutingKey
• Similar to Fanout functionality if multiple queues have the same RoutingKey

What are the common annotations for declaring queues and exchanges based on the @RabbitListener annotation?

• @Queue
• @Exchange

6.Topic

6.1. Description

TopicExchangeCompared with the other types Direct, messages can RoutingKeybe routed to different queues according to the type. It's just that Topicthe type Exchangeallows the queue Routing keyto use wildcards when binding!

RoutingkeyGenerally, it consists of one or more words, and multiple words are separated by ".", for example:item.insert

Wildcard rules:

#: match one or more words

*: Match exactly 1 word

Example:

item.#: can match item.spu.insertoritem.spu

item.*: can only matchitem.spu

​

Graphic:

explain:

• Queue1: is bound china.#, so all that china.start routing keywill be matched. Including china.news and china.weather
• Queue2: The binding is #.news, so all .newsending routing keywill be matched. Including china.news and japan.news

Case requirements:

The implementation idea is as follows:

1. And use @RabbitListener to declare Exchange, Queue, RoutingKey

2. In the consumer service, write two consumer methods to listen to topic.queue1 and topic.queue2 respectively

3. Write a test method in the publisher and send a message to zqd.topic

6.2. Message sending

Add a test method to the SpringAmqpTest class of the publisher service:

/**
     * topicExchange
     */
@Test
public void testSendTopicExchange() {
    
    
    // 交换机名称
    String exchangeName = "zqd.topic";
    // 消息
    String message = "喜报！孙悟空大战哥斯拉，胜!";
    // 发送消息
    rabbitTemplate.convertAndSend(exchangeName, "china.news", message);
}

6.3. Message reception

Add the method in the SpringRabbitListener of the consumer service:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue1"),
    exchange = @Exchange(name = "zqd.topic", type = ExchangeTypes.TOPIC),
    key = "china.#"
))
public void listenTopicQueue1(String msg){
    
    
    System.out.println("消费者接收到topic.queue1的消息：【" + msg + "】");
}

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "topic.queue2"),
    exchange = @Exchange(name = "zqd.topic", type = ExchangeTypes.TOPIC),
    key = "#.news"
))
public void listenTopicQueue2(String msg){
    
    
    System.out.println("消费者接收到topic.queue2的消息：【" + msg + "】");
}

6.4. Summary

Describe the difference between a Direct switch and a Topic switch?

• The message RoutingKey received by the Topic switch must be multiple words, **.**separated
• The bindingKey when the Topic switch is bound to the queue can specify a wildcard
• #: represents 0 or more words
• *: represents 1 word

7. Message Converter

As I said before, Spring will serialize the message you send into bytes and send it to MQ, and when receiving the message, it will also deserialize the bytes into Java objects.

However, by default, the serialization method used by Spring is JDK serialization. As we all know, JDK serialization has the following problems:

• Data size is too large
• has a security hole
• poor readability

7.1. Testing the default converter

We modify the code for message sending to send a Map object:

@Test
public void testSendMap() throws InterruptedException {
    
    
    // 准备消息
    Map<String,Object> msg = new HashMap<>();
    msg.put("name", "Jack");
    msg.put("age", 21);
    // 发送消息
    rabbitTemplate.convertAndSend("simple.queue","", msg);
}

Stop the consumer service

Check the console after sending the message:

7.2. Configure the JSON converter

Obviously, the JDK serialization method is not suitable. We want the message body to be smaller and more readable, so we can use the JSON method for serialization and deserialization.

Introduce dependencies in both publisher and consumer services:

<dependency>
    <groupId>com.fasterxml.jackson.dataformat</groupId>
    <artifactId>jackson-dataformat-xml</artifactId>
    <version>2.9.10</version>
</dependency>

Configure message converters.

Just add a Bean to the startup class:

@Bean
public MessageConverter jsonMessageConverter(){
    
    
    return new Jackson2JsonMessageConverter();
}