Spring boot + RabbitMQ delay queue combat

1. Background

As the name implies, the delayed queue means that the messages placed in the queue do not need to be consumed immediately, but are taken out after waiting for a period of time.
So, why do you need to delay consumption? Let's look at the following scenario:

Order business: In e-commerce/ordering, the order is automatically cancelled if there is no payment within 30 minutes after the order is placed.
SMS notification: Send a SMS notification to the user 60s after the order is successfully placed.
Failure retry: After a business operation fails, a failure retry is performed at a certain interval.

Traditional order processing:

Take timed tasks to train database orders and process them in batches. Its disadvantages are also obvious; there will be great requirements for server and database performance, and it will be very powerless when processing a large number of orders, and the real-time performance is not particularly good.

Of course, the traditional method can be optimized, that is, when the order is stored, the expiration time of the order is inserted into the database. When setting a timed task to query the database, you only need to query the expired order, and then do other business operations.

However, the real-time performance is not very good when it is executed by timed tasks.

Second, the realization plan

Option 1: Realize through dead letter queue

Reference: https://www.cnblogs.com/xmf3628/p/12097101.html

Solution 2: Implement rabbitmq-delayed-message-exchange through delayed routing plug-in

Reference: https://www.cnblogs.com/wintercloud/p/10877399.html

This demo is implemented through plugins

Three, plug-in installation

Download link:
https://github.com/rabbitmq/rabbitmq-delayed-message-exchange/releases/tag/v3.8.0

Some plug-ins of RabbitMQ are not integrated in the initial installation. They need to be installed additionally. The suffix of these files is .ez. You need to copy the .ez file to the installed plug-in directory during installation. The following are the directory paths of plugins installed by default in different systems:

//查看已安装的插件
rabbitmq-plugins list
//启用插件
rabbitmq-plugins enable rabbitmq_delayed_message_exchange
//重启服务
service rabbitmq-server restart
//再次查看，插件是否生效
rabbitmq-plugins list

The plug-in is successfully installed and successfully activated.

Four, mechanism explanation

After installing the plug-in, a new Exchange type will be generated x-delayed-message. This type of message supports a delayed delivery mechanism. After receiving the message, the message is not delivered to the target queue immediately, but stored in mnesiaa table (a distributed data system) to detect the message delay Time, such as when the delivery time is reached, it will be x-delayed-typedelivered to the target queue through the switch type marked by the type.

Five, code combat

1. Dependence

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-amqp</artifactId>
    </dependency>

    <dependency>
        <groupId>com.alibaba</groupId>
        <artifactId>fastjson</artifactId>
        <version>1.2.44</version>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-devtools</artifactId>
        <scope>runtime</scope>
        <optional>true</optional>
    </dependency>
    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <optional>true</optional>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

2. Queue connection information

spring.rabbitmq.host=localhost
spring.rabbitmq.port=5672
spring.rabbitmq.username=admin
spring.rabbitmq.password=123456
spring.rabbitmq.virtual-host=/

3. Producer code

Declare the delay queue and the switch of the delay queue, and establish the binding relationship

@Configuration
public class TestDelayQueueConfig {

    @Bean
    public CustomExchange delayExchange() {
        Map<String, Object> args = new HashMap<String, Object>();
        args.put("x-delayed-type", "direct");
        return new CustomExchange(ExchangeEnum.DELAY_EXCHANGE.getValue(), "x-delayed-message", true, false, args);
    }

    /**
     * 延迟消息队列
     * @return
     */
    @Bean
    public Queue delayQueue() {
        return new Queue(QueueEnum.TEST_DELAY.getName(), true);
    }

    @Bean
    public Binding deplyBinding() {
        return BindingBuilder.bind(delayQueue()).to(delayExchange()).with(QueueEnum.TEST_DELAY.getRoutingKey()).noargs();
    }
}

@Getter
public enum ExchangeEnum {
    DELAY_EXCHANGE("test.deply.exchange");

    private String value;

    ExchangeEnum(String value) {
        this.value = value;
    }
}

@Getter
public enum QueueEnum {
    /**
     * delay
     */
    TEST_DELAY("test.delay.queue", "delay");
    /**
     * 队列名称
     */
    private String name;
    /**
     * 队列路由键
     */
    private String routingKey;

    QueueEnum(String name, String routingKey) {
        this.name = name;
        this.routingKey = routingKey;
    }
}

Delayed queue producer service:

@Component
@Slf4j
public class DeplyProducer {
    @Autowired
    private RabbitTemplate rabbitTemplate;
    
    public void send(String msg, int delayTime) {
        log.info("msg= " + msg + ".delayTime" + delayTime);
        MessageProperties messageProperties = new MessageProperties();
        messageProperties.setDelay(delayTime);
        Message message = new Message(msg.getBytes(), messageProperties);
        rabbitTemplate.send(ExchangeEnum.DELAY_EXCHANGE.getValue(), QueueEnum.TEST_DELAY.getRoutingKey(), message);
    }
}

unit test:

@Test
public void sendDeplyMsgTest() {
    SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
    String currentTime = sdf.format(new Date());
    log.info("发送测试消息的时间：" +currentTime );
    deplyProducer.send(currentTime + "发送一个测试消息,延迟10秒", 10000);//10秒
    deplyProducer.send(currentTime + "发送一个测试消息，延迟20秒", 20000);//2秒
    deplyProducer.send(currentTime + "发送一个测试消息，延迟30秒", 30000);//1秒
}

consumer:

@Component
@RabbitListener(queues = "test.delay.queue")
@Slf4j
public class DeplyConsumer {
    @RabbitHandler
    public void onMessage(byte[] message,
                          @Headers Map<String, Object> headers,
                          Channel channel) {
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        //log.info("收到延时消息时间：" + sdf.format(new Date()) + " Delay sent.");
        log.info(sdf.format(new Date())+"接收到延时消息:" + new String(message));
    }
}

Execution test result: When

executing the message sending code, the message will not be pushed to the corresponding queue immediately.
The message will be pushed into the queue only when the corresponding time comes.

You can observe the growth of messages in the queue through the management interface when executing the unit test:
every 10s, add 1 message .

More exciting, follow me.