[Microservices] Advanced RabbitMQ

During the use of message queues, there are many practical problems that need to be considered:

1. Message reliability

From sending the message to receiving it by the consumer, it will manage multiple processes:

Each of these steps can lead to message loss, common reasons for loss include:

• Lost on send:
  • The message sent by the producer was not delivered to the exchange
  • The message did not reach the queue after it arrived at the exchange
• MQ is down, the queue will lose the message
• The consumer crashes without consuming the message after receiving the message

For these problems, RabbitMQ gives solutions respectively:

• Producer Confirmation Mechanism
• mq persistence
• Consumer Confirmation Mechanism
• Failure retry mechanism

Below we demonstrate each step through a case.

First, create a demo project, the project structure is as follows:

1.1. Producer message confirmation

RabbitMQ provides a publisher confirm mechanism to avoid loss of messages sent to MQ. This mechanism must assign a unique ID to each message. After the message is sent to MQ, a result will be returned to the sender, indicating whether the message is processed successfully.

There are two ways to return results:

• publisher-confirm, the sender confirms
  • The message is successfully delivered to the switch and returns ack
  • The message was not delivered to the exchange, return nack
• publisher-return, sender receipt
  • The message was delivered to the exchange, but not routed to the queue. Return ACK and the reason for routing failure.

Notice:

1.1.1. Modify configuration

First, modify the application.yml file in the publisher service and add the following content:

spring:
  rabbitmq:
    publisher-confirm-type: correlated
    publisher-returns: true
    template:
      mandatory: true
   

illustrate:

• publish-confirm-type: Enable publisher-confirm, here supports two types:
  • simple: Synchronously wait for the confirm result until timeout
  • correlated: Asynchronous callback, define ConfirmCallback, this ConfirmCallback will be called back when MQ returns the result
• publish-returns: Enable the publish-return function, which is also based on the callback mechanism, but defines the ReturnCallback
• template.mandatory: Defines the policy when message routing fails. true, call ReturnCallback; false: discard the message directly

1.1.2. Define the Return callback

Each RabbitTemplate can only configure one ReturnCallback, so it needs to be configured when the project is loaded:

Modify the publisher service and add one:

package cn.itcast.mq.config;

import lombok.extern.slf4j.Slf4j;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.beans.BeansException;
import org.springframework.context.ApplicationContext;
import org.springframework.context.ApplicationContextAware;
import org.springframework.context.annotation.Configuration;

@Slf4j
@Configuration
public class CommonConfig implements ApplicationContextAware {
    
    
    @Override
    public void setApplicationContext(ApplicationContext applicationContext) throws BeansException {
    
    
        // 获取RabbitTemplate
        RabbitTemplate rabbitTemplate = applicationContext.getBean(RabbitTemplate.class);
        // 设置ReturnCallback
        rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
    
    
            // 投递失败，记录日志
            log.info("消息发送失败，应答码{}，原因{}，交换机{}，路由键{},消息{}",
                     replyCode, replyText, exchange, routingKey, message.toString());
            // 如果有业务需要，可以重发消息
        });
    }
}

1.1.3. Define ConfirmCallback

ConfirmCallback can be specified when sending a message, because the logic of each business processing confirm success or failure is not necessarily the same.

In the cn.itcast.mq.spring.SpringAmqpTest class of the publisher service, define a unit test method:

public void testSendMessage2SimpleQueue() throws InterruptedException {
    
    
    // 1.消息体
    String message = "hello, spring amqp!";
    // 2.全局唯一的消息ID，需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 3.添加callback
    correlationData.getFuture().addCallback(
        result -> {
    
    
            if(result.isAck()){
    
    
                // 3.1.ack，消息成功
                log.debug("消息发送成功, ID:{}", correlationData.getId());
            }else{
    
    
                // 3.2.nack，消息失败
                log.error("消息发送失败, ID:{}, 原因{}",correlationData.getId(), result.getReason());
            }
        },
        ex -> log.error("消息发送异常, ID:{}, 原因{}",correlationData.getId(),ex.getMessage())
    );
    // 4.发送消息
    rabbitTemplate.convertAndSend("task.direct", "task", message, correlationData);

    // 休眠一会儿，等待ack回执
    Thread.sleep(2000);
}

1.2. Message Persistence

The producer confirms that the message can be delivered to the RabbitMQ queue, but after the message is sent to RabbitMQ, if there is a sudden downtime, the message may also be lost.

To ensure that messages are safely stored in RabbitMQ, the message persistence mechanism must be enabled.

• switch persistence
• queue persistence
• message persistence

1.2.1. Switch Persistence

The switch in RabbitMQ is non-persistent by default, and it will be lost after mq restarts.

In SpringAMQP, switch persistence can be specified by code:

@Bean
public DirectExchange simpleExchange(){
    
    
    // 三个参数：交换机名称、是否持久化、当没有queue与其绑定时是否自动删除
    return new DirectExchange("simple.direct", true, false);
}

In fact, by default, switches declared by Spring AMQP are persistent.

You can see the mark on the persistent switch in the RabbitMQ console D:

1.2.2. Queue Persistence

The queue in RabbitMQ is non-persistent by default, and it will be lost after mq restarts.

In SpringAMQP, switch persistence can be specified by code:

@Bean
public Queue simpleQueue(){
    
    
    // 使用QueueBuilder构建队列，durable就是持久化的
    return QueueBuilder.durable("simple.queue").build();
}

In fact, by default, queues declared by Spring AMQP are persistent.

You can see the mark on the persistent queue in the RabbitMQ console D:

1.2.3. Message persistence

When using SpringAMQP to send messages, you can set the properties of the message (MessageProperties) and specify the delivery-mode:

• 1: non-persistent
• 2: Persistence

Specify with java code:

By default, any message sent by Spring AMQP is persistent, without specifying it.

1.3. Consumer message confirmation

RabbitMQ is a mechanism that burns after reading . RabbitMQ confirms that the message will be deleted immediately after being consumed by the consumer.

RabbitMQ confirms whether the consumer has successfully processed the message through the consumer receipt: after the consumer obtains the message, it should send an ACK receipt to RabbitMQ to indicate that it has processed the message.

Imagine this scenario:

• 1) RabbitMQ delivers messages to consumers
• 2) After the consumer gets the message, it returns ACK to RabbitMQ
• 3) RabbitMQ delete message
• 4) The consumer is down, and the message has not been processed

In this way, the message is lost. Therefore, the timing of the consumer returning ACK is very important.

SpringAMQP allows configuration of three confirmation modes:

•manual: manual ack, you need to call the api to send ack after the business code ends.

•auto: automatic ack, the listener code is monitored by spring to see if there is an exception, if there is no exception, it will return ack; if an exception is thrown, it will return nack

•none: Close ack, MQ assumes that the consumer will successfully process the message after getting it, so the message will be deleted immediately after delivery

From this we can see:

• In none mode, message delivery is unreliable and may be lost
• The auto mode is similar to the transaction mechanism. When an exception occurs, it returns nack, and the message is rolled back to mq; if there is no exception, it returns ack
• manual: According to the business situation, judge when to ack

Generally, we can use the default auto.

1.3.1. Demo none mode

Modify the application.yml file of the consumer service and add the following content:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: none # 关闭ack

Modify the method in the SpringRabbitListener class of the consumer service to simulate a message processing exception:

@RabbitListener(queues = "simple.queue")
public void listenSimpleQueue(String msg) {
    
    
    log.info("消费者接收到simple.queue的消息：【{}】", msg);
    // 模拟异常
    System.out.println(1 / 0);
    log.debug("消息处理完成！");
}

The test can find that when the message processing throws an exception, the message is still deleted by RabbitMQ.

1.3.2. Demo auto mode

Change the confirmation mechanism to auto again:

spring:
  rabbitmq:
    listener:
      simple:
        acknowledge-mode: auto # 关闭ack

Break the point at the abnormal position and send the message again. When the program is stuck at the break point, you can find that the message status is unack (undetermined status):

After an exception is thrown, because Spring will automatically return nack, the message returns to the Ready state and is not deleted by RabbitMQ:

1.4. Consumption failure retry mechanism

When the consumer has an exception, the message will continue to requeue (re-enter the queue) to the queue, and then resend to the consumer, and then the exception again, requeue again, infinite loop, causing the message processing of mq to soar, bringing unnecessary pressure:

How to do it?

1.4.1. Local retry

We can use Spring's retry mechanism to use local retry when an exception occurs in the consumer, instead of unlimited requeue to the mq queue.

Modify the application.yml file of the consumer service and add the content:

spring:
  rabbitmq:
    listener:
      simple:
        retry:
          enabled: true # 开启消费者失败重试
          initial-interval: 1000 # 初识的失败等待时长为1秒
          multiplier: 1 # 失败的等待时长倍数，下次等待时长 = multiplier * last-interval
          max-attempts: 3 # 最大重试次数
          stateless: true # true无状态；false有状态。如果业务中包含事务，这里改为false

Restart the consumer service and repeat the previous test. It can be found:

• After retrying 3 times, SpringAMQP will throw an exception AmqpRejectAndDontRequeueException, indicating that the local retry triggers
• Check the RabbitMQ console and find that the message has been deleted, indicating that SpringAMQP returned ack at the end, and mq deleted the message

in conclusion:

• When local retry is enabled, if an exception is thrown during message processing, it will not be requeed to the queue, but will be retried locally by the consumer
• After the maximum number of retries is reached, Spring will return ack and the message will be discarded

1.4.2. Failure strategy

In the previous test, after reaching the maximum number of retries, the message will be discarded, which is determined by the internal mechanism of Spring.

After the retry mode is turned on, the number of retries is exhausted. If the message still fails, the MessageRecovery interface is required to handle it. It contains three different implementations:

• RejectAndDontRequeueRecoverer: After the retries are exhausted, directly reject and discard the message. This is the default

• ImmediateRequeueMessageRecoverer: After retries are exhausted, nack is returned, and the message is re-queued

• RepublishMessageRecoverer: After the retries are exhausted, deliver the failure message to the specified exchange

A more elegant solution is RepublishMessageRecoverer. After a failure, the message will be delivered to a designated queue dedicated to storing exception messages, and subsequent processing will be done manually.

1) Define the switch and queue for processing failed messages in the consumer service

@Bean
public DirectExchange errorMessageExchange(){
    
    
    return new DirectExchange("error.direct");
}
@Bean
public Queue errorQueue(){
    
    
    return new Queue("error.queue", true);
}
@Bean
public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    
    
    return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
}

2) Define a RepublishMessageRecoverer, associate queues and switches

@Bean
public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    
    
    return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
}

Full code:

package cn.itcast.mq.config;

import org.springframework.amqp.core.Binding;
import org.springframework.amqp.core.BindingBuilder;
import org.springframework.amqp.core.DirectExchange;
import org.springframework.amqp.core.Queue;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.retry.MessageRecoverer;
import org.springframework.amqp.rabbit.retry.RepublishMessageRecoverer;
import org.springframework.context.annotation.Bean;

@Configuration
public class ErrorMessageConfig {
    
    
    @Bean
    public DirectExchange errorMessageExchange(){
    
    
        return new DirectExchange("error.direct");
    }
    @Bean
    public Queue errorQueue(){
    
    
        return new Queue("error.queue", true);
    }
    @Bean
    public Binding errorBinding(Queue errorQueue, DirectExchange errorMessageExchange){
    
    
        return BindingBuilder.bind(errorQueue).to(errorMessageExchange).with("error");
    }

    @Bean
    public MessageRecoverer republishMessageRecoverer(RabbitTemplate rabbitTemplate){
    
    
        return new RepublishMessageRecoverer(rabbitTemplate, "error.direct", "error");
    }
}

1.5. Summary

How to ensure the reliability of RabbitMQ messages?

• Enable the producer confirmation mechanism to ensure that the producer's message can reach the queue
• Enable the persistence function to ensure that the message will not be lost in the queue before it is consumed
• Turn on the consumer confirmation mechanism as auto, and the spring will complete the ack after confirming that the message is processed successfully
• Enable the consumer failure retry mechanism and set MessageRecoverer. After multiple retries fail, the message will be delivered to the abnormal switch and handed over to manual processing.

2. Dead letter switch

2.1. Getting to know dead-letter switches

2.1.1. What is a dead letter switch

What is dead letter?

When a message in a queue meets one of the following conditions, it can become a dead letter:

• The consumer uses basic.reject or basic.nack to declare consumption failure, and the requeue parameter of the message is set to false
• The message is an expired message, no one consumes after timeout
• The queue message to be delivered is full and cannot be delivered

If the queue containing dead letters is configured with dead-letter-exchangeattributes and a switch is specified, the dead letters in the queue will be delivered to this switch, and this switch is called a dead letter exchange (Dead Letter Exchange, check DLX).

As shown in the figure, a message is rejected by the consumer and becomes a dead letter:

Because simple.queue is bound to the dead letter exchange dl.direct, the dead letter will be delivered to this exchange:

If the dead letter switch is also bound to a queue, the message will eventually enter the queue where the dead letter is stored:

In addition, when the queue delivers the dead letter to the dead letter exchange, it must know two pieces of information:

• dead letter switch name
• The RoutingKey bound to the dead letter exchange and the dead letter queue

Only in this way can we ensure that the delivered message can reach the dead letter exchange and be correctly routed to the dead letter queue.

2.1.2. Using the dead letter exchange to receive dead letters (extension)

In the failure retry strategy, the default RejectAndDontRequeueRecoverer will send reject to RabbitMQ after the number of local retries is exhausted, and the message becomes a dead letter and is discarded.

We can add a dead letter switch to simple.queue and bind a queue to the dead letter switch. In this way, the message will not be discarded after it becomes a dead letter, but will be finally delivered to the dead letter exchange and routed to the queue bound to the dead letter exchange.

In the consumer service, we define a set of dead letter switches and dead letter queues:

// 声明普通的 simple.queue队列，并且为其指定死信交换机：dl.direct
@Bean
public Queue simpleQueue2(){
    
    
    return QueueBuilder.durable("simple.queue") // 指定队列名称，并持久化
        .deadLetterExchange("dl.direct") // 指定死信交换机
        .build();
}
// 声明死信交换机 dl.direct
@Bean
public DirectExchange dlExchange(){
    
    
    return new DirectExchange("dl.direct", true, false);
}
// 声明存储死信的队列 dl.queue
@Bean
public Queue dlQueue(){
    
    
    return new Queue("dl.queue", true);
}
// 将死信队列 与 死信交换机绑定
@Bean
public Binding dlBinding(){
    
    
    return BindingBuilder.bind(dlQueue()).to(dlExchange()).with("simple");
}

2.1.3. Summary

What kind of message becomes dead letter?

• The message is rejected by the consumer or returns nack
• Message timed out and not consumed
• queue is full

What is the usage scenario of the dead letter exchange?

• If the queue is bound to a dead letter exchange, the dead letter will be delivered to the dead letter exchange;
• The dead letter switch can be used to collect all messages (dead letters) that consumers fail to process and hand them over to manual processing to further improve the reliability of the message queue.

2.2.TTL

If a message in a queue is not consumed after a timeout, it will become a dead letter. There are two cases of timeout:

• The queue where the message is located has a timeout set
• The message itself sets a timeout

2.2.1. Dead letter switch receiving timeout dead letter

In the SpringRabbitListener of the consumer service, define a new consumer, and declare the dead letter switch and dead letter queue:

@RabbitListener(bindings = @QueueBinding(
    value = @Queue(name = "dl.ttl.queue", durable = "true"),
    exchange = @Exchange(name = "dl.ttl.direct"),
    key = "ttl"
))
public void listenDlQueue(String msg){
    
    
    log.info("接收到 dl.ttl.queue的延迟消息：{}", msg);
}

2.2.2. Declare a queue and specify TTL

To set a timeout for a queue, you need to configure the x-message-ttl attribute when declaring the queue:

@Bean
public Queue ttlQueue(){
    
    
    return QueueBuilder.durable("ttl.queue") // 指定队列名称，并持久化
        .ttl(10000) // 设置队列的超时时间，10秒
        .deadLetterExchange("dl.ttl.direct") // 指定死信交换机
        .build();
}

Note that this queue sets the dead-letter switch todl.ttl.direct

Declare the switch and bind the ttl to the switch:

@Bean
public DirectExchange ttlExchange(){
    
    
    return new DirectExchange("ttl.direct");
}
@Bean
public Binding ttlBinding(){
    
    
    return BindingBuilder.bind(ttlQueue()).to(ttlExchange()).with("ttl");
}

Send a message, but don't specify a TTL:

@Test
public void testTTLQueue() {
    
    
    // 创建消息
    String message = "hello, ttl queue";
    // 消息ID，需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 发送消息
    rabbitTemplate.convertAndSend("ttl.direct", "ttl", message, correlationData);
    // 记录日志
    log.debug("发送消息成功");
}

Log of sent message:

View the log of the received message:

Because the TTL value of the queue is 10000ms, which is 10 seconds. You can see that the time difference between message sending and receiving is exactly 10 seconds.

2.2.3. When sending a message, set TTL

When sending a message, it is also possible to specify a TTL:

@Test
public void testTTLMsg() {
    
    
    // 创建消息
    Message message = MessageBuilder
        .withBody("hello, ttl message".getBytes(StandardCharsets.UTF_8))
        .setExpiration("5000")
        .build();
    // 消息ID，需要封装到CorrelationData中
    CorrelationData correlationData = new CorrelationData(UUID.randomUUID().toString());
    // 发送消息
    rabbitTemplate.convertAndSend("ttl.direct", "ttl", message, correlationData);
    log.debug("发送消息成功");
}

View sent message log:

Receive message log:

This time, the delay between sending and receiving was only 5 seconds. Note that when TTL is set for both the queue and the message, any expired one will become a dead letter.

2.2.4. Summary

What are the two ways of message timeout?

• Set the ttl attribute for the queue, and the messages that exceed the ttl time after entering the queue become dead letters
• Set the ttl attribute for the message, and the queue will become a dead letter after receiving the message exceeding the ttl time

How to realize that the consumer receives the message 20 seconds after sending a message?

• Specify a dead-letter exchange for the message's destination queue
• Bind the queue listened by the consumer to the dead letter exchange
• When sending a message, set the timeout period for the message to 20 seconds

2.3. Delay Queue

Using TTL combined with dead-letter switches, we realize the effect that consumers delay receiving messages after messages are sent. This message mode is called the delay queue (Delay Queue) mode.

Use cases for delay queues include:

• Delay in sending SMS
• The user places an order, if the user does not pay within 15 minutes, it will be automatically canceled
• Schedule a work meeting and automatically notify all participants 20 minutes later

Because there are so many demands for delay queues, RabbitMQ officially launched a plug-in that natively supports delay queue effects.

This plugin is the DelayExchange plugin. Refer to RabbitMQ's plugin list page: https://www.rabbitmq.com/community-plugins.html

For usage, please refer to the official website address: https://blog.rabbitmq.com/posts/2015/04/scheduling-messages-with-rabbitmq

2.3.1. Install the DelayExchange plugin

References [Microservices] Advanced RabbitMQ Deployment

2.3.2. Principle of Delay Exchange

DelayExchange requires an exchange to be declared as delayed. When we send a message to delayExchange, the flow is as follows:

• receive message
• Determine whether the message has the x-delay attribute
• If there is an x-delay attribute, it means that it is a delayed message, which is persisted to the hard disk, and the x-delay value is read as the delay time
• Return the routing not found result to the message sender
• After the x-delay time expires, re-deliver the message to the specified queue

2.3.3. Using DelayExchange

The use of the plug-in is also very simple: declare a switch, the type of the switch can be any type, just set the delayed attribute to true, and then declare the queue to bind it.

1) Declare the DelayExchange switch

Annotation-based (recommended):

It can also be based on @Bean:

2) Send a message

When sending a message, be sure to carry the x-delay attribute to specify the delay time:

2.3.4. Summary

What are the steps to use the delay queue plugin?

• Declare a switch and add the delayed attribute to true

• When sending a message, add the x-delay header, the value is the timeout time

3. Lazy queue

3.1. Message accumulation problem

When the speed at which producers send messages exceeds the speed at which consumers can process messages, messages in the queue will accumulate until the queue stores messages to the upper limit. Messages sent later will become dead letters and may be discarded. This is the problem of message accumulation.

There are two ways to solve message accumulation:

• Add more consumers and increase consumption speed. That is the work queue mode we said before
• Expand the queue volume and increase the stacking limit

To increase the queue capacity, it is obviously not possible to store messages in memory.

3.2. Lazy queue

Starting from version 3.6.0 of RabbitMQ, the concept of Lazy Queues, that is, lazy queues, has been added. The characteristics of lazy queue are as follows:

• After receiving the message, store it directly on disk instead of memory
• Consumers only read from disk and load them into memory when they want to consume messages
• Support millions of message storage

3.2.1. Set lazy-queue based on the command line

To set a queue as a lazy queue, you only need to specify the x-queue-mode attribute as lazy when declaring the queue. A running queue can be changed to a lazy queue via the command line:

rabbitmqctl set_policy Lazy "^lazy-queue$" '{"queue-mode":"lazy"}' --apply-to queues  

Command interpretation:

• rabbitmqctl: command-line tool for RabbitMQ
• set_policy: add a strategy
• Lazy: Policy name, which can be customized
• "^lazy-queue$": Match the name of the queue with a regular expression
• '{"queue-mode":"lazy"}': Set the queue mode to lazy mode
• --apply-to queues : The target of the policy is all queues

3.2.2. Declare lazy-queue based on @Bean

3.2.3. Declare LazyQueue based on @RabbitListener

3.3. Summary

The solution to the message accumulation problem?

• Bind multiple consumers on the queue to increase consumption speed
• Using lazy queues, you can store more messages in mq

What are the advantages of lazy queues?

• Based on disk storage, the upper limit of messages is high
• There is no intermittent page-out, and the performance is relatively stable

What are the disadvantages of lazy queues?

• Based on disk storage, message timeliness will be reduced
• Performance is limited by disk IO

4. MQ cluster

4.1. Cluster classification

RabbitMQ is written based on the Erlang language, and Erlang is a concurrency-oriented language that naturally supports cluster mode. The RabbitMQ cluster has two modes:

• Ordinary cluster : It is a distributed cluster, which distributes the queues to each node of the cluster, thereby improving the concurrency capability of the entire cluster.

• Mirror cluster : It is a master-slave cluster. On the basis of ordinary clusters, a master-slave backup function is added to improve the data availability of the cluster.

Although the mirror cluster supports master-slave, master-slave synchronization is not strongly consistent, and there may be a risk of data loss in some cases. Therefore, after RabbitMQ version 3.8, a new function was introduced: the arbitration queue to replace the mirror cluster, and the bottom layer uses the Raft protocol to ensure the data consistency between the master and the slave.

4.2. Ordinary cluster

4.2.1. Cluster structure and characteristics

Ordinary clusters, or classic clusters, have the following characteristics:

• Part of the data will be shared between each node in the cluster, including: switch and queue metadata. Messages in the queue are not included.
• When accessing a node in the cluster, if the queue is not on the node, it will be passed from the node where the data is located to the current node and returned
• If the node where the queue is located goes down, the messages in the queue will be lost

The structure is shown in the figure:

4.2.2. Deployment

References: [Microservices] Advanced RabbitMQ Deployment

4.3. Mirror cluster

4.3.1. Cluster structure and characteristics

Mirror cluster: the essence is the master-slave mode, with the following characteristics:

• Switches, queues, and messages in queues will be backed up synchronously between mirror nodes of each mq.
• The node that creates the queue is called the primary node of the queue, and the other nodes that are backed up to are called the mirror nodes of the queue.
• A queue's master node may be another queue's mirror node
• All operations are completed by the master node, and then synchronized to the mirror node
• After the master goes down, the mirror node will be replaced by the new master

The structure is shown in the figure:

4.3.2. Deployment

References: [Microservices] Advanced RabbitMQ Deployment

4.4. Arbitration Queue

4.4.1. Cluster characteristics

Arbitration queue: Arbitration queue is a new function available after version 3.8. It is used to replace the mirror queue and has the following characteristics:

• Like the mirror queue, it is a master-slave mode and supports master-slave data synchronization
• Very easy to use, no complicated configuration
• Master-slave synchronization based on Raft protocol, strong consistency

4.4.2. Deployment

References: [Microservices] Advanced RabbitMQ Deployment

4.4.3. Java code to create an arbitration queue

@Bean
public Queue quorumQueue() {
    
    
    return QueueBuilder
        .durable("quorum.queue") // 持久化
        .quorum() // 仲裁队列
        .build();
}

4.4.4. SpringAMQP connects to MQ cluster

Note that address is used here instead of host and port

spring:
  rabbitmq:
    addresses: 192.168.150.105:8071, 192.168.150.105:8072, 192.168.150.105:8073
    username: root 
    password: 1234
    virtual-host: /

If there are any deficiencies, please give more advice,
to be continued, continue to update!
Let's make progress together!