12. RabbitMQ message Confirm mode
12.1 Introduction to Confirm Mode
The confirmation confirmation mechanism of the message means that after the producer delivers the message and reaches the exchange switch in the message server Broker, it will give the producer a response, and the producer receives the response to determine whether the message is sent to the broker normally In the exchange, this is also an important guarantee for the reliable delivery of messages;
12.2 Specific code settings
| 1 Configure the file application.yml to enable the confirmation mode: spring.rabbitmq.publisher-confirm-type=_correlated_2 Write a class to implement implements RabbitTemplate.ConfirmCallback to judge the successful and failed ack results. According to the specific results, if the ack is false, the Messages are resent or logged; set the confirmation callback method of rabbitTemplate 3 rabbitTemplate.setConfirmCallback(messageConfirmCallBack); |
|---|
Reference Code:
| @Component
public class MessageConfirmCallBack implements RabbitTemplate.ConfirmCallback { /** * After the switch receives the message, it will call back the method * * @param correlationData associated data * @param ack has two values, true and false, true means Success: the message arrived at the switch correctly, otherwise false means the message did not reach the switch correctly * @param cause What is the reason why the message did not reach the switch correctly */ @Override public void confirm(CorrelationData correlationData, boolean ack, String cause) { System .out.println("correlationData = " + correlationData); System.out.println("ack = " + ack); System.out.println("cause = " + cause); if (ack) { //normal } else {
// Abnormal, may need to log or resend
}
}
| } |
|---|
Send message reference code
| @Service
public class MessageService { @Resource private RabbitTemplate rabbitTemplate; @Resource private MessageConfirmCallBack messageConfirmCallBack; @PostConstruct //When the bean is initialized, it will call this method once, only once, and play the role of initialization public void init() { rabbitTemplate.setConfirmCallback(messageConfirmCallBack); } /** * Send message */ public void sendMessage() { //correlation data object CorrelationData correlationData = new CorrelationData(); correlationData.setId("O159899323"); //For example, set an order ID, when the time comes in the confirm callback, you can know which order has not been sent to the exchange
rabbitTemplate.convertAndSend(RabbitConfig.EXCHANGE + 123, “info”, “hello”, correlationData);
System.out.println("The message has been sent...");
}
}
|
| — |
Two examples: Interfaces can be implemented directly when sending messages
13. RabbitMQ message Return mode
The path of rabbitmq's entire message delivery is:
producer —> exchange —> queue —> consumer
A confirmCallback will be returned when the message is sent from the producer to the exchange;
a returnCallback will be returned if the message fails to be delivered from the exchange –> queue;
we can use these two callbacks to control the reliable delivery of the message;
enable the confirmation mode;
use rabbitTemplate.setConfirmCallback to set the callback function , when the message is sent to the exchange, the confirm method is called back. Judging ack in the method, if it is true, the sending is successful, if it is false, the sending fails and needs to be processed;
note that in the configuration file, the return mode is turned on;
| spring.rabbitmq.publisher-returns: true |
|---|
Use rabbitTemplate.setReturnCallback to set the return function. When the message fails to be routed from the exchange to
the queue, the message will be returned to the producer and the callback function returnedMessage will be executed;
|
@Component
public class MessageReturnCallBack implements RabbitTemplate.ReturnsCallback { /** * This method will be triggered when the message does not arrive in the queue correctly from the exchange * If the message arrives in the queue correctly from the exchange, then this method will not be triggered * * @param returned */ @Override public void returnedMessage(ReturnedMessage returned) { System.out.println("Message return mode: " + returned); }
| } |
|---|
Reference sending code
| @Service
public class MessageService { @Resource private RabbitTemplate rabbitTemplate; @Resource private MessageReturnCallBack messageReturnCallBack; @PostConstruct //When the bean is initialized, it will call this method once, only once, and play the role of initialization public void init() { rabbitTemplate.setReturnsCallback(messageReturnCallBack); } /** * Send message */ public void sendMessage() { rabbitTemplate.convertAndSend(RabbitConfig.EXCHANGE, “info123”, “hello”); System.out.println("The message has been sent... ”); } } | | — |
Code 2, directly implement RabbitTemplate.ReturnsCallback when sending a message
| @Service
public class MessageService implements RabbitTemplate.ReturnsCallback { @Resource private RabbitTemplate rabbitTemplate; @PostConstruct //When the bean is initialized, it will call this method once, only once, and play the role of initialization public void init() { rabbitTemplate. setReturnsCallback(this); } /** * When the message does not arrive at the queue correctly from the switch, this method will be triggered * If the message arrives at the queue correctly from the switch, then this method will not be triggered * * @param returned / @ Override public void returnedMessage(ReturnedMessage returned) { System.out.println("Message return mode: " + returned); } / * * Send message */
public void sendMessage() { rabbitTemplate.convertAndSend(RabbitConfig.EXCHANGE, “info123”, “hello”); System.out.println("The message has been sent..."); } } | | — |
Code Reference 3 Implement RabbitTemplate.ConfirmCallback and RabbitTemplate.ReturnsCallback at the same time
|
@Service
public class MessageService implements RabbitTemplate.ConfirmCallback, RabbitTemplate.ReturnsCallback { @Resource private RabbitTemplate rabbitTemplate; @PostConstruct //When the bean is initialized, it will call this method once, only once, and play the role of initialization public void init( ) { rabbitTemplate.setConfirmCallback(this); rabbitTemplate.setReturnsCallback(this); } /** * After the switch receives the message, it will call back this method * * @param correlationData associated data * @param ack has two values, true and false, true means success: the message arrived at the switch correctly, otherwise false means that the message did not reach the switch correctly * @param cause What is the reason why the message did not reach the switch correctly / @Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) { System.out.println("correlationData = " + correlationData); System.out.println("ack = " + ack); System.out.println("cause = " + cause); if (ack) { // OK } else { // abnormal } } /
*
* This method will be triggered when the message from the exchange did not arrive correctly in the queue
* if the message arrived correctly from the exchange queue, then the method will not be triggered
*
* @param returned
/
@Override
public void returnedMessage(ReturnedMessage returned) { System.out.println("Message return mode: " + returned);
}
/ *
* Send message
*/
public void sendMessage() { //correlation data object CorrelationData correlationData = new CorrelationData(); correlationData.setId("O159899323"); //For example, set an order ID, then in the confirm callback , you can know which order is not sent to the exchange rabbitTemplate.convertAndSend(RabbitConfig.EXCHANGE, "info123", "hello", correlationData); System.out.println("The message has been sent..."); } } | | — |
14. RabbitMQ switch detailed properties
14.1 Specific parameters
1. Name: the name of the switch; it is a string
2. Type: the type of the switch, direct, topic, fanout, headers 3.
Durability: Persistence, which declares whether the switch is durable, indicating whether the switch still exists after the server restarts;
4. Auto delete: whether to delete automatically, once a queue was bound to the switch, and later unbound, the switch will be automatically deleted; 5,
Internal: used internally, if yes, the client cannot directly send messages to This switch, it can only be used for switch-to-switch binding.
6. Arguments: There is only one value, alternate-exchange, which means the alternate switch;
14.2 Code Demonstration
Conclusion 1: The switch and queue will not be created before the message is sent.
Conclusion 2: After the message is sent, if the switch does not exist, the switch will be created. If the queue does not exist, a new queue will be created. Conclusion
3: The switch or queue is created After re-creating, if you modify the switch or queue parameters
, an error 406 will be reported (inequivalent arg 'durable' for exchange 'exchange.durability' in vhost 'powernode': received 'false' but current is 'true', class-id= 40, method-id=10))
Conclusion 4: Set persistence to false, restart rabbitmq-server, then the switch will be lost, experiment with the durable parameter, first look at the console, and then restart rabbitmq-server
Conclusion 5: Experiment with automatic deletion as true , unbind manually from the console, you will find that it is automatically deleted
14.3 Standby switch
14.3.1 Usage Scenarios of Standby Switch
When the message is ready to be routed to the queue through the switch, it is found that there is no corresponding queue to deliver the information. In rabbitmq, the message will be discarded by default. If we want to monitor which messages are delivered to the queue that does not correspond, we can use the backup switch to Realized, it can receive the message of the standby switch, and then record the log or send the alarm information.
14.3.2 Main codes and notes
The example of the backup switch is as follows:
Note: the backup switch generally uses a fanout switch.
When testing: specify a wrong route
. Important: the normal switch setting parameters are bound to the backup switch
| Map<String, Object> arguments = new HashMap<>();
//Specify who is the backup switch of the current normal switch
arguments.put(“alternate-exchange”, EXCHANGE_ALTERNATE);
//DirectExchange(String name, boolean durable , boolean autoDelete, Map<String, Object> arguments)
return new DirectExchange(EXCHANGE, true, false, arguments);
| //return ExchangeBuilder.directExchange(EXCHANGE).withArguments(args).build(); |
|---|
14.3.3 Reference configuration code
| @Configuration
public class RabbitConfig { //The name of the switch is a string public static final String EXCHANGE = "exchange"; //The name of the queue is a string public static final String QUEUE = "queue"; //Definition A routing key public static final String INFO = "info"; //---------------------------------- --------- //The name of the exchange is a string public static final String EXCHANGE_ALTERNATE = “exchange.alternate”; //The name of the queue is a string public static final String QUEUE_ALTERNATE = “queue. alternate"; //A defined routing key public static final String ALTERNATE = "alternate"; @Bean public DirectExchange directExchange() {
Map<String, Object> arguments = new HashMap<>();
arguments.put(“alternate-exchange”, EXCHANGE_ALTERNATE); //Specify who is the backup switch of the current normal switch
//DirectExchange(String name, boolean durable, boolean autoDelete, Map<String, Object> arguments)
return new DirectExchange(EXCHANGE, true, false, arguments);
}
/**
* Declare a queue
*
* @return
/
@Bean
public Queue queue() { return QueueBuilder.durable( QUEUE).build(); } /
*
* @Qualifier The name of the qualified bean is the bean of directExchange
*
* @param directExchange
* @return
/
@Bean
public Binding binding(DirectExchange directExchange, Queue queue) {
return BindingBuilder.bind(queue).to(directExchange).with(INFO);
}
//-----------------------------------------
/*
* 备用交换机需要用Fanout交换机；
*
* @return
*/
@Bean
public FanoutExchange alternateExchange() {
//DirectExchange(String name, boolean durable, boolean autoDelete, Map<String, Object> arguments)
return new FanoutExchange(EXCHANGE_ALTERNATE, true, false);
}
@Bean
public Queue alternateQueue() {
return QueueBuilder.durable(QUEUE_ALTERNATE).build();
}
@Bean
public Binding alternateBnding(FanoutExchange alternateExchange, Queue alternateQueue) {
return BindingBuilder.bind(alternateQueue).to(alternateExchange);
}
}
|
| — |
14.3.4 Refer to send message codes
| @Service
public class MessageService { @Resource private RabbitTemplate rabbitTemplate; /** * send a message */ public void sendMessage() { //We intentionally write the wrong routing key, because our normal switch has a backup switch, so the message will Enter the standby switch//to enter the standby queue, we can write a program to receive the message of the standby queue, and notify the relevant personnel to deal with it after receiving it//If the normal switch does not have a standby switch, the message will be discarded. rabbitTemplate.convertAndSend(RabbitConfig.EXCHANGE, "info1223", "hello"); System.out.println("The message has been sent..."); } } | | — |
15. RabbitMQ queue detailed properties
15.1 Specific parameters
Type: Queue type
Name: Queue name, which is a string, any string is fine;
Durability: Declare whether the queue is persistent, indicating whether the queue still exists after the server restarts;
Auto delete: Whether to automatically delete, if true, When no consumer is connected to this queue, the queue will be automatically deleted; Exclusive: The queue
with the exclusive attribute is only visible to the connection that declares it for the first time, and will be automatically deleted when the connection is disconnected; basically do not set it, set it to false Arguments : Other attributes of the queue, such as specifying DLX (dead letter exchange, etc.); 1. x-expires: Number When the Queue (queue) is not accessed within the specified time, the queue will be automatically deleted; 2. x-message-ttl : How long does a message published by Number be canceled after it exists in the queue (in milliseconds); 3. x-overflow: String sets the queue overflow behavior, what will happen to the message when the maximum length of the queue is reached, and the valid value is Drop Head Or Reject Publish; 4. x-max-length: The maximum length of the message that the Number queue can accommodate. When the length is exceeded, the new message will overwrite the front message, similar to the LRU algorithm of Redis; 5. x-single -active-consumer: The default is false to activate a single consumer, that is, the queue can only have one messager to consume messages; 6. x-max-length-bytes: Number
Limit the maximum occupied space of the queue, and use the LRU algorithm similar to Redis when it exceeds;
7. x-dead-letter-exchange: String
specifies the dead letter exchange associated with the queue. Sometimes we hope that the queue will overflow when the message reaches the upper limit The message will not be deleted, but will be stored in another queue;
8.x-dead-letter-routing-key: String
specifies the routing key of the dead letter switch, generally defined together with 6;
9.x- max-priority: Number
If you add a priority parameter to a queue, then the queue is a priority queue;
(1), add a priority parameter to the queue to make it a priority queue
x-max-priority=10【0- 255 value range]
(2) Add priority attribute to the message
Through the priority feature, a queue can be queued for consumption;
| MessageProperties messageProperties=new MessageProperties();
| messageProperties.setPriority(8); |
|---|
10. x-queue-mode: String (you can understand it)
queue type x-queue-mode=lazy lazy queue, keep as many messages as possible on disk to reduce RAM usage, if not set, the queue will reserve memory Cache to deliver messages as quickly as possible;
11. x-queue-master-locator: String (less used, not to mention)
set the location information of the master node assigned to the queue in cluster mode;
each queue has a master Node, all operations on the queue are completed on the master in advance, and then the same operation is performed on the slave;
each different queue can be located on a different cluster node, and if these queues are configured with mirror queues, there will be 1 A master and multiple slaves.
Basically all operations fall on the master, so if the masters of these queues fall on individual service nodes, and other nodes are idle, load balancing cannot be achieved, and performance will inevitably be affected; about the
master There are several strategies for queue host allocation. You can use the x-queue-master-locator parameter when declaring the queue, or set the queue-master-locator on the policy, or define queue_master_locator directly in the rabbitmq configuration file. There are three options Alternative strategies:
(1) min-masters: select the service node host with the least number of master queues; (
2) client-local: select the service node host connected to the client;
(3) random: randomly assign;
15.2 Reference code
|
@Configuration
public class RabbitConfig {
public static final String EXCHANGE = “exchange”;
public static final String QUEUE = “queue”;
public static final String KEY = “info”;
QueueBuilder builder;
@Bean
public DirectExchange directExchange() {
return ExchangeBuilder.directExchange(EXCHANGE).build();
}
@Bean
public Queue queue() {
Map<String, Object> arguments = new HashMap<>();
//arguments.put(“x-expires”, 5000);
//arguments.put(“x-max-length”, 5);
//arguments.put(“x-overflow”, “reject-publish”);
arguments.put(“x-single-active-consumer”, false); //TODO ???
//arguments.put(“x-max-length-bytes”, 20); // unit is byte
// arguments.put("x-max-priority", 10); // 0-255 //Indicates that the currently declared queue is set as a priority queue, then this queue allows messages to be queued //Set the queue to
delay Mode, keep as many messages as possible on the disk to reduce the use of RAM memory, if not set, the queue will retain memory cache to deliver messages as fast as possible; //Sometimes we call
this kind of queue: lazy queue
/ /arguments.put("x-queue-mode", "lazy");
//Set the queue version. Defaults to version 1.
// Version 1 has a log based index which embeds small messages.
// Version 2 has a different index that improves memory usage and performance in many scenarios, and provides per-queue storage for previously embedded messages.
//arguments.put(“x-queue-version”, 2);
// x-queue-master-locator: Set the master node information of the mirror queue in cluster mode.
//arguments.put("x-queue-master-locator", QueueBuilder.LeaderLocator.clientLocal.getValue());
//-------------------------
//arguments.put("x-expires", 10000); //Automatic expiration, 10 seconds
/ /arguments.put("x-message-ttl", 10000); //Automatic expiration, 10 seconds, will not delete the queue
//QueueBuilder class has a definition, set the queue overflow behavior, when the maximum length of the queue is reached, the message will What happens, valid values are drop-head, reject-publish
//arguments.put("x-max-length", 5);
//arguments.put("x-overflow", QueueBuilder.Overflow.dropHead.getValue( ));
//Indicates whether the queue is a single active consumer. When true, only one consumer consumes messages in the registered consumer group, and the others are ignored. When false, the message is circulated to all consumers (default false) //arguments
. put("x-single-active-consumer", true);
// x-max-length-bytes, the queue message content occupies the maximum space, limited by the memory size, if the threshold is exceeded, the message will be deleted from the head of the queue;
//arguments.put("x-max-length-bytes", 10);
//The parameter is a positive integer between 1 and 255, indicating the maximum priority that the queue should support. The larger the number, the higher the priority. If the priority field is not set, the value of the priority field defaults to 0; if the priority queue The priority attribute is set to be greater than x-max-priority, then the value of priority is set to the value of x-max-priority.
//arguments.put("x-max-priority", 10);
//Set the queue to delayed mode, keeping as many messages as possible on disk to reduce RAM usage; if not set, the queue will reserve memory Caching to deliver messages as fast as possible;
//arguments.put("x-queue-mode", "lazy");
arguments.put("x-queue-version", 2);
// x-queue-master -locator: Set the master node information of the mirror queue in cluster mode.
arguments.put(“x-queue-master-locator”, QueueBuilder.LeaderLocator.clientLocal.getValue());
//----------------------- ----------------------
// Queue(String name, boolean durable, boolean exclusive, boolean autoDelete, @Nullable Map<String, Object> arguments)
return new Queue(QUEUE, true, false, false, arguments);
}
@Bean
public Binding binding(DirectExchange directExchange, Queue queue) {
return BindingBuilder.bind(queue).to(directExchange).with(KEY);
}
}
|
| — |
Experiment with the durable parameter to restart rabbitmq-server, and the queue is lost.
Experiment with the autodelete parameter: add the receiver and find that the service is stopped. If there are no consumers for so long, the queue will be automatically deleted
16. Reliable delivery of messages
The reliable delivery of messages is to ensure that every link in the message delivery process is successful, so this will definitely sacrifice some performance, and performance and reliability cannot be achieved at the same time; if the real-time consistency requirements of the business are not particularly high, you
can Sacrificing some reliability for performance.
① represents that the message is sent from the producer to the Exchange;
② represents that the message is routed from the Exchange to the Queue;
③ represents that the message is stored in the Queue;
④ represents that the consumer listens to the Queue and consumes the message; 1. Ensure that
the message is sent to the switch of the RabbitMQ server.
Network or Broker problems cause ① to fail, and at this time the producer should know whether the message has been sent to the Broker’s exchange correctly;
there are two solutions:
the first is to enable the Confirm (confirmation) mode; (asynchronous)
the second It is to enable the Transaction mode; (low performance, rarely used in actual projects)
2. Ensure that the message is routed to the correct queue. The
② failure may be caused by the wrong routing keyword, or the queue does not exist, or the queue name is wrong.
Using the return mode, you can return to the producer when the message cannot be routed;
of course, in the actual production environment, we will not have this kind of problem, and we will go online after strict testing (rarely such a problem);
another The first way is to use an alternate-exchange. Messages that cannot be routed will be sent to this alternate exchange; 3.
Ensure that messages are stored correctly in the queue.
It may be caused by system downtime, restart, shutdown, etc. The message is lost, that is, there is a problem with ③;
solution:
(1), queue persistence
code:
| QueueBuilder.durable(QUEUE).build(); |
|---|
(2), switch persistence
code:
| ExchangeBuilder.directExchange(EXCHANGE).durable(true).build(); |
|---|
(3), message persistence
code:
default persistence
| MessageProperties messageProperties = new MessageProperties();
//Set message persistence, of course, it is persistent by default, so you don't need to set it, you can view the source code
| messageProperties.setDeliveryMode(MessageDeliveryMode.PERSISTENT); |
|---|
(4) Clusters, mirrored queues, high availability
(5) Ensure that messages are delivered from the queue to consumers correctly.
Use the manual ack confirmation mechanism during message consumption to ensure that;
if consumers receive messages and cannot process them in time, an exception will occur. Or an exception occurs during processing, which will cause ④ to fail.
In order to ensure that messages reach consumers reliably from the queue, RabbitMQ provides a message acknowledgment mechanism (message acknowledgment);
#Enable manual ack message consumption confirmation
spring.rabbitmq.listener.simple.acknowledge-mode=manual
When consumers subscribe to the queue, they pass The above configuration does not automatically confirm, but uses manual confirmation. RabbitMQ will wait for the consumer to explicitly reply to the confirmation signal before deleting the message from the queue;
if the message consumption fails, you can also call basicReject() or basicNack() to reject the current message rather than confirmation. If the requeue parameter is set to true, this message can be re-stored in the queue to be sent to the next consumer (of course, when there is only one consumer, this method may cause infinite loop repeated consumption, which can be delivered to new queue, or just print the exception log);
17. Message idempotence
Idempotency when consuming messages (messages are not consumed repeatedly)
. The same message is received for the first time and the business is processed normally. If the message is received for the second time, the business cannot be processed anymore, otherwise the processing will be repeated;
idempotence Equality is: for a resource, whether you request it once or multiple times, the impact on the resource itself should be the same, and the resource cannot be affected repeatedly because of repeated requests; take interface idempotency as an example
:
interface Idempotency means: an interface is called repeatedly with the same parameters without causing business errors, then this interface is idempotent;
registration interface;
sending SMS verification code interface;
for example, I pay twice for the same order, but Only one payment will be deducted, and the second payment will not be deducted, which shows that this payment interface is idempotent;
how to avoid the problem of repeated consumption of messages? (Idempotency during message consumption)
Globally unique ID +
When Redis producers send messages, they set a globally unique messageId for each message. After consumers get the message, they use the setnx command to put messageId as a key in redis Middle: setnx(messageId, 1), if it returns 1, it means that it has not been consumed before, and it is consumed normally; if it returns 0, it means that this message has been consumed before, and it is discarded; the specific code refers to the following code; reference
code
:
| //1. Write the unique ID of the message into redis
boolean flag = stringRedisTemplate.opsForValue().setIfAbsent(“idempotent:” + orders.getId(), String.valueOf(orders.getId())); //If If the key does not exist in redis, then set it, and if it exists, do not set it
if (flag) { //The key does not exist and returns true
//It is equivalent to consuming this message for the first time
//TODO processing business
System.out.println("Normal Handle business..." + orders.getId());
}
| |
— |