.Net Core&RabbitMQ message storage reliable mechanism

message transit point

The producer sends a message to RabbitMQ. If the switch can match the queue according to its own type and RoutingKey, it will be stored in the relevant queue. Lost or sent back to the producer, depending on the producer's configuration of the message.

  • If the producer sets Mandatory and it is true, the message will be returned to the producer.

  • When the producer sets Mandatory or false, in order to avoid message loss, the switch can be routed to the backup switch to be responsible for storage.

Mandatory

When the producer sends a message, you can set a parameter mandatory to determine the direction of the message if the switch cannot find a suitable queue according to its own type and RoutingKey after the message arrives at RabbitMQ.

  • When mandatory is true, the message is returned to the producer.

  • When mandatory is false, the message is discarded.

 producer code

When setting mandatory to true in the BasicPublish method parameter and the queue is not declared yet, there is only one switch, and the message will be returned.

var connFactory = new ConnectionFactory
{
    HostName = "xxx.xxx.xxx.xxx",
    Port = 5672,
    UserName = "rabbitmqdemo",
    Password = "rabbitmqdemo@test",
    VirtualHost = "rabbitmqdemo"};
using (var conn = connFactory.CreateConnection())
{
    using (var channel = conn.CreateModel())
    {
        var exchangeName = "mandatory_publishsubscribe_exchange";
        channel.ExchangeDeclare(exchange: exchangeName, type: "fanout");
        while (true)
        {
            Console.WriteLine("消息内容(exit退出):");
            var message = Console.ReadLine();
            if (message.Trim().ToLower() == "exit")
            {
                break;
            }

            var body = Encoding.UTF8.GetBytes(message);
            channel.BasicPublish(exchange: exchangeName, routingKey: "", mandatory: true, basicProperties: null, body: body);
            Console.WriteLine("消息内容发送完毕:" + message);
        }
    }
}

The producer sends a message, the exchange receives the message but there is no corresponding queue, and the message is returned.

 In order to intuitively know that the message has been returned to the producer, we can add a listener to monitor the returned message.

Listen for fallback messages

When mandatory is set to true, the message can be listened to when the message rolls back

channel.BasicReturn += new EventHandler<RabbitMQ.Client.Events.BasicReturnEventArgs>((sender, e) =>
{
    var message = Encoding.UTF8.GetString(e.Body.ToArray());
    Console.WriteLine($"收到回退消息:{message}");
});

The producer sends a message, because there is no matching queue, the message is returned, and the returned message can be seen intuitively.

 

 backup switch

When mandatory is set to false, messages are lost, which is not a good situation. A backup switch can be used to store messages that were to be discarded, and to obtain these messages when needed. In fact, there is nothing special about the backup switch, it is the same as the main switch and only acts as a backup.

 producer code

  1. When creating a master switch, give the parameter argument, set the backup switch of the master switch, and specify the name of the backup switch.

  2. Then declare a backup exchange and bind a queue to store discarded messages.

  3. The mandatory parameter is set to false when sending the message.

var connFactory = new ConnectionFactory
{
    HostName = "xxx.xxx.xxx.xxx",
    Port = 5672,
    UserName = "rabbitmqdemo",
    Password = "rabbitmqdemo@test",
    VirtualHost = "rabbitmqdemo"};
using (var conn = connFactory.CreateConnection())
{
    using (var channel = conn.CreateModel())
    {
        var exchangeName = "aedemo_publishsubscribe_exchange";
        var alternateExchangeName = "aedemo_ae_publishsubscribe_exchange";
        var arguments = new Dictionary<string, object>
        {
            { "alternate-exchange", alternateExchangeName }
        };
        channel.ExchangeDeclare(exchange: exchangeName, type: "fanout", arguments: arguments);
        channel.ExchangeDeclare(exchange: alternateExchangeName, type: "fanout");

        var alternateExchangeQueueName = alternateExchangeName + "_worker";
        channel.QueueDeclare(queue: alternateExchangeQueueName, durable: false, exclusive: false, autoDelete: false, arguments: null);
        channel.QueueBind(queue: alternateExchangeQueueName, exchange: alternateExchangeName, routingKey: "");

        while (true)
        {
            Console.WriteLine("消息内容(exit退出):");
            var message = Console.ReadLine();
            if (message.Trim().ToLower() == "exit")
            {
                break;
            }

            var body = Encoding.UTF8.GetBytes(message);
            channel.BasicPublish(exchange: exchangeName, routingKey: "", mandatory: false, basicProperties: null, body: body);
            Console.WriteLine("消息内容发送完毕:" + message);
        }
    }
}

Start the program, and you can see from the web panel that both the master switch and the backup switch have been created, and the master switch is marked with an AE mark.

The producer sends a message, after being matched by the main switch but without a suitable queue, it is forwarded to the backup switch and routed to its queue for storage.

 Note: It is recommended to use a Fanout switch. If other switches such as Direct, when the master switch forwards to the backup switch, when matching, if the given RoutingKey of the message does not match the corresponding queue, the message will be lost. In this way, initially predictions have deviated.

Persistence

When RabbitMQ is under abnormal conditions, such as system downtime, restart, shutdown, etc., it may cause data loss and reduce reliability. In response to this situation, RabbitMQ provides a persistence mechanism that saves the message itself and metadata (queues, switches, binding information) to disk. Specifically divided into three types of persistence

  • switch persistence

  • queue persistence

  • message persistence

switch persistence

When RabbitMQ encounters an abnormal situation (such as a service restart), if the switch persistence is not set, the data related to the switch will be lost, and the producer will fail when sending a message to the specified switch.

Service restart exception

1. Create a new switch in the Web and specify the non-persistent mode.

 2. Create a new queue and specify the non-persistent mode.

 3. Set the binding relationship between the switch and the queue.

 4. The front part of the producer sends messages normally. After the service is restarted in the middle, the switch, queue, and binding relationship are all cleared, and the producer continues to send messages, and an exception occurs.

  

Persistence settings

You can specify the durable parameter to be set to true when declaring the switch (it can also be set in the Web panel).

channel.ExchangeDeclare(exchange: exchangeName, type: "fanout", durable: true, arguments: null);

The RabbitMQ service restarts, and the producer continues to send messages to the switch.

 

queue persistence

The persistence of the queue is to set the durable parameter to true when the queue is declared. If the queue is not durable, after an abnormal situation (such as a service restart), the metadata of the queue will be lost, and the messages stored in it will also be lost.

Service restart exception

1. Create a switch in the Web and set it to persistent mode.

2. Create a queue and set it to non-persistent mode

3. Set the binding relationship between the switch and the queue.

4. The front part of the producer sends messages normally. After the service restarts in the middle, the queue and binding relationship are cleared, and the production continues to send messages. If the matching queue fails, the message is returned to the producer.

Persistence settings

When declaring the queue, you can specify the durable parameter to be set to true (it can also be set in the Web panel).

channel.QueueDeclare(queue: queueName, durable: true, exclusive: false, autoDelete: false, arguments: null);

The RabbitMQ service restarts, and the producer continues to send messages to the switch.

message persistence

The persistence of the queue can only guarantee that its own data will not be lost, but the messages stored in it cannot be guaranteed not to be lost.

Persistence settings

Persistence needs to be set for the message to ensure that the message itself will not be lost due to abnormal conditions (such as service restart). When sending a message, you can set the basic properties of the message to support the persistence of the message.

var basicProperties = channel.CreateBasicProperties();
basicProperties.DeliveryMode = 2;// 1非持久化 2持久化
channel.BasicPublish(exchange: exchangeName, routingKey: "", mandatory: true, basicProperties: basicProperties, body: body);

In this way, when there is an abnormal situation (such as after the service is restarted), the message still exists.

 Note: Message persistence will affect performance, and only ensure valuable message persistence to balance reliability and throughput.

Guess you like

Origin blog.csdn.net/qq_41872328/article/details/126785876