2.1 Defects of the old copy function
2.2 Implementation of the new version of the copy function
The replication function of redis is divided into two operations: synchronization and command propagation. The synchronization operation is used to update the database status of the slave server to the database status of the main server. The command propagation operation is used when the database status of the master server is modified, which causes the database status of the master and slave servers to be inconsistent, so that the databases of the master and slave servers return to a consistent state. The replication function of redis is mainly realized by executing the SLAVEEOF command.
1. Synchronization
The synchronization operation of the slave server to the master server needs to be completed by sending a SYNC command to the master server. The following are the execution steps of SYNC:
- Send the SYNC command from the server to the master server;
- The main server that receives the SYNC command executes the BGSAVE command, generates an RDB file in the background, and uses a buffer to record all write commands executed from now on;
- When the BGSAVE command is executed, the master server sends the RDB file to the slave server, receives the RDB file from the server, and updates its database status to the database status when the master server executes the BGSAVE command;
- The master server sends all write commands recorded in the buffer to the slave server, and the slave server executes these write commands to update its own database status to the current state of the master server database;
2. Command propagation
2.1 Defects of the old copy function
The replication of the slave server to the master server before Redis2.8 can be divided into two situations:
- First replication: The slave server has never replicated any server before, this is the first replication
- Replication after disconnection: The master-slave server in the command propagation stage interrupted replication due to network reasons, but the slave server automatically reconnects to the master server and continues to replicate the master server.
For the first copy, the old version of the copy function can complete the task very well, but for the re-copy after a disconnection, the old version of the copy function is very inefficient. Because the old version of the replication after the disconnection from the server will send a SYNC command to the master server, and the master server will generate RDB files for all key-value pairs in the database and send them to the slave server. This approach allows the main server to re-execute the SYNC command, which is very inefficient.
2.2 Implementation of the new version of the copy function
In order to solve the problem of inefficiency of replication when the old version of the function is disconnected, Redis starts from version 2.8 and uses the PSYNC command instead of the SYNC command to perform the synchronization operation during replication. The complete resynchronization of PSYNC is used to handle the initial copy situation, and the complete resynchronization is the same as the initial copy situation of the SYNC command. And the portion re-synchronization is used for the replication of the treated disconnection, reconnection time after disconnection from the master server connection is disconnected during the write command sent from the server, from the server only receives these commands, can be The database is updated to the current state of the main server. The partial resynchronization function of PSYNC consists of the following three parts:
(1) Copy offset:
The two parties of the master-slave replication will maintain a replication offset. When the master server copies N bytes, it adds its own replication offset to N. Each time the slave server receives N bytes, it will Add N to your copy offset. By comparing the replication offset of the master and slave servers, the program can easily determine whether the master and slave servers are in a consistent state:
- If the master and slave servers are in a consistent state, the offset of the master and slave servers is always the same;
- On the contrary, if the offsets of the master and slave servers are different, then the master and slave servers are not in a consistent state
(2) Copy the backlog buffer:
The replication backlog buffer is a fixed-length first-in-first-out queue maintained by the master server. The default size is 1M, or it can be set according to the repl-backlog-size in the configuration file . When the main server performs command propagation, it will not only send the write command to all heavy servers, but also queue the write command to the copy backlog buffer, and the copy backlog buffer will record the corresponding copy for each byte in the queue Offset. When the slave server reconnects to the master server, the slave server will send its own replication offset offset to the master server through the PSYNC command, and the master server will decide which synchronization operation to perform on the slave server according to the replication offset:
- If the data after the offset is still in the copy backlog buffer, the master server will perform a partial resynchronization operation on the slave server
- If the data after the offset is no longer in the replication backlog buffer, the master server will perform a complete resynchronization operation on the slave server.
(3) Server running ID:
In addition to the copy offset and the copy backlog buffer, the server's running ID is also used to achieve partial resynchronization. Each Redis server, whether the main server or the heavy server, will have its own running ID. When the first replication is performed, The master server will send its own running ID to the slave server, and the slave server will save this ID. When the slave server disconnects and reconnects to the master server, the slave server will send the previously saved running ID to the currently connected master server:
- After the main server receives this ID, if it is the same as its own ID, it will try to perform partial resynchronization;
- If the received ID is different from its own running ID, the main server performs a complete resynchronization operation.