Table of contents
2. Redis master-slave replication
1. What is master-slave replication?
2. The role of master-slave replication
3. Workflow principle of master-slave replication
3.2. Data synchronization stage
3.3. Command propagation stage
3. Frequently Asked Questions about Master-Slave Replication
2. Frequent network interruptions
1. What is Redis
Redis is one of the most popular NoSQL databases today. Redis is an open source, written in ANSI C, contains a variety of data structures, supports network, memory-based, optional persistence key-value pair storage database. It has the following characteristics:
- Runs based on memory and has efficient performance
- Supports distribution and can theoretically be expanded infinitely
- key-value storage system
- The open source is written in ANSI C language, complies with the BSD protocol, supports the network, can be memory-based and persistent log-type, Key-Value database, and provides APIs in multiple languages.
OK! Let’s briefly talk about what Redis is, which is NoSQL for storing data. It is generally used for caching. This article is also a further understanding of the underlying layer of Redis based on Redis!
2. Redis master-slave replication
Simple drawing for easy understanding
Multi-server setup: A master server only writes data and synchronizes it to other slave servers. The problem is data synchronization . So we use master-slave replication to solve the problem of data synchronization.
1. What is master-slave replication?
Master-slave replication is to copy the master's data to the slave immediately and effectively . This is called master-slave replication!
Features: A master can have multiple slaves, and one slave only corresponds to one master.
Responsibilities:
- master
- write data
- When performing a write operation, the changed data is automatically synchronized to the slave.
- Read data (usually only write operations are performed, and reading is done by the slave server)
- salve
- Read data
- Forbidden to write
2. The role of master-slave replication
Separation of reading and writing: master writes, slave reads, improving the server's read and write load capacity
Load balancing: Based on the master-slave structure, combined with read-write separation, the slave shares the master load, and changes the number of slaves according to changes in demand, and shares the data reading load through multiple slave nodes, greatly improving the concurrency and throughput of the Redis server. .
Fault recovery: When a problem occurs on the master, the slave provides services to achieve rapid fault recovery.
Data redundancy: Implementing hot data backup is a data redundancy method in addition to persistence.
High-availability foundation: Based on master-slave replication, build sentinel mode and cluster to implement Redis high-availability solution.
3. Workflow principle of master-slave replication
Let me give you an overview first. I divide the master-slave replication principle into two stages. The working principle is divided into: connection establishment stage , data synchronization stage, and command propagation stage .
3.1. Link establishment stage
The slave server sends the master 's IP address and port number , and the master receives and responds back. After receiving the response, the slave saves the master's IP and port number, establishes a stocket channel based on the saved information, and starts periodic ping, and the master will also respond with pong. Finally, the slave sends its own port number to the master, and the master saves the slave's port number.
Step 1: The slave sets the address and port of the master and saves the master information.
Step 2: slave establishes socket connection
Step 3: slave sends ping command (timer task)
Step 4: Authentication (because redis is used internally by the server and is not provided to the outside world, it is not set)
Step 5: Master saves slave port information
3.2. Data synchronization stage
When the master server turns on AOF or becomes the master server, a replication buffer will be created.
① When the slave sends a psync2 request for the first time, it will carry two data psync2 ? -1(psync2 <runid> <offset>), which means that it needs to be copied in full for the first time. ②The master executes bgsave to generate an RDB file and records the current copy offset offset. ③When receiving the instruction, it will send +fullresync its own uid offset. When sending the RDB file to the slave through the stocket, the client command is accepted, and the offset changes. ④The slave receives the fullresync, saves the master's uid and offset (offset), clears all current data, accepts the RDB file through the stocket, and restores the RDB data.
The above is a full copy
⑤Send command: psync2 masterid offset. ⑥Accept the command, determine whether the id matches, and determine whether the offset is in the copy buffer.
⑦ If one of the masterid or offset is not satisfied, a full copy will be made. If the ones sent are the same as those in the master, they will be omitted. If the masterid or offset is satisfied, but the offset is inconsistent with the offset saved in the master, continue offset will be sent, and passed The socket sends the data from the offset in the master in the copy buffer to the transferred master offset. ⑧The slave receives +concatiue and saves the master's offset. After receiving the message, execute bgrewriteaof to restore the data.
Incremental copy above
3.3. Command propagation stage
Send the replconf ack offset command during the command propagation phase
3. Frequently Asked Questions about Master-Slave Replication
1. Frequent full copy
Problem symptoms:
The network environment is not good, there is a network interruption, and the slave does not provide services.
problem causes:
The copy buffer is too small. After the network is disconnected, the offset of the slave goes out of bounds, triggering full copy.
Final Results:
The slave repeatedly performs full replication
solve:
Modify copy buffer size
repl-backlog-size2. Frequent network interruptions
Phenomenon:
The connection between salve and master is disconnected
reason:
The master sends ping commands less frequently
The master sets a short timeout
The ping command causes packet loss in the network
solve:
Increase the frequency of sending ping commands
repl-ping-slave-periodThe timeout repl-time should be at least 5-10 times the frequency of the ping command, otherwise the slave will easily timeout.
3. Data is inconsistent
Phenomenon:
Multiple slaves obtain the same data without synchronization
reason:
Network information is out of sync and there is a delay in data sending.
plan:
1. Optimize the network environment of master and slave, usually deployed in the same computer room, such as Alibaba Cloud, etc.
2. Monitor the delay in the master-slave stage (via offset). If the slave delay is too large, temporarily block the program's data access to the slave.
slave-serve-stale-data yes|noAfter opening, it only responds to a few commands such as info and slaveof (use with caution unless data consistency requirements are high)