1. What is high availability?
The amount of concurrency supported by single-machine redis is limited (generally, QPS ranges from 1W to several W). If the amount of concurrency is large, redis may hang up; if the redis configuration is unreasonable, the synchronization of fork sub-processes may also cause redis to hang up (pseudo-hang); operation System exceptions (JVM hangs, OOM, CPU overload, disk full of IOYICHANG) or network changes, migrations, etc. may also cause redis to hang (suspend external services). These, etc., may cause the normal operation of the business system to be non-highly available. Ideally, the suspension of background redis does not affect (or affects to a lesser extent) the access of the business system to the cache system, that is, high availability. High availability is embodied in: failover (failover), also known as active-standby switching.
2. Pseudo high availability
Use the redis persistence scheme (RDB, AOF) to restart and load the cached data after the process hangs.
3. Data synchronization mechanism
Generally, the pressure of redis reading is high. In order to provide better services, master-slave (one master and multiple slaves) replication (synchronization) is supported. According to whether full replication is performed, it is divided into: full synchronization and incremental synchronization.
Full sync
a. Redis full replication generally occurs in the Slave initialization phase. At this time, the Slave needs to copy all the data on the Master.
b. After the master-slave is configured, the slave will send the PSYNC command to the master regardless of whether the slave is connected to the master for the first time or reconnected.
If it is a reconnection and the conditions for incremental synchronization are met, then redis will send the commands in the memory cache queue to the slave to complete the incremental synchronization. Otherwise, perform full synchronization.
The specific steps are as follows:
1) The slave server connects to the master server and sends the SYNC command;
2) After the master server receives the SYNC name, it starts to execute the BGSAVE command to generate the RDB file and uses the buffer to record all the write commands executed thereafter;
3) The master server BGSAVE After execution, send the snapshot file to all slave servers, and continue to record the executed write commands during the sending period;
4) After receiving the snapshot file from the server, discard all old data and load the received snapshot (the snapshot file is received and saved first.
5) After the master server snapshot is sent, it starts to send the write command in the buffer to the slave server; 6) The slave server completes the loading
of the snapshot, starts to receive the command request, and executes the command from the master server. write command to the server buffer;
Incremental synchronization
Redis incremental replication refers to the process of synchronizing the write operations from the master server to the slave server when the slave starts to work normally after initialization. The process of incremental replication is mainly that every time the master server executes a write command, it sends the same write command to the slave server, and the slave server receives and executes the received write command.
Conditions for incremental synchronization:
Several important concepts:
- In-memory backlog: used to record write operations received by the master when the connection is disconnected
- Replication offset: Both master and slave have an offset to record the position of the current synchronization record
- master server id (master run ID): the unique identifier of the master
After the current network connection is disconnected, the slave will try to reconnect to the master. When the following conditions are met, incremental synchronization (resuming from a breakpoint) will be performed after reconnection:
1. The master server id recorded by the slave is the same as the master server id currently connected to
2. The replication offset of the slave is ahead of the offset of the master. For example, slave is 1000, master is 1100
3. The data specified by the replica offset of the slave is still stored in the memory cache queue of the master server
Redis master-slave synchronization strategy
主从刚刚连接的时候，进行全量同步；全同步结束后，进行增量同步。当然，如果有需要，slave 在任何时候都可以发起全量同步。redis 策略是，无论如何，首先会尝试进行增量同步，如不成功，要求从机进行全量同步。
redis replication的核心机制
（1）redis采用异步方式复制数据到slave节点，不过redis 2.8开始，slave node会周期性地确认自己每次复制的数据量
（2）一个master node是可以配置多个slave node的
（3）slave node也可以连接其他的slave node
（4）slave node做复制的时候，是不会block master node的正常工作的
（5）slave node在做复制的时候，也不会block对自己的查询操作，它会用旧的数据集来提供服务; 但是复制完成的时候，需要删除旧数据集，加载新数据集，这个时候就会暂停对外服务了
（6）slave node主要用来进行横向扩容，做读写分离，扩容的slave node可以提高读的吞吐量，与高可用有关
（1）master和slave都会维护一个offset
master会在自身不断累加offset，slave也会在自身不断累加offset
slave每秒都会上报自己的offset给master，同时master也会保存每个slave的offset
这个倒不是说特定就用在全量复制的，主要是master和slave都要知道各自的数据的offset，才能知道互相之间的数据不一致的情况
（2）backlog
master node有一个backlog，默认是1MB大小
master node给slave node复制数据时，也会将数据在backlog中同步写一份
backlog主要是用来做全量复制中断候的增量复制的
（3）master run id
info server，可以看到master run id
如果根据host+ip定位master node，是不靠谱的，如果master node重启或者数据出现了变化，那么slave node应该根据不同的run id区分，run id不同就做全量复制
如果需要不更改run id重启redis，可以使用redis-cli debug reload命令