1. Comparison of Redis cluster solutions
sentinel mode
In versions prior to redis3.0, to implement clusters, the sentinel tool is generally used to monitor the status of the master node. If the master node is abnormal
Usually, it will be a master-slave switch, and a certain slave will be used as the master. The configuration of the sentinel is slightly complicated, and the performance and high availability are excellent.
In general, especially at the moment of master-slave switching, there is a situation of
instantaneous access interruption
, and only one master node provides external services in sentinel mode, which cannot be supported
High concurrency, and the memory of a single master node should not be set too large, otherwise the persistent file will be too large, affecting data recovery or master-slave synchronization
efficiency
High availability cluster mode
Redis cluster is a distributed server group composed of multiple
master-slave node groups
, which has the characteristics of
replication, high availability and fragmentation
. Redis cluster does not need
Sentinel Sentinels can also complete node removal and failover functions. Each node needs to be set to cluster mode, which is not in the cluster mode
The core node can be expanded horizontally. According to the official
document it can be linearly expanded to tens of thousands of nodes (
the official recommendation is no more than 1,000 nodes
). redis cluster
The performance and high availability are better than the previous version of Sentinel mode, and the cluster configuration is very simple
2. Redis high availability cluster construction
Redis cluster construction
The redis cluster needs
at least three master nodes
. Here we build three master nodes and build a slave node for each master.
Point, there are 6 redis nodes in total. Here, 6 redis instances are deployed on three machines. Each machine has one master and one slave. The steps to build a cluster are as follows:
第一步：在第一台机器的/usr/local下创建文件夹redis‐cluster，然后在其下面分别创建2个文件夾如下
（1）mkdir ‐p /usr/local/redis‐cluster
（2）mkdir 8001 8004
第一步：把之前的redis.conf配置文件copy到8001下，修改如下内容：
（1）daemonize yes
（2）port 8001（分别对每个机器的端口号进行设置）
（3）pidfile /var/run/redis_8001.pid # 把pid进程号写入pidfile配置的文件
（4）dir /usr/local/redis‐cluster/8001/（指定数据文件存放位置，必须要指定不同的目录位置，不然会丢失数据）
（5）cluster‐enabled yes（启动集群模式）
（6）cluster‐config‐file nodes‐8001.conf（集群节点信息文件，这里800x最好和port对应上）
（7）cluster‐node‐timeout 10000
(8)# bind 127.0.0.1（bind绑定的是自己机器网卡的ip，如果有多块网卡可以配多个ip，代表允许客户端通
过机器的哪些网卡ip去访问，内网一般可以不配置bind，注释掉即可）
(9)protected‐mode no （关闭保护模式）
(10)appendonly yes
如果要设置密码需要增加如下配置：
(11)requirepass yanqiuxiang(设置redis访问密码)
(12)masterauth yanqiuxiang(设置集群节点间访问密码，跟上面一致)
第三步：把修改后的配置文件，copy到8004，修改第2、3、4、6项里的端口号，可以用批量替换：
:%s/源字符串/目的字符串/g
第四步：另外两台机器也需要做上面几步操作，第二台机器用8002和8005，第三台机器用8003和8006
第五步：分别启动6个redis实例，然后检查是否启动成功
（1）/usr/local/redis‐5.0.3/src/redis‐server /usr/local/redis‐cluster/800*/redis.conf
（2）ps ‐ef | grep redis 查看是否启动成功
第六步：用redis‐cli创建整个redis集群(redis5以前的版本集群是依靠ruby脚本redis‐trib.rb实现)
# 下面命令里的1代表为每个创建的主服务器节点创建一个从服务器节点
# 执行这条命令需要确认三台机器之间的redis实例要能相互访问，可以先简单把所有机器防火墙关掉，如果不
关闭防火墙则需要打开redis服务端口和集群节点gossip通信端口16379(默认是在redis端口号上加1W)
# 关闭防火墙
# systemctl stop firewalld # 临时关闭防火墙
# systemctl disable firewalld # 禁止开机启动
# 注意：下面这条创建集群的命令大家不要直接复制，里面的空格编码可能有问题导致创建集群不成功
（1）/usr/local/redis‐5.0.3/src/redis‐cli ‐a yanqiuxiang‐‐cluster create ‐‐cluster‐replicas 1 192.168.30.130:8001 192.168.30.131:8002 192.168.30.132:8003 192.168.30.132:8004 192.168.0.62:8005 192.168.30.133:8006
第七步：验证集群：
（1）连接任意一个客户端即可：./redis‐cli ‐c ‐h ‐p (‐a访问服务端密码，‐c表示集群模式，指定ip地址
和端口号）
如：/usr/local/redis‐5.0.3/src/redis‐cli ‐a zhuge ‐c ‐h 192.168.30.130 ‐p 800*
（2）进行验证： cluster info（查看集群信息）、cluster nodes（查看节点列表）
（3）进行数据操作验证
（4）关闭集群则需要逐个进行关闭，使用命令：
/usr/local/redis‐5.0.3/src/redis‐cli ‐a yanqiuxiang‐c ‐h 192.168.30.130 ‐p 800* shutdown
3. Analysis of the principle of Redis cluster
Redis Cluster divides all data into 16384 slots (slots), and each node is responsible for a part of the slots. Slot information is stored in each
in a node.
When the client of Redis Cluster connects to the cluster, it will also get a copy of the slot configuration information of the cluster and cache it locally on the client. this
In this way, when the client wants to find a certain key, it can directly locate the target node. At the same time, because the slot information may exist differently between the client and the server
In the case of consistency, a correction mechanism is needed to realize the verification and adjustment of the slot information.
slot location algorithm
By default, Cluster will use the crc16 algorithm to hash the key value to obtain an integer value, and then use this integer value to modulo 16384
to get specific slots.
HASH_SLOT = CRC16(key) mod 16384
jump relocation
When the client sends an instruction to a wrong node, the node will find that the slot where the key of the instruction is located is not managed by itself.
The client sends a special jump command carrying the node address of the target operation, telling the client to connect to this node to obtain data. The client receives the
After the command, in addition to jumping to the correct node for operation, it will also update and correct the local slot mapping table cache synchronously, and all subsequent keys will use new slots
Bitmap.
Communication mechanism between Redis cluster nodes
Redis cluster nodes use gossip protocol to communicate
There are two ways to maintain cluster metadata (cluster node information, master-slave roles, number of nodes, data shared by each node, etc.): centralized
formula and gossip
centralized:
The advantage lies in the update and reading of metadata, and the timeliness is very good. Once the metadata changes, it will be updated to the centralized storage immediately.
It can be perceived immediately when the point is read; the shortcoming is that all metadata update pressures are concentrated in one place, which may cause metadata
Data storage pressure. Many middleware use zookeeper to store metadata in a centralized manner.
gossip:
gossip protocol contains a variety of messages, including ping, pong, meet, fail and so on.
meet
: A node sends meet to the newly joined node, let the new node join the cluster, and then the new node will start to communicate with other nodes
letter;
ping
: Each node will frequently send pings to other nodes, which contain its own status and cluster metadata maintained by itself, and pass through each other
Ping exchange metadata (similar to the addition and removal of cluster nodes perceived by oneself, hash slot information, etc.);
pong
: The return of ping and meet messages, including its own status and other information, can also be used for information broadcast and update;
fail
: After a node judges that another node fails, it sends fail to other nodes to notify other nodes that the specified node is down.
The advantage of the gossip protocol is that the update of metadata is relatively scattered, instead of being concentrated in one place, update requests will be sent to all nodes one after another
To update, there is a certain delay, which reduces the pressure; the disadvantage is that the delay in metadata update may cause some operations of the cluster to lag behind.
Port 10000 for gossip communication
Each node has a port dedicated to gossip communication between nodes, which is the port number +10000 for its own service, such as 7001, then
Port 17001 is used for communication between nodes. Each node sends ping messages to several other nodes every once in a while, and the other several nodes
The point returns a pong message after receiving the ping message.