使用keepalived实现Redis双机热备、高可用

Redis高可用

需求

(1)Redis运行在docker容器中;（2）实现双机热备,当redis-master异常时，redis-slave快速切换成master提供服务。
(3)当redis-master数据更新时，redis-slave数据能同步更新。

解决方案

利用keepalived实现redis的高可用。keepalived利用shell脚本，定期检测redis服务是否正常。当redis服务异常时，利用虚拟IP的漂移实现故障切换。

部署实现

1.环境说明

对象	版本	备注
redis-master主机	centos 7	IP:XX.XX.93.179
redis-slave主机	centos 7	IP:XX.XX.93.182

keepalived软件版本：1.2.13

天擎6的skylar-redis镜像包版本：1600

虚拟IP地址：XX.XX.93.180。注意：要选择没有被占用的虚拟IP地址

2.安装

（1）在master、slave主机上，都需安装keepalived、ipvsadm

$ yum -y install ipvsadm keepalived

查看keepalived是否安装成功：

$ keepalived -v
Keepalived v1.2.13 (03/06,2015)

(2)在master、slave主机上，都需启动skylar-redis容器

$ docker run --name=redis --restart=always -d --net=host --privileged=true -v /data/redis:/var/lib/redis/ skylar_redis

（3）配置keepalived，添加shell脚本。

master、slave主机的/etc/keepavlied/keepavlied.conf都需要修改，内容略微不同。
实际安装时，需要将此配置文件中的interface、virtual_ipaddress、master_ip、slave_ip进行修改

在master主机上，keepalived.conf内容如下：

$ cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   router_id redis100
}
vrrp_script chk_redis
{
     script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
     interval 2
     timeout 2
     fall 3
}
vrrp_instance redis {
     state MASTER # master set to SLAVE also
     interface eno16777984  ## 修改为实际网卡名称
     virtual_router_id 50
     priority  150
     nopreempt # no seize,must add
     advert_int 1
authentication {   #all node must same
#         auth_type PASS
#         auth_pass 1111
    }
    virtual_ipaddress {  
        XX.XX.93.180   ## 修改为实际分配的虚拟IP地址
    }
    track_script {
         chk_redis
    }
     notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 10.16.93.182 6379"       ## 10.16.93.182应修改为实际的slave IP地址
     notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 10.16.93.182 6379"       ## 10.16.93.182应修改为实际的slave IP地址
     notify_fault /etc/keepalived/scripts/redis_fault.sh
     notify_stop /etc/keepalived/scripts/redis_stop.sh
}

在slave主机上，keepalived.conf内容如下：

$ cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {
   router_id redis101
}
vrrp_script chk_redis
{
     script "/etc/keepalived/scripts/redis_check.sh 127.0.0.1 6379"
     interval 2
     timeout 2
     fall 3
}
vrrp_instance redis {
    state BACKUP
    interface eno16777984   ##需要修改为实际网卡名称
    virtual_router_id 50  
    priority  100
    advert_int 1
authentication {   #all node must same
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        XX.XX.93.180     ##需要修改为实际分配的虚拟IP
    }
    track_script {
         chk_redis
    }
    notify_master "/etc/keepalived/scripts/redis_master.sh 127.0.0.1 XX.XX.93.179 6379"     ## XX.XX.93.179需要修改为实际的master主机IP
    notify_backup "/etc/keepalived/scripts/redis_backup.sh 127.0.0.1 XX.XX.93.179 6379"     ## XX.XX.93.179需要修改为实际的master主机IP
    notify_fault /etc/keepalived/scripts/redis_fault.sh
    notify_stop /etc/keepalived/scripts/redis_stop.sh
}

在master和slave主机上，添加/etc/keepalived/scripts目录和redis_master.sh、redis_backup.sh、redis_fault.sh、redis_stop.sh文件。最终目录结构如下：

$ pwd
/etc/keepalived

$ tree
.
├── keepalived.conf
└── scripts
    ├── redis_backup.sh
    ├── redis_check.sh
    ├── redis_fault.sh
    ├── redis_master.sh
    └── redis_stop.sh


1 directory, 6 files

redis_master.sh、redis_backup.sh、redis_fault.sh、redis_stop.sh文件在github中可以下载，下载链接：
https://github.com/qinguanri/Redis_HA

下面简单列出几个shell文件的内容：

$ cat redis_master.sh 

#!/bin/bash
REDISCLI="docker exec redis /usr/bin/redis-cli -h $1 -p $3" 
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[master]" >> $LOGFILE
date >> $LOGFILE
echo "Being master...." >> $LOGFILE
echo "Run MASTER cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE
sleep 10 #delay 10 s wait data async cancel sync
echo "Run SLAVEOF NO ONE cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF NO ONE >> $LOGFILE


$ cat redis_backup.sh 

#!/bin/bash
REDISCLI="docker exec redis /usr/bin/redis-cli" 
LOGFILE="/var/log/keepalived-redis-state.log"
echo "[backup]" >> $LOGFILE
date >> $LOGFILE
echo "Run SLAVEOF cmd ..." >> $LOGFILE
$REDISCLI SLAVEOF $2 $3 >> $LOGFILE 2>&1
sleep 15 #delay 15 s wait data sync exchange role


$ cat redis_check.sh 

#!/bin/bash
ALIVE=`docker exec redis /usr/bin/redis-cli -h $1 -p $2 PING`
LOGFILE="/var/log/keepalived-redis-check.log"
echo "[CHECK]" >> $LOGFILE
date >> $LOGFILE
if [ "$ALIVE" == "PONG" ]; then :
    echo "Success: redis-cli -h $1 -p $2 PING $ALIVE" >> $LOGFILE 2>&1
    exit 0
else
    echo "Failed: redis-cli -h $1 -p $2 PING $ALIVE" >> $LOGFILE 2>&1
    exit 1
fi

3.启动keepalived
在master主机、slave主机上执行以下命令启动keepalived

$ systemctl start keepalived.service

4.验证\模拟故障