## 1.需求
实现数据交换系统的主备自动切换功能，当数据交换的master主机故障,网络故障以及数据交换mysql服务出现故障时，能自动切换到备机运行。
## 2.实现思路
采用keepalived+rsync+ inotify双backup模式，并设置不抢占资源。当master挂机，切换为backup,完成修复后，不会抢占为maste!避免不必要的切换。 假设两台主机如下。
• master host：192.168.1.151，debain
• backup host：192.168.1.152，debain
安装步骤如下：
1. 安装apt-get install keepalived
2. 安装apt-get install rsync
3. 安装apt-get install inotify-tools
4. 启动
## 3.安装keeplived
```
apt-get install keeplived
```
#### 修改master主机上的/etc/keepalived/keepalived.conf配置文件
```
cat /etc/keepalived/keepalived.conf
```
```
! Configuration File for keepalived
global_defs {
router_id xchange_router # all node must same
}
vrrp_instance xchange {
state BACKUP # master or backup
interface eth1 # 热备通信网口
virtual_router_id 55 # all node must same
priority 100 # 优先级，高的先运行。主机异常时切换到低优先级热备机，副机一般设为99
advert_int 1 # send keepalived msg every 1 second
nopreempt # must need
authentication { # all node must same
auth_type PASS
auth_pass 1111
}
track_interface { #需要检测的其它
eth0 #在些添加的网络接口会被检测，eth0不正常会切换热备服务
}
notify_master /etc/keepalived/to_master.sh #切换为master时执行
notify_backup /etc/keepalived/to_backup.sh #切换为backup时执行
}
virtual_server 127.0.0.1 3306 { #表示需要检测的服务
delay_loop 2
lb_algo rr
lb_kind NAT
persistence_timeout 60
protocol TCP
real_server 127.0.0.1 3306 {
weight 3
notify_down /etc/keepalived/shutdown.sh #服务检测失败执行操作
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
```
#### 修改backup主机上的/etc/keepalived/keepalived.conf
```
! Configuration File for keepalived
global_defs {
router_id xchange_router # all node must same
}
vrrp_instance xchange {
state BACKUP # master or backup
interface eth1 # 热备通信网口
virtual_router_id 55 # all node must same
priority 99 # 优先级，高的先运行。主机异常时切换到低优先级热备机，副机一般设为99
advert_int 1 # send keepalived msg every 1 second
#nopreempt # must need
authentication { # all node must same
auth_type PASS
auth_pass 1111
}
track_interface { #需要检测的其它
eth0 #在些添加的网络接口会被检测，eth0不正常会切换热备服务
}
notify_master /etc/keepalived/to_master.sh #切换为master时执行
notify_backup /etc/keepalived/to_backup.sh #切换为backup时执行
}
virtual_server 127.0.0.1 3306 { #表示需要检测的服务
delay_loop 2
lb_algo rr
lb_kind NAT
persistence_timeout 60
protocol TCP
real_server 127.0.0.1 3306 {
weight 3
notify_down /etc/keepalived/shutdown.sh #服务检测失败执行操作
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
```
#### 创建shell脚本，放在/etc/keepalived目录下
> 脚本1：shutdown.sh
```
#!/bin/bash
echo "shutdown.sh" >> /var/log/messages
killall keepalived
/etc/init.d/manager stop
```
>脚本2：to_master.sh
```
#!/bin/bash
echo "to_master" >> /var/log/messages
/etc/init.d/manager stop
/etc/init.d/manager start
```
>脚本3:to_backup.sh
```
#!/bin/bash
echo "to_backup" >> /var/log/messages
/etc/init.d/manager stop
```
>脚本4:keepalived_sync.sh,
```
#!/bin/bash
# usage: sh keepalived_sync.sh
USER=appbackup # the account to login rsyc server. Notice, it's not the linux user
IP=192.168.1.152 # the rsync server's ip.主备对应修改，192.168.1.151/192.168.1.152
DES="xchange"
SRC="/usr/app/xchange/repository"
PASSWD_FILE="/etc/rsync_client.secret"
LOGFILE="/var/log/messages.log"
/usr/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f%e' -e close_write,modify,delete,create,attrib,move ${src} | while read file
do
/usr/bin/rsync -vzrtopg --delete --progress --password-file=$PASSWD_FILE $SRC $USER@$IP::$DES
echo "${files} was rsynced" >>$LOGFILE 2>&1
done
```
#### keepalived的相关配置文件存储位置如下：
```
root@deb:/etc/keepalived$ tree /etc/keepalived/
/etc/keepalived/
├── keepalived.conf
├── to_backup.sh
├── shutdown.sh
├── to_master.sh
└── keepalived_sync.sh
1 directory, 5 files
```
## 安装Rsync
```
apt-get install rsync
```
#### 修改/etc/default/rsync文件
```
RSYNC_ENABLE=true
```
#### 配置rsyncd的/etc/rsyncd.conf
```
cat /etc/rsyncd.conf
```
```
# sample rsyncd.conf configuration file
# GLOBAL OPTIONS
#motd file=/etc/motd
log file=/var/log/messages
# for pid file, do not use /var/run/rsync.pid if
# you are going to run rsync out of the init.d script.
# The init.d script does its own pid file handling,
# so omit the "pid file" line completely in that case.
# pid file=/var/run/rsyncd.pid
syslog facility=daemon
#socket options=
# MODULE OPTIONS
[xchange]
comment = public archive
path = /usr/app/xchange/repository
use chroot = yes
# max connections=10
lock file = /var/lock/rsyncd
# the default for read only is yes...
read only = no
list = yes
uid = root
gid = root
# exclude =
# exclude from =
# include =
# include from =
auth users =appbackup
secrets file = /etc/rsyncd.secrets
strict modes = yes
# hosts allow =
# hosts deny =
ignore errors = no
ignore nonreadable = yes
transfer logging = no
# log format = %t: host %h (%a) %o %f (%l bytes). Total %b bytes.
timeout = 600
refuse options = checksum dry-run
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz
```
#### 修改/etc/rsync.secret
```
sudo cat /etc/rsync.secret
```
```
appbackup:passwd
```
> 修改/etc/rsync.secret文件属性:
```
chown root:root /etc/rsync.secret
chmod 600 /etc/rsync.secret
```
#### 启动rsync服务器
```
service rsync restart
```
#### 配置rsync客户端,
> 文件/etc/rsync_client.secret内容如下
```
passwd
```
> 修改文件权限
```
sudo chmod 600 /etc/rsync_client.secret
```
** 注意：如果不修改权限，则会报错ERROR: password file must not be other-accessible**
## 安装 inotify-tools
```
apt-get install inotify-tools
```
## 5.启动
#### 1. 启动master主机上的rsync
```
service rsync restart
```
#### 2. 启动master主机上的keepalived
```
service keepalived restart
```
#### 5. 查看启动日志
```
tail -f /var/log/messages.log
```
## 6.自动切换测试
#### 确认keepalived、rsync已经启动
```
ps -ef | grep keepalived
ps -ef | grep rsync
```
## 7.加入开机脚本
#echo "/data/sh/inotifyrsync.sh &" >> /etc/rc.local
## 8.manager去掉开机启动
#### 切换测试
> 1.停止master主机上的mysql进程，查看master主机上的manager服务是否停止，检查backup主机是否启动manager服务。
> 2.或者关机master，查backup主机是否启动manager服务。
执行流程说明：
默认情况下主副机器都不会运行manager服务，主副机启动keepalived服务，首行执行优先级高的主机to_master脚本，启动manager服务,如果出现mysql停止或宕机情况，会执行shutdown.sh停止热备和manager服务，此时主副切换，副机执行to_master脚本,过程中如果主机修复，主机会抢占热备服务，副机执行to_backup脚本并停止manager服务，主机重新执行to_master并开启manager
实现数据交换系统的主备自动切换功能，当数据交换的master主机故障,网络故障以及数据交换mysql服务出现故障时，能自动切换到备机运行。
## 2.实现思路
采用keepalived+rsync+ inotify双backup模式，并设置不抢占资源。当master挂机，切换为backup,完成修复后，不会抢占为maste!避免不必要的切换。 假设两台主机如下。
• master host：192.168.1.151，debain
• backup host：192.168.1.152，debain
安装步骤如下：
1. 安装apt-get install keepalived
2. 安装apt-get install rsync
3. 安装apt-get install inotify-tools
4. 启动
## 3.安装keeplived
```
apt-get install keeplived
```
#### 修改master主机上的/etc/keepalived/keepalived.conf配置文件
```
cat /etc/keepalived/keepalived.conf
```
```
! Configuration File for keepalived
global_defs {
router_id xchange_router # all node must same
}
vrrp_instance xchange {
state BACKUP # master or backup
interface eth1 # 热备通信网口
virtual_router_id 55 # all node must same
priority 100 # 优先级，高的先运行。主机异常时切换到低优先级热备机，副机一般设为99
advert_int 1 # send keepalived msg every 1 second
nopreempt # must need
authentication { # all node must same
auth_type PASS
auth_pass 1111
}
track_interface { #需要检测的其它
eth0 #在些添加的网络接口会被检测，eth0不正常会切换热备服务
}
notify_master /etc/keepalived/to_master.sh #切换为master时执行
notify_backup /etc/keepalived/to_backup.sh #切换为backup时执行
}
virtual_server 127.0.0.1 3306 { #表示需要检测的服务
delay_loop 2
lb_algo rr
lb_kind NAT
persistence_timeout 60
protocol TCP
real_server 127.0.0.1 3306 {
weight 3
notify_down /etc/keepalived/shutdown.sh #服务检测失败执行操作
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
```
#### 修改backup主机上的/etc/keepalived/keepalived.conf
```
! Configuration File for keepalived
global_defs {
router_id xchange_router # all node must same
}
vrrp_instance xchange {
state BACKUP # master or backup
interface eth1 # 热备通信网口
virtual_router_id 55 # all node must same
priority 99 # 优先级，高的先运行。主机异常时切换到低优先级热备机，副机一般设为99
advert_int 1 # send keepalived msg every 1 second
#nopreempt # must need
authentication { # all node must same
auth_type PASS
auth_pass 1111
}
track_interface { #需要检测的其它
eth0 #在些添加的网络接口会被检测，eth0不正常会切换热备服务
}
notify_master /etc/keepalived/to_master.sh #切换为master时执行
notify_backup /etc/keepalived/to_backup.sh #切换为backup时执行
}
virtual_server 127.0.0.1 3306 { #表示需要检测的服务
delay_loop 2
lb_algo rr
lb_kind NAT
persistence_timeout 60
protocol TCP
real_server 127.0.0.1 3306 {
weight 3
notify_down /etc/keepalived/shutdown.sh #服务检测失败执行操作
TCP_CHECK {
connect_timeout 10
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
```
#### 创建shell脚本，放在/etc/keepalived目录下
> 脚本1：shutdown.sh
```
#!/bin/bash
echo "shutdown.sh" >> /var/log/messages
killall keepalived
/etc/init.d/manager stop
```
>脚本2：to_master.sh
```
#!/bin/bash
echo "to_master" >> /var/log/messages
/etc/init.d/manager stop
/etc/init.d/manager start
```
>脚本3:to_backup.sh
```
#!/bin/bash
echo "to_backup" >> /var/log/messages
/etc/init.d/manager stop
```
>脚本4:keepalived_sync.sh,
```
#!/bin/bash
# usage: sh keepalived_sync.sh
USER=appbackup # the account to login rsyc server. Notice, it's not the linux user
IP=192.168.1.152 # the rsync server's ip.主备对应修改，192.168.1.151/192.168.1.152
DES="xchange"
SRC="/usr/app/xchange/repository"
PASSWD_FILE="/etc/rsync_client.secret"
LOGFILE="/var/log/messages.log"
/usr/bin/inotifywait -mrq --timefmt '%d/%m/%y %H:%M' --format '%T %w%f%e' -e close_write,modify,delete,create,attrib,move ${src} | while read file
do
/usr/bin/rsync -vzrtopg --delete --progress --password-file=$PASSWD_FILE $SRC $USER@$IP::$DES
echo "${files} was rsynced" >>$LOGFILE 2>&1
done
```
#### keepalived的相关配置文件存储位置如下：
```
root@deb:/etc/keepalived$ tree /etc/keepalived/
/etc/keepalived/
├── keepalived.conf
├── to_backup.sh
├── shutdown.sh
├── to_master.sh
└── keepalived_sync.sh
1 directory, 5 files
```
## 安装Rsync
```
apt-get install rsync
```
#### 修改/etc/default/rsync文件
```
RSYNC_ENABLE=true
```
#### 配置rsyncd的/etc/rsyncd.conf
```
cat /etc/rsyncd.conf
```
```
# sample rsyncd.conf configuration file
# GLOBAL OPTIONS
#motd file=/etc/motd
log file=/var/log/messages
# for pid file, do not use /var/run/rsync.pid if
# you are going to run rsync out of the init.d script.
# The init.d script does its own pid file handling,
# so omit the "pid file" line completely in that case.
# pid file=/var/run/rsyncd.pid
syslog facility=daemon
#socket options=
# MODULE OPTIONS
[xchange]
comment = public archive
path = /usr/app/xchange/repository
use chroot = yes
# max connections=10
lock file = /var/lock/rsyncd
# the default for read only is yes...
read only = no
list = yes
uid = root
gid = root
# exclude =
# exclude from =
# include =
# include from =
auth users =appbackup
secrets file = /etc/rsyncd.secrets
strict modes = yes
# hosts allow =
# hosts deny =
ignore errors = no
ignore nonreadable = yes
transfer logging = no
# log format = %t: host %h (%a) %o %f (%l bytes). Total %b bytes.
timeout = 600
refuse options = checksum dry-run
dont compress = *.gz *.tgz *.zip *.z *.rpm *.deb *.iso *.bz2 *.tbz
```
#### 修改/etc/rsync.secret
```
sudo cat /etc/rsync.secret
```
```
appbackup:passwd
```
> 修改/etc/rsync.secret文件属性:
```
chown root:root /etc/rsync.secret
chmod 600 /etc/rsync.secret
```
#### 启动rsync服务器
```
service rsync restart
```
#### 配置rsync客户端,
> 文件/etc/rsync_client.secret内容如下
```
passwd
```
> 修改文件权限
```
sudo chmod 600 /etc/rsync_client.secret
```
** 注意：如果不修改权限，则会报错ERROR: password file must not be other-accessible**
## 安装 inotify-tools
```
apt-get install inotify-tools
```
## 5.启动
#### 1. 启动master主机上的rsync
```
service rsync restart
```
#### 2. 启动master主机上的keepalived
```
service keepalived restart
```
#### 5. 查看启动日志
```
tail -f /var/log/messages.log
```
## 6.自动切换测试
#### 确认keepalived、rsync已经启动
```
ps -ef | grep keepalived
ps -ef | grep rsync
```
## 7.加入开机脚本
#echo "/data/sh/inotifyrsync.sh &" >> /etc/rc.local
## 8.manager去掉开机启动
#### 切换测试
> 1.停止master主机上的mysql进程，查看master主机上的manager服务是否停止，检查backup主机是否启动manager服务。
> 2.或者关机master，查backup主机是否启动manager服务。
执行流程说明：
默认情况下主副机器都不会运行manager服务，主副机启动keepalived服务，首行执行优先级高的主机to_master脚本，启动manager服务,如果出现mysql停止或宕机情况，会执行shutdown.sh停止热备和manager服务，此时主副切换，副机执行to_master脚本,过程中如果主机修复，主机会抢占热备服务，副机执行to_backup脚本并停止manager服务，主机重新执行to_master并开启manager