搭建lvs + keepalived 服务器

搭建lvs + keepalived 服务器
环境： linux redhat 7.2
四台主机划分
LVS 1:10.6.248.196
LVS 2:10.6.248.197
MYSQL MASTER:10.6.248.198
MYSQL MASTER:10.6.248.199
VIP:10.6.248.192

其中问题：
1，两台LVS争夺VIP
原因：Keepalived 配置文件中的 ID 号在同一个IP地址段中，被其他Keepalived的使用。
2，无法绑定VIP
VIP 地址被占用，更换VIP 地址

一，安装及配置mysql 主主

二，安装keepalived + lvs

 1，先安装所需软件包

执行命令：
#yum install -y kernel-devel openssl openssl-devel gcc* popt-devel ncurses* bison* ipvsadm httpd perl* libart* rrdtool* c* gc* libnl*

 2，确认内核支持 ipvs

执行命令：
#modprobe -l | grep ipvs
#lsmod | grep ip_vs
操作系统中7.2版本下，我只能用#lsmod | grep ip_vs 具体原因没有细究

 3，可以通过YUM 安装ipvsadm(lvs)

执行命令：
#yum -y install ipvsadm
或者
#wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.26.tar.gz

 4，通过执行ipvsadm 命令来检验是否安装成功

执行命令：
#ipvsadm
参数一般使用 -Ln 或者-ln

 5，安装keepalived

执行命令：
#wget http://www.keepalived.org/software/keepalived-1.2.13.tar.gz
#tar -zxvf keepalived-1.2.13.tar.gz
#cd keepalived-1.2.13
#make && make install
#./configure --prefix=/usr/local/keepalived --with-kernel-dir=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64/
创建目录并进行配置：
# mkdir /etc/keepalived

     #cp /usr/local/keepalived/etc/rc.d/init.d/keepalived  /etc/rc.d/init.d/​

     #cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/​

     #cp /usr/local/keepalived/etc/keepalived/keepalived.conf  /etc/keepalived/

​ #cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

​ #chkconfig keepalived on
修改keepalived配置文件

执行命令：
#vim /etc/keepalived/keepalived.conf
内如如下

! Configuration File for keepalived

global_defs {
notification_email {
[email protected]
[email protected]
[email protected]
}
notification_email_from [email protected]
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id LVS_DEVEL
}

vrrp_instance VI_1 {
state MASTER 备库修改为BACKUP
interface eno16777984
virtual_router_id 123 备库ID 一样
priority 130 备库优先级要设置降低 130一下
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
10.6.248.192 虚拟IP
}
}

virtual_server 10.6.248.192 3306 { 虚拟IP
delay_loop 6
lb_algo rr 策略
lb_kind DR 策略
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP

real_server 10.6.248.199 3306 {                  受虚拟IP 监控的地址
    weight 1                                     权重
    SSL_GET {
        connect_timeout 3
        nb_get_retry 3
        delay_before_retry 3
    }
}


real_server 10.6.248.198 3306 {
    weight 1
    SSL_GET {
        connect_timeout 3
        nb_get_retry 3
        delay_before_retry 3
    }
}

}

keepalived 的配置文件默认为抢夺模式，
在vrrp_instance VI_1下面增加一个nopreempt 参数，设置为非抢夺模式。
但是HA的非抢夺模式时将master 和 backup的state都设置backup ，让其通过priority来竞争。

Keepalived LVS2上面也需要如此配置

配置LVS （此脚本需要添加到LVS1,MYSQL-MASTER-1,MYSQL-MASTER-2）上。
mysql-master1 及mysql-master2上如果没有此脚本，那么在应用端通过mysql客户端连接的时候，是无法进入mysql数据库的。

执行命令：
#vim /etc/init.d/realserver.sh

内容如下：
脚本一：

#!/bin/bash
#description: Config realserver

VIP=10.6.248.192

/etc/rc.d/init.d/functions

case “$1” in
start)
/sbin/ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo “1” >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/lo/arp_announce
echo “1” >/proc/sys/net/ipv4/conf/all/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo “RealServer Start OK”
;;
stop)
/sbin/ifconfig lo:0 down
/sbin/route del $VIP >/dev/null 2>&1
echo “1” >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/lo/arp_announce
echo “1” >/proc/sys/net/ipv4/conf/all/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/all/arp_announce
echo “RealServer Stoped”
;;
*)
echo “Usage: $0 {start|stop}”
exit 1
esac

exit 0

脚本二： （仅用与参考，网上有）

#!/bin/bash
#description: Config realserver
VIP=10.6.248.170
/etc/rc.d/init.d/functions
case “$1” in
start)
/sbin/ifconfig lo down
/sbin/ifconfig lo up
ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
/sbin/route add -host $VIP dev lo:0
echo “1” >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/lo/arp_announce
echo “1” >/proc/sys/net/ipv4/conf/all/arp_ignore
echo “2” >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo “RealServer Start OK”
;;
stop)
ifconfig lo:0 down
route del KaTeX parse error: Expected 'EOF', got '&' at position 18: …P >/dev/null 2>&̲1 echo "…VIP"isRoOn=/bin/netstat -rn | grep “ $VIP"` if [ "$isLoON” == “” -a “$isRoOn” == “” ]; then
echo “LVS-DR real server has run yet.”
else
echo “LVS-DR real server is running.”
fi
exit 3
;;
*)
echo “Usage: $0 {start|status|stop}”
exit 1
esac
exit 0

将LVS 脚本加入开机自启动
执行如下命令：
#chmod +x /etc/init.d/realserver
#echo “/etc/init.d/realserver” >> /etc/rc.d/rc.local
分别启动lVS 和 keepalived
执行如下命令：
# service realserver start / sh /etc/init.d/realserver.sh start
# service keepalived start

在mysql 主主的服务器上 执行

          # service realserver start   /  sh /etc/init.d/realserver.sh start

LVS + keepalived 配置到此结束

linux 环境配置
执行命令：
#vim /etc/hosts
127.0.0.1 Kmaster localhost.localdomain localhost4 localhost4.localdomain4
::1 Kmaster localhost.localdomain localhost6 localhost6.localdomain6
10.6.248.199 Mymaster
10.6.248.198 Myslave
10.6.248.197 Monitor
10.6.248.196 Kmaster
#10.6.248.170 VIP

          #vim  /etc/sysctl.conf

net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.ip_forward = 1
net.ipv4.conf.lo.arp_ignore = 1
net.ipv4.conf.lo.arp_announce = 2
net.ipv4.conf.all.arp_ignore = 1
net.ipv4.conf.all.arp_announce = 2
net.ipv4.ip_nonlocal_bind = 1
#sysctl -p

测试如下：
1，#ipvsadm
显示结果如下
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 10.6.248.192:mysql rr persistent 50
-> Myslave:mysql Route 1 0 0
-> Mymaster:mysql Route 1 0 0

2，ipvsadm -ln --stats 当有连接的时候，ActiveConn 以及 InActConn 会有变化

3，查看keepalived 检测状态以及查看 /var/log/messages的日志
#service keepalived status

4，关闭Master 1 上的mysql 服务
查看LVS1 上的日志 vim /var/log/messages 可以发现已经Master 1 踢出
重启Master 1 上的mysql 服务
可以发现Master 1已经加入
5，关闭LVS 1 上的 keepalived 的服务，会发现，VIP 已经转移到 LVS 2上，且日志中会有显示，可以查看LVS 2上的keepalived的状态。

select host,user,authentication_string from mysql.user;

show grants for ‘repl’@‘10.6.248.%’
grant replication slave on . to ‘repl’@‘10.6.248.%’;

insert into mysql.user(host,user,authentication_string,ssl_cipher,x509_issuer,x509_subject) values (‘10.6.249.%’,‘rep1’,password(‘cyh123@’),’’,’’,’’);

update mysql.user set select_priv=‘y’, insert_priv=‘y’,update_priv=‘y’, Alter_priv=‘y’,delete_priv=‘y’,create_priv=‘y’,drop_priv=‘y’,reload_priv=‘y’,shutdown_priv=‘y’,Process_priv=‘y’,file_priv=‘y’,grant_priv=‘y’,References_priv=‘y’,index_priv=‘y’,create_user_priv=‘y’,show_db_priv=‘y’,super_priv=‘y’,create_tmp_table_priv=‘y’,Lock_tables_priv=‘y’,execute_priv=‘y’,repl_slave_priv=‘y’,repl_client_priv=‘y’,create_view_priv=‘y’,show_view_priv=‘y’,create_routine_priv=‘y’,alter_routine_priv=‘y’,create_user_priv=‘y’ where host=‘10.6.249.%’ and user=‘rep1’;