简介
这种方案,使用两个VIP地址,前端使用2台机器,互为主备,同时有两台机器工作,当其中一台机器出现故障,两台机器的请求转移到一台机器负担,非常适合于生产架构环境。
一、网络拓扑
四台虚拟机如下所示:
二、配置
软件安装配置和前篇文章是一样的,只不过keepalived的配置文件有些区别,将两台机器上的/etc/keepalived/keepalived.conf配置文件修改成如下:
Nginx-A:
[root@Nginx-A src]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived ############################ 全局配置 ############################# global_defs { # 定义管理员邮件地址,表示keepalived在发生诸如切换操作时需要发送email通知,以及email发送给哪些邮件地址,可以有多个,每行一个 notification_email { #设置报警邮件地址,可以设置多个,每行一个。 需开启本机的sendmail服务 [email protected] } #keepalived在发生诸如切换操作时需要发送email通知地址,表示发送通知的邮件源地址是谁 notification_email_from [email protected] #指定发送email的smtp服务器 smtp_server 127.0.0.1 #设置连接smtp server的超时时间 smtp_connect_timeout 30 #运行keepalived的机器的一个标识,通常可设为hostname。故障发生时,发邮件时显示在邮件主题中的信息。 router_id swarm01 } ############################ VRRPD配置 ############################# # 定义chk_http_port脚本,脚本执行间隔10秒,权重-5,检测nginx服务是否在运行。有很多方式,比如进程,用脚本检测等等 vrrp_script chk_http_port { #这里通过脚本监测 script "/opt/chk_nginx.sh" #脚本执行间隔,每2s检测一次 interval 2 #脚本结果导致的优先级变更,检测失败(脚本返回非0)则优先级 -5 weight -5 #检测连续2次失败才算确定是真失败。会用weight减少优先级(1-255之间) fall 2 #检测1次成功就算成功。但不修改优先级 rise 1 } #定义vrrp实例,VI_1 为虚拟路由的标示符,自己定义名称,keepalived在同一virtual_router_id中priority(0-255)最大的会成为master,也就是接管VIP,当priority最大的主机发生故障后次priority将会接管 vrrp_instance VI_1 { #指定keepalived的角色,MASTER表示此主机是主服务器,BACKUP表示此主机是备用服务器。注意这里的state指定instance(Initial)的初始状态,就是说在配置好后,这台服务器的初始状态就是这里指定的, #但这里指定的不算,还是得要通过竞选通过优先级来确定。如果这里设置为MASTER,但如若他的优先级不及另外一台,那么这台在发送通告时,会发送自己的优先级,另外一台发现优先级不如自己的高, #那么他会就回抢占为MASTER state MASTER #指定HA监测网络的接口。与本机 IP 地址所在的网络接口相同,可通过ip addr 查看 interface ens33 # 发送多播数据包时的源IP地址,这里注意了,这里实际上就是在哪个地址上发送VRRP通告,这个非常重要, #一定要选择稳定的网卡端口来发送,这里相当于heartbeat的心跳端口,如果没有设置那么就用默认的绑定的网卡的IP,也就是interface指定的IP地址 mcast_src_ip 192.168.182.110 #虚拟路由标识,这个标识是一个数字,同一个vrrp实例使用唯一的标识。即同一vrrp_instance下,MASTER和BACKUP必须是一致的 virtual_router_id 51 #定义优先级,数字越大,优先级越高,在同一个vrrp_instance下,MASTER的优先级必须大于BACKUP的优先级 priority 101 #设定MASTER与BACKUP负载均衡器之间同步检查的时间间隔,单位是秒 advert_int 1 #设置验证类型和密码。主从必须一样 authentication { #设置vrrp验证类型,主要有PASS和AH两种 auth_type PASS #设置vrrp验证密码,在同一个vrrp_instance下,MASTER与BACKUP必须使用相同的密码才能正常通信 auth_pass 1111 } #VRRP HA 虚拟地址 如果有多个VIP,继续换行填写 #设置VIP,它随着state变化而增加删除,当state为master的时候就添加,当state为backup的时候则删除,由优先级决定 virtual_ipaddress { 192.168.182.156 } #执行nginx检测脚本。注意这个设置不能紧挨着写在vrrp_script配置块的后面(实验中碰过的坑),否则nginx监控失效!! track_script { #引用VRRP脚本,即在 vrrp_script 部分指定的名字。定期运行它们来改变优先级,并最终引发主备切换。 chk_http_port } } #定义vrrp实例,VI_2 为虚拟路由的标示符,自己定义名称,keepalived在同一virtual_router_id中priority(0-255)最大的会成为master,也就是接管VIP,当priority最大的主机发生故障后次priority将会接管 vrrp_instance VI_2 { #指定keepalived的角色,MASTER表示此主机是主服务器,BACKUP表示此主机是备用服务器。注意这里的state指定instance(Initial)的初始状态,就是说在配置好后,这台服务器的初始状态就是这里指定的, #但这里指定的不算,还是得要通过竞选通过优先级来确定。如果这里设置为MASTER,但如若他的优先级不及另外一台,那么这台在发送通告时,会发送自己的优先级,另外一台发现优先级不如自己的高, #那么他会就回抢占为MASTER state BACKUP #指定HA监测网络的接口。与本机 IP 地址所在的网络接口相同,可通过ip addr 查看 interface ens33 # 发送多播数据包时的源IP地址,这里注意了,这里实际上就是在哪个地址上发送VRRP通告,这个非常重要, #一定要选择稳定的网卡端口来发送,这里相当于heartbeat的心跳端口,如果没有设置那么就用默认的绑定的网卡的IP,也就是interface指定的IP地址 mcast_src_ip 192.168.182.110 #虚拟路由标识,这个标识是一个数字,同一个vrrp实例使用唯一的标识。即同一vrrp_instance下,MASTER和BACKUP必须是一致的 virtual_router_id 52 #定义优先级,数字越大,优先级越高,在同一个vrrp_instance下,MASTER的优先级必须大于BACKUP的优先级 priority 100 #设定MASTER与BACKUP负载均衡器之间同步检查的时间间隔,单位是秒 advert_int 1 #设置验证类型和密码。主从必须一样 authentication { #设置vrrp验证类型,主要有PASS和AH两种 auth_type PASS #设置vrrp验证密码,在同一个vrrp_instance下,MASTER与BACKUP必须使用相同的密码才能正常通信 auth_pass 1111 } #VRRP HA 虚拟地址 如果有多个VIP,继续换行填写 #设置VIP,它随着state变化而增加删除,当state为master的时候就添加,当state为backup的时候则删除,由优先级决定 virtual_ipaddress { 192.168.182.157 } #执行nginx检测脚本。注意这个设置不能紧挨着写在vrrp_script配置块的后面(实验中碰过的坑),否则nginx监控失效!! track_script { #引用VRRP脚本,即在 vrrp_script 部分指定的名字。定期运行它们来改变优先级,并最终引发主备切换。 chk_http_port } }
Nginx-B:
[root@Nginx-B src]# vim /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { [email protected] } notification_email_from [email protected] smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id swaram02 } vrrp_script chk_http_port { script "/opt/chk_nginx.sh" interval 2 weight -5 fall 2 rise 1 } vrrp_instance VI_1 { state BACKUP interface ens33 mcast_src_ip 192.168.182.111 virtual_router_id 51 priority 90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.182.156 } track_script { chk_http_port } } vrrp_instance VI_2 { state MASTER interface ens33 mcast_src_ip 192.168.182.111 virtual_router_id 52 priority 100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 192.168.182.157 } track_script { chk_http_port } }
然后修改两台机器上的nginx首页
[root@Nginx-A html]# vim /usr/local/nginx/html/index.html <!DOCTYPE html> <html> <head> <title>Welcome to Nginx-Master!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to Nginx-A!</h1> <h1><b>Nginx-A:192.168.182.110</b></h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br /> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> [root@Nginx-B html]# vim /usr/local/nginx/html/index.html <!DOCTYPE html> <html> <head> <title>Welcome to Nginx-Master!</title> <style> body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; } </style> </head> <body> <h1>Welcome to Nginx-B!</h1> <h1><b>Nginx-B:192.168.182.111</b></h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br /> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html>
chk_nginx.sh脚本也不用改,然后重启keepalived
[root@Nginx-A keepalived]# systemctl start keepalived.service [root@Nginx-A keepalived]# systemctl status keepalived.service ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled) Active: active (running) since 二 2018-05-29 14:46:25 CST; 4s ago Process: 10590 ExecStart=/usr/local/keepalived/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 7396 (keepalived) Tasks: 6 Memory: 3.0M CGroup: /system.slice/keepalived.service ├─ 7396 /usr/local/keepalived/sbin/keepalived -D ├─ 7397 /usr/local/keepalived/sbin/keepalived -D ├─ 7398 /usr/local/keepalived/sbin/keepalived -D ├─ 99999 nginx: master process /usr/local/nginx/sbin/nginx ├─100001 nginx: worker process └─100002 nginx: worker process 5月 29 14:46:25 Nginx-A systemd[1]: Starting LVS and VRRP High Availability Monitor... 5月 29 14:46:25 Nginx-A Keepalived[10590]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2 5月 29 14:46:25 Nginx-A Keepalived[10590]: Unable to resolve default script username [root@Nginx-B conf]# /etc/init.d/keepalived status ● keepalived.service - LVS and VRRP High Availability Monitor Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled) Active: active (running) since 二 2018-05-29 14:45:34 CST; 4min 17s ago Process: 5770 ExecStart=/usr/local/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS) Main PID: 2771 (keepalived) Tasks: 3 Memory: 960.0K CGroup: /system.slice/keepalived.service ├─2771 /usr/local/sbin/keepalived -D ├─2772 /usr/local/sbin/keepalived -D └─2773 /usr/local/sbin/keepalived -D 5月 29 14:45:34 Nginx-B systemd[1]: Starting LVS and VRRP High Availability Monitor... 5月 29 14:45:34 Nginx-B Keepalived[5770]: Starting Keepalived v1.3.5 (03/19,2017), git commit v1.3.5-6-g6fa32f2 5月 29 14:45:34 Nginx-B systemd[1]: Started LVS and VRRP High Availability Monitor.三、keepalived+nginx的高可用测试
3.1 查看服务器上的IP地址
查看Nginx-A的地址:
[root@Nginx-A keepalived]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:e0:69:a0 brd ff:ff:ff:ff:ff:ff inet 192.168.182.110/24 brd 192.168.182.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet 192.168.182.156/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.182.157/32 scope global ens33 valid_lft forever preferred_lft forever inet6 fe80::3064:9b6a:9819:180a/64 scope link noprefixroute valid_lft forever preferred_lft forever查看Nginx-B的地址:
[root@Nginx-B keepalived]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:25:44:71 brd ff:ff:ff:ff:ff:ff inet 192.168.182.111/24 brd 192.168.182.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::2818:83f9:1989:3130/64 scope link noprefixroute valid_lft forever preferred_lft forever inet6 fe80::3064:9b6a:9819:180a/64 scope link tentative noprefixroute dadfailed valid_lft forever preferred_lft forever
会多出两个虚拟IP
inet 192.168.182.156/32 scope global ens33 valid_lft forever preferred_lft forever inet 192.168.182.157/32 scope global ens33
首先通过两个VIP地址访问如下:
3.2、关闭Nginx-A上的nginx,keepalived将在2s内将它重新启动
[root@Nginx-A keepalived]# /usr/local/nginx/sbin/nginx -s stop
3.3、关闭Nginx-A上的keepalived,VIP会切换到Nginx-B上
[root@Nginx-A keepalived]# service keepalived stop
不管访问的是:http://192.168.182.157/ 还是http://192.168.182.156/ 都将导到Nginx-B上去。
同时会发现Nginx-A上的虚拟IP会消失
[root@Nginx-A keepalived]# ip addr 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000 link/ether 00:0c:29:e0:69:a0 brd ff:ff:ff:ff:ff:ff inet 192.168.182.110/24 brd 192.168.182.255 scope global noprefixroute ens33 valid_lft forever preferred_lft forever inet6 fe80::3064:9b6a:9819:180a/64 scope link noprefixroute valid_lft forever preferred_lft forever