nginx keepalived lvs

一、环境准备

一共4台虚拟机，我用的mac的parallels desktop创建的4台centos7虚拟机

master 192.168.20.104
salve 192.168.20.103
node1 192.168.20.102
node2 192.168.20.98

可以先安装一个虚拟机，然后进行克隆就行。

二、配置虚拟机

1.安装master的keepalived，参考以下脚本：

# -------------------------------------------------------- #
            ## Keepalived_intsall
# -------------------------------------------------------- #
# Keepalived installation
yum install -y gcc openssl-devel popt-devel
# error libnfnetlink headers missing
yum install -y libnfnetlink-devel
cd /root/software
[ ! -e keepalived-1.2.24.tar.gz ] &&  wget http://www.keepalived.org/software/keepalived-1.2.24.tar.gz
tar -zxvf keepalived-1.2.24.tar.gz
cd keepalived-1.2.24
./configure --prefix=/usr/local/keepalived
make && make install
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/keepalived
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
mkdir -p /etc/keepalived
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf
chmod +x /etc/init.d/keepalived
echo $? || [ $? != 0  ] || print " installation keepalived  failed" || exit 1
chkconfig --add keepalived
chkconfig --level 345 keepalived on

 以上是通过下载相应的压缩包，然后进行安装配置。

centos7这次我用的是yum安装，直接在命令行运行命令：

yum install keepalived //安装
keepalived -v //检查版本 

配置master和相应的salve的keepalived，运行命令：

vim /usr/local/keepalived/keepalived.conf //编辑配置文件

! Configuration File for keepalived

global_defs {
   notification_email {
        root@localhost
        [email protected]                              ##设置邮件报警地址
   }
   notification_email_from [email protected]
   smtp_server 127.0.0.1
   smtp_connect_timeout 3
   router_id LVS_DEVEL
}
vrrp_instance VI_1 {                          ##配置vrrp实例1
    state MASTER                           ##BACKUP修改为BACKUP
    interface eth0
    virtual_router_id 51
    priority 101                           ##BACKUP修改为100或更小
    advert_int 1
    garp_master_delay 5
    authentication {
        auth_type PASS
        auth_pass 1111
    }
virtual_ipaddress {
        192.168.20.10
    }
virtual_server 192.168.20.10 80 {
    delay_loop 6
    lb_algo rr
    lb_kind NAT #负载均衡转发规则 DR NAT TUN。和您将启动的LVS的工作模式设置一致
    nat_mask 255.255.255.0
    persistence_timeout 5
    protocol TCP
    real_server 192.168.20.98 80 {
        weight 10
        HTTP_GET {
            url {
              path /
            status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
      }
    }
real_server 192.168.20.102 80 {
        weight 10
        HTTP_GET {
            url {
              path /
            status_code 200
            }
            connect_timeout 3
            nb_get_retry 3
            delay_before_retry 3
        }
    }
}

2.防火墙设置，运行命令：

sysemctl stop firewalld.service //关闭
sysemctl disable firewalld.service //开机禁止启动

3.安装ipvsadm，运行命令：

yum install ipvsadm //安装
ipvsadm -v //检查版本

4.（可选）安装iptables防火墙，需要配置相应的防火墙策略，运行命令：

vim /etc/sysconfig/iptables

eg:

-A INPUT -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
-A INPUT -s 172.21.4.51 -j ACCEPT
-A INPUT -s 172.21.4.52 -j ACCEPT
-A INPUT -s 172.21.4.91 -j ACCEPT
-A INPUT -s 172.21.4.92 -j ACCEPT

 开放相应的端口。

5.节点安装nginx，注意设置相应的防火墙，以便nginx 80端口能够访问，参考一下脚本：

#!/bin/bash
# author: kuangl
# mail: [email protected]
# description: The installation of Nginx files.
# -------------------------------------------------------- #
         ## Nginx_install
# -------------------------------------------------------- #
# Nginx installation
#CURRENT_PATH=$(pwd)
for i in $(rpm -q gcc gcc-c++ kernel-devel openssl-devel zlib-devel popt-devel popt-static libnl-devel wget make |grep 'not installed' | awk '{print $2}')
do
    yum -y install $i
done
[ -d /root/software ]
[ "$?" != 0 ] && mkdir /root/software
cd /root/software
[ !  -e pcre-8.40.tar.gz ] && wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.40.tar.gz
tar -zxvf pcre-8.40.tar.gz
cd pcre-8.40
./configure
make && make install
echo $? || [ $? != 0  ] || echo  " installation pcre  failed" || exit 1
cd /root/software
[ ! -e nginx-1.11.5.tar.gz ] && wget http://nginx.org/download/nginx-1.11.5.tar.gz
tar -zxvf nginx-1.11.5.tar.gz
cd nginx-1.11.5
./configure  --prefix=/usr/local/nginx --with-http_ssl_module --with-http_sub_module --with-http_stub_status_module  --with-http_gzip_static_module
make && make install
echo $? || [ $? != 0  ] || echo  " installation  nginx  failed" || exit 1

 6.在nginx节点上设置lvs

ipvsadm -C
ipvsadm -At 192.168.20.10:80 -s rr
ipvsadm -at 192.168.20.10:80 -r 192.168.20.98 -m
ipvsadm -at 192.168.20.10:80 -r 192.168.20.102 -m
-a指定真实服务器 -t lvs上VIP -r真实服务器ip及端口 -w权重值 -g先择DR模式 -m为NAT模式

 7.启动并验证

keepalived两种启动方式
（1）自己压缩包安装，参考该目录下的安装sh脚本，启动命令：/etc/init.d/keepalived start
（2）centos7 yum安装，配置成服务
    systemctl daemon-reload  重新加载
    systemctl enable keepalived.service  设置开机自动启动
    systemctl disable keepalived.service 取消开机自动启动
    systemctl start keepalived.service 启动
    systemctl stop keepalived.service停止
（3）查看启动状态
    systemctl status keepalived.service
nginx启动后外部无法访问
（1）检查linux防火墙
（2）查看防火墙配置
    Linux防火墙(Iptables)重启系统生效
    开启： chkconfig iptables on
    关闭： chkconfig iptables off
    
    Linux防火墙(Iptables) 即时生效，重启后失效
    开启： service iptables start
    关闭： service iptables stop
nginx相关命令：
    nginx -v //查看版本
    ps -ef|grep nginx //查看进程，有两个，主进程和子进程
    kill -9 进程号 //杀死进程
    pkill -9 nginx //强制停止
    nginx -c /usr/local/nginx/nginx.conf //启动
    nginx -s stop //快速停止或关闭
    nginx -s qiut //正常停止或关闭
    nginx -s reload //配置文件修改后重新装载

 

8.注意事项

1、输出的日志信息： /var/log/messages ，更具体的日志信息输出需要在启动keepalived时加 -d 参数。
2、在都为MASTER且priority一样的情况下，后启的节点（service vrrp start）会取代正在运行的节点变成主用的。
3、一台为MASTER且priority较高的情况下，不受次节点down/up影响，并且其本身再从down变为up时，会抢夺控制权。
4、在都为MASTER且priority一样的情况下，正在运行的主节点down（断网），次节点会自动接管，主节点再起来时不会去抢夺控制权。
#keepalived会定时执行脚本并对脚本执行的结果进行分析，动态调整vrrp_instance的优先级。
#如果脚本执行结果为0，并且weight配置的值大于0，则优先级相应的增加
#如果脚本执行结果非0，并且weight配置的值小于0，则优先级相应的减少
#其他情况，维持原本配置的优先级，即配置文件中priority对应的值。
#这里需要注意的是：
#1） 优先级“不会”不断的提高或者降低，当track的对象恢复时，又是一致的
#2） 可以编写多个检测脚本并为每个检测脚本设置不同的weight
#3） 不管提高优先级还是降低优先级，最终优先级的范围是在[1,254]，不会出现优先级小于等于0或者优先级大于等于255的情况
#这样可以做到利用脚本检测业务进程的状态，并动态调整优先级从而实现主备切换。 

9、参考链接

http://blog.csdn.net/yinwenjie/article/details/47211551

http://www.linuxidc.com/Linux/2015-07/120179.htm

http://blog.csdn.net/nimasike/article/details/51867046

http://os.51cto.com/art/201103/249045.htm

https://github.com/jiji87432/nginx_sh //相关安装配置脚本