MYSQL to build the architecture of MHA

I. Introduction of MHA

• Currently in terms of high availability MySQL is a relatively mature solution that youshimaton developed by the Japanese, is a good switch as a failure at the primary and MySQL high availability environments from an elevated high availability software.

• In MySQL failover process, MHA can be done automatically within 0-30 seconds failed database switching operation, and performing the handover procedure failure, to ensure the consistency of the database MHA greatest extent, in order to achieve a real sense high availability.

• When the Master fails, it can automatically upgrade to the latest data Slave new Master, and then all the other Slave redirected to the new Master. The entire failover process is completely transparent to the application.

1.1, MHA composition

• MHA Manager (management node):
  Manager can be deployed individually manage a plurality of master-slave cluster on a separate machine, can be deployed on a slave node.
• MHA Node (node data):
  the Node MySQL running on each server, master node MHA Manager will regularly detect the cluster, when the master fails, it can automatically slave to the latest data of the upgrade to the new master, then all other slave redirected to the new master.

1.2, MHA Features

• Automatic failover process, MHA attempt. Save binary log on from the primary server is down, the greatest degree of assurance without losing data
• MySQL 5.5 using semi-synchronous replication, it can greatly reduce the risk of data loss

Two, MHA case

2.1, case demand

• This case requires automatic switching in case of failure by MHA monitoring MySQL database does not affect business.

2.2, case realization of ideas

• 1, install the MySQL database
• 2, a main configuration MySQl from two
• 3, install the software MHA
• 4, no configuration password authentication
• 5, configure high availability MySQL MHA
• 6, the analog master failover

2.3 Case environment

management	IP addresses
manager	192.168.220.130
master	192.168.220.136
slave1	192.168.220. 164
slave2	192.168.220.159

• Operating System: Cent0S7-6 version, MHA version is version 0.57, Mysql database version 5.7 is installed

First, install MYSQL database on three servers. Here I will not show the specific structures before the process see blog, link to: Mysql5.7 version installed Detailed steps

Third, the specific build process

It is recommended that experiments with hostnamectl host command to modify, easy to experiment.

#将manager服务器的主机名改为manager
hostnamectl set-hostname manager
su
#修改服务器名称，将manster服务器的主机名改为manster
hostnamectl set-hostname manster
su
 #修改服务器名称，将slave1服务器的主机名改为slaveq
hostnamectl set-hostname slave1
 su
 #修改服务器名称，将slave2服务器的主机名改为slave2
hostnamectl set-hostname slave2
 su

Like this, the configuration process will not be confused.

3.1, MySQL achieve a master's from two

A, MYSQL database installation has been completed, the next one is Mysql from two main deployment.

• Master master:

1, mysql main server configuration

vim /etc/my.cnf
server-id = 1               //指定id号，服务器的唯一标识，不能相同
log-bin=master-bin           //主服务器日志文件
log-slave-updates=true       //从服务器更新二进制日志

2, restart the service

systemctl restart mysqld

• From the server slave1:

1, modify the configuration file

vim /etc/my.cnf

server-id = 2       
relay-log=relay-log-bin     //从主服务器上同步日志文件记录到本地  
relay-log-index=slave-relay-bin.index   //定义relay-log的位置和名称

2, restart the service

systemctl restart mysqld

Also modify id and logs from the server slave2, restart the service.

vim /etc/my.cnf
server-id = 3       
relay-log=relay-log-bin     //从主服务器上同步日志文件记录到本地  
relay-log-index=slave-relay-bin.index   //定义relay-log的位置和名称
#重启服务
systemctl restart mysqld

3, to master, slave1, slave2 soft links were established

ln -s /usr/local/mysql/bin/mysql /usr/sbin/
ln -s /usr/local/mysql/bin/mysqlbinlog /usr/sbin/

4, the authorization database. Authorization database on all nodes of the two users, the simultaneous use of a library is, the other is used to monitor manager, i.e. master, slave1, slave2 build on.

mysql -u root -pabc123

grant replication slave on *.* to 'myslave'@'192.168.220.%' identified by '123';
grant all privileges on *.* to 'mha'@'192.168.220.%' identified by 'manager';
flush privileges;      //刷新数据库的权限相关表

View binary file and location on the master server, File and Position values both in the back from time to configure the server will be used.

On the plus slave1 and slave2 server permissions.

mysql -u root -pabc123
#授权要根据在master上看到的日志文件和position填写，不能乱填
change master to master_host='192.168.220.136',master_user='myslave',master_password='123123',master_log_file='master-bin.000002',master_log_pos=774;

5, start the synchronization, viewing slave status, ensure that the following two values ​​YES

start slave;    //开启从服务       
show slave status\G;    //查看状态

必须设置两个从库slave为只读模式
set global read_only=1;

flush privileges;      //刷新数据库的权限相关表

3.2, install the MHA

1, to install all the server environment dependence MHA


2, all components of the server node must be installed, and then mounted on the manager node manager component.

• Because the node manager component dependent

Install all server node components

#挂载共享文件
mount.cifs //192.168.220.128/share /mnt
cd ~
tar zxvf /mnt/mysql/mha/mha4mysql-node-0.57.tar.gz
cd mha4mysql-node-0.57/
perl Makefile.PL     
make && make install

Mounted on the manager node manager component

cd ~
tar zxvf /mnt/mysql/mha/mha4mysql-manager-0.57.tar.gz
cd mha4mysql-manager-0.57/
perl Makefile.PL    
make && make install

• manager 安装后在 /usr/local/bin/ 目录下会生成几个工具

主要包括以下几个：

masterha_check_ssh	检查 MHA 的 SSH 配置状况
masterha_check_repl	检查mysql复制状况
masterha_master_monitor	检查master是否宕机
masterha_master_switch	控制故障转移
masterha_check_status	检查当前MHA运行状态
masterha_conf_host	添加或删除配置的server信息
masterha_stop	关闭manager
masterha_manager	启动manager的脚本

• node 安装后也会在/usr/local/bin/目录下会生成几个脚本。
  这些工具通常由Manager 的脚本触发，无需人为操作， 主要如下:

save_binary_logs	保存和复制 master 的二进制日志
apply_diff_relay_logs	识别差异的中继日志事件并将其差异的事件应用于其他的slave
filter_mysqlbinlog	去除不必要的 rollback 事件(MHA 己不再使用这个工具)
purge_relay_logs	清除中继日志 (不会阻塞 SQL 线程)

3.3、配置无密码访问

//1、在 manager 上配置到所有数据库节点的无密码认证
ssh-keygen -t rsa     //一路按回车键
ssh-copy-id 192.168.220.136     按yes回车密码：123
ssh-copy-id 192.168.220.164
ssh-copy-id 192.168.220.159

//2、在 master 上配置到数据库节点 slave1 和 slave2 的无密码认证
ssh-keygen -t rsa     //一路按回车键
ssh-copy-id 192.168.220.164   按yes回车密码：123
ssh-copy-id 192.168.220.159

//3、在 slave1 上配置到数据库节点 master 和slave2 的无密码认证
ssh-keygen -t rsa     //一路按回车键
ssh-copy-id 192.168.220.136   按yes回车密码：123
ssh-copy-id 192.168.220.159

//4、在 slave2 上配置到数据库节点 master 和 slave1 的无密码认证
ssh-keygen -t rsa     //一路按回车键
ssh-copy-id 192.168.220.136    按yes回车密码：123
ssh-copy-id 192.168.220.164

3.4、配置MHA

1、在manager节点上复制相关脚本到/usr/local/bin 的目录

cp -ra /root/mha4mysql-manager-0.57/samples/scripts /usr/local/bin

拷贝后会有四个执行文件：

master_ip_failover	自动切换时 VIP 管理的脚本
master_ip_online_change	在线切换时 vip 的管理
power_manager	故障发生后关闭主机的脚本
send_report	因故障切换后发送报警的脚本

2、复制上述的自动切换时VIP管理的脚本到/usr/local/bin 目录，这里使用脚本管理VIP

cp /usr/local/bin/scripts/master_ip_failover /usr/local/bin

重新编写master_ip_failover脚本，内容如下：

vim /usr/local/bin/master_ip_failover
#!/usr/bin/env perl
use strict;
use warnings FATAL => 'all';

use Getopt::Long;

my (
$command, $ssh_user, $orig_master_host, $orig_master_ip,
$orig_master_port, $new_master_host, $new_master_ip, $new_master_port
);
#############################添加内容部分#########################################
my $vip = '192.168.195.200';
my $brdc = '192.168.195.255';
my $ifdev = 'ens33';
my $key = '1';
my $ssh_start_vip = "/sbin/ifconfig ens33:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig ens33:$key down";
my $exit_code = 0;
#my $ssh_start_vip = "/usr/sbin/ip addr add $vip/24 brd $brdc dev $ifdev label $ifdev:$key;/usr/sbin/arping -q -A -c 1 -I $ifdev $vip;iptables -F;";
#my $ssh_stop_vip = "/usr/sbin/ip addr del $vip/24 dev $ifdev label $ifdev:$key";
##################################################################################
GetOptions(
'command=s' => \$command,
'ssh_user=s' => \$ssh_user,
'orig_master_host=s' => \$orig_master_host,
'orig_master_ip=s' => \$orig_master_ip,
'orig_master_port=i' => \$orig_master_port,
'new_master_host=s' => \$new_master_host,
'new_master_ip=s' => \$new_master_ip,
'new_master_port=i' => \$new_master_port,
);

exit &main();

sub main {

print "\n\nIN SCRIPT TEST====$ssh_stop_vip==$ssh_start_vip===\n\n";

if ( $command eq "stop" || $command eq "stopssh" ) {

my $exit_code = 1;
eval {
print "Disabling the VIP on old master: $orig_master_host \n";
&stop_vip();
$exit_code = 0;
};
if ($@) {
warn "Got Error: $@\n";
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "start" ) {

my $exit_code = 10;
eval {
print "Enabling the VIP - $vip on the new master - $new_master_host \n";
&start_vip();
$exit_code = 0;
};
if ($@) {
warn $@;
exit $exit_code;
}
exit $exit_code;
}
elsif ( $command eq "status" ) {
print "Checking the Status of the script.. OK \n";
exit 0;
}
else {
&usage();
exit 1;
}
}
sub start_vip() {
`ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# A simple system call that disable the VIP on the old_master
sub stop_vip() {
`ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}

sub usage {
print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

Note: Copy the contents of this script into a script file, if each line more than a # appears, you can enter the end of the line mode:% s / # / press Enter

3, MHA software to create a directory and copy the configuration file

mkdir /etc/masterha  #创建MHA目录
#复制配置文件到新建的MHA目录下
cp /root/mha4mysql-manager-0.57/samples/conf/app1.cnf /etc/masterha  
vim /etc/masterha/app1.cnf

[server default]
manager_workdir=/var/log/masterha/app1
manager_log=/var/log/masterha/app1/manager.log
master_binlog_dir=/usr/local/mysql/data
master_ip_failover_script=/usr/local/bin/master_ip_failover
master_ip_online_change_script=/usr/local/bin/master_ip_online_change
password=manager
remote_workdir=/tmp
repl_password=123
repl_user=myslave
secondary_check_script=/usr/local/bin/masterha_secondary_check -s 192.168.220.164 -s 192.168.220.159
shutdown_script=""
ssh_user=root
user=mha

[server1]
hostname=192.168.220.136
port=3306

[server2]
candidate_master=1
check_repl_delay=0
hostname=192.168.220.164
port=3306

[server3]
hostname=192.168.220.159
port=3306                                                                                                                                                                

4, on a test server manager ssh without password authentication, if normal, eventually output successfully.

masterha_check_ssh -conf=/etc/masterha/app1.cnf

Check the health

masterha_check_repl -conf=/etc/masterha/app1.cnf

5, manually turn on the master virtual IP

/sbin/ifconfig ens33:1 192.168.220.200/24

6, start MHA

nohup masterha_manager --conf=/etc/masterha/app1.cnf --remove_dead_master_conf --ignore_last_failover < /dev/null > /var/log/masterha/app1/manager.log 2>&1 &

7, see the MHA state, you can see the current master node is Mysqll

masterha_check_status --conf=/etc/masterha/app1.cnf

You can also view the MHA logs, displays the current master is 192.168.220.136

cat /var/log/masterha/app1/manager.log

3.5, fault simulation

1, enable monitoring observation logging on the manager node.

tailf /var/log/masterha/app1/manager.log  

2, in the closed mysql master.

systemctl stop mysql

Can be seen from the state library, vip will switch to one of the upper from the library.