mysql主从
1. 主从简介
在现代企业中，数据显得尤为重要，而存储数据的数据库选择又五花八门，但无论是何种数据库，均存在着一种隐患。
想几个问题：
- 用一台数据库存放数据，若此数据库服务器宕机了导致数据丢失怎么办？
- 业务量大了，数据多了，访问的人多了，一台数据库无法保证服务质量了怎么办？
1.1 主从作用
- 实时灾备，用于故障切换
- 读写分离，提供查询服务
- 备份，避免影响业务
1.2 主从形式
- 一主一从
- 主主复制
- 一主多从—扩展系统读取的性能，因为读是在从库读取的
- 多主一从—5.7开始支持
- 联级复制
2. 主从复制原理
主从复制步骤：
- 主库将所有的写操作记录到binlog日志中并生成一个log dump线程，将binlog日志传给从库的I/O线程
- 从库生成两个线程，一个I/O线程，一个SQL线程
- I/O线程去请求主库的binlog，并将得到的binlog日志写到relay log（中继日志） 文件中
- SQL线程，会读取relay log文件中的日志，并解析成具体操作，来实现主从的操作一致，达到最终数据一致的目的
3. 主从复制配置
主从复制配置步骤：
- 确保从数据库与主数据库里的数据一样
- 在主数据库里创建一个同步账号授权给从数据库使用
- 配置主数据库（修改配置文件）
- 配置从数据库（修改配置文件）
需求：
搭建两台MySQL服务器，一台作为主服务器，一台作为从服务器，主服务器进行写操作，从服务器进行读操作
环境说明：
| 数据库角色 | IP | 应用与系统版本 | 有无数据 |
|---|---|---|---|
| 主数据库 | 172.16.12.128 | centos7/redhat7 mysql-5.7 | 有数据 |
| 从数据库 | 172.16.12.129 | centos7/redhat7 mysql-5.7 | 无数据 |
3.1 mysql安装
分别在主从两台服务器上安装mysql-5.7版本，此处略过安装步骤，若有疑问请参考《mysql基础》与《mysql进阶》两篇文章。
3.2 mysql主从配置
3.2.1 确保从数据库与主数据库里的数据一样
为确保从数据库与主数据库里的数据一样，先全备主数据库并还原到从数据库中
查看主数据库里有哪些库
[root@windfantasy ~]# mysql -uroot -p1 -e 'show databases'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------------------+
| Database |
+--------------------+
| information_schema |
| classmate |
| mysql |
| performance_schema |
| sys |
| zabbix |
+--------------------+
查看从数据库里有哪些库
[root@windfantasy ~]# mysql -uroot -p1 -e 'show databases'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
全备主库
注意：全备主库时需要另开一个终端，禁止数据库写入（只读），避免在备份期间有其他人在写入导致数据不一致
mysql> flush tables with read lock;
Query OK, 0 rows affected (0.01 sec)
//此锁表的终端必须在主从配置完成以后才能退出
备份主库并将备份文件传送到从库
[root@windfantasy ~]# mysqldump -uroot -p1 --all-databases > all.sql
mysqldump: [Warning] Using a password on the command line interface can be insecure.
[root@windfantasy ~]# scp all.sql [email protected]:/root
[email protected]'s password:
all.sql 100% 7254KB 17.6MB/s 00:00
解除主库的锁表状态，直接退出交互式界面即可
mysql> quit
Bye
在从库上恢复主库的备份并查看从库有哪些库，确保与主库一致
[root@windfantasy ~]# mysql -uroot -p1 < all.sql
mysql: [Warning] Using a password on the command line interface can be insecure.
[root@windfantasy ~]# mysql -uroot -p1 -e 'show databases'
mysql: [Warning] Using a password on the command line interface can be insecure.
+--------------------+
| Database |
+--------------------+
| information_schema |
| classmate |
| mysql |
| performance_schema |
| sys |
| zabbix |
+--------------------+
3.2.2 在主数据库里创建一个同步账号授权给从数据库使用
mysql> grant replication slave on *.* to 'wml'@'192.168.43.128' identified by '1';
Query OK, 0 rows affected, 1 warning (1 min 6.41 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
3.2.3 配置主数据库
[root@windfantasy ~]# vim /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
log-bin=mysql-bin //启用binlog日志
server-id=1 //数据库服务器唯一标识符，主库的server-id值必须比从库的小
[root@windfantasy ~]# systemctl restart mysql
[root@windfantasy ~]# mysql -uroot -p1 -e 'show master status'
mysql: [Warning] Using a password on the command line interface can be insecure.
+------------------+----------+--------------+------------------+-------------------+
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+------------------+----------+--------------+------------------+-------------------+
| mysql-bin.000001 | 154 | | | |
+------------------+----------+--------------+------------------+-------------------+
3.2.4 配置从数据库
[root@windfantasy ~]# vim /etc/my.cnf
[mysqld]
basedir = /usr/local/mysql
datadir = /opt/data
socket = /tmp/mysql.sock
port = 3306
pid-file = /opt/data/mysql.pid
user = mysql
skip-name-resolve
server-id=2 //设置从库的唯一标识符，从库的server-id值必须大于主库的该值
relay-log=mysql-relay-bin //启用中继日志relay-log
配置并启动主从复制
[root@windfantasy ~]# mysql -uroot -p1
mysql> change master to master_host='192.168.43.129',master_user='wml',master_password='1',master_log_file='mysql-bin.000001',master_log_pos=154;
Query OK, 0 rows affected, 2 warnings (0.05 sec)
mysql> start slave;
Query OK, 0 rows affected (0.00 sec)
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.43.129
Master_User: wml
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 154
Relay_Log_File: mysql-relay-bin.000005
Relay_Log_Pos: 367
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes //此处需为yes
Slave_SQL_Running: Yes //此处需为yes
...
3.2.5 测试验证
在主服务器的classmate库的class1表中插入数据：
mysql> use classmate
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
mysql> select * from class1;
Empty set (0.00 sec)
mysql> insert class1 values(1,'wind',26),(2,'fantasy',25);
Query OK, 2 rows affected (0.04 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> select * from class1;
+----+---------+------+
| id | name | age |
+----+---------+------+
| 1 | wind | 26 |
| 2 | fantasy | 25 |
+----+---------+------+
2 rows in set (0.00 sec)
在从数据库中查看数据是否同步：
mysql> use classmate
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from class1;
+----+---------+------+
| id | name | age |
+----+---------+------+
| 1 | wind | 26 |
| 2 | fantasy | 25 |
+----+---------+------+
2 rows in set (0.00 sec)
3.2.6 使用zabbix监控mysql的主从状态
编辑脚本，当mysql主从正在运行时，输出0；当mysql主从关闭时，输出1
在家目录下编辑配置文件，使mysql命令免登陆，避免弹出报警
[root@windfantasy ~]# cat .my.cnf
[client]
user=root
password=1
[root@windfantasy ~]# cat mysql_sm_state.sh
#!/bin/bash
a=$(mysql -e 'show slave status\G'|grep Slave_SQL_Running:|awk -F: '{print $2}')
b=$(mysql -e 'show slave status\G'|grep Slave_IO_Running:|awk -F: '{print $2}')
if [ $a == Yes -a $b == Yes ];then
echo 0
else
echo 1
fi
[root@windfantasy ~]# chmod +x mysql_sm_state.sh
[root@windfantasy ~]# chown zabbix.zabbix mysql_sm_state.sh
编辑mysql_agentd配置文件，启用自定义功能；然后指定脚本的位置和key值；重启服务
[root@windfantasy ~]# vim /usr/local/etc/zabbix_agentd.conf
UnsafeUserParameters=1
UserParameter=check_mysql_sm_state,/root/mysql_sm_state.sh
[root@windfantasy ~]# pkill zabbix
[root@windfantasy ~]# zabbix_agentd
进入到zabbix页面，创建监控项
配置触发器
手动停止mysql的主从服务，发现zabbix成功报警并顺利发送告警邮件
[root@windfantasy ~]# mysql -e 'stop slave'
3.2.7 使用zabbix监控mysql主从数据传输的延迟
编辑脚本，当不存在延迟时，输出0；当存在延迟时，输出延迟的具体数字
[root@windfantasy ~]# touch mysql_sm_delay.sh
[root@windfantasy ~]# chmod +x mysql_sm_
chmod: cannot access ‘mysql_sm_’: No such file or directory
[root@windfantasy ~]# chmod +x mysql_sm_delay.sh
[root@windfantasy ~]# chown zabbix.zabbix mysql_sm_delay.sh
编辑mysql-agentd，启用自定义功能，指定key值和脚本路径;重启服务
[root@windfantasy ~]# vim /usr/local/etc/zabbix_agentd.conf
UnsafeUserParameters=1
serParameter=check_mysql_sm_delay,/root/mysql_sm_delay.sh
[root@windfantasy ~]# pkill zabbix
[root@windfantasy ~]# zabbix_agentd
进入zabbix界面，新建监控项
新建触发器
4.GTID模式
4.1简介
GTID模式是MySQL-5.6.2开始支持，MySQL-5.6.10后完善，GTID 分成两部分，一部分是服务的UUid,UUID保存在mysql数据目录的auto.cnf文件中,
这是一个非常重要的文件，不能删除，这一部分是不会变的。另外一部分就是事务ID了，随着事务的增加，值一次递增,如下图
| File | Position | Binlog_Do_DB | Binlog_Ignore_DB | Executed_Gtid_Set |
+---------------+----------+--------------+------------------+--------------------------------------------+
| binlog.000016 | 1562 | | | ad3690ff-bc92-11e8-b175-005056ae0385:1-45 |
+---------------+----------+--------------+------------------+--------------------------------------------+
GTID:ad3690ff-bc92-11e8-b175-005056ae0385:1-45
UUID:ad3690ff-bc92-11e8-b175-005056ae0385
transactionId:1-362
GTID:724afcc2-29d6-11e4-9902-000c290c0121:1-362
UUID:724afcc2-29d6-11e4-9902-000c290c0121
transactionId:1-362
在整个复制架构中GTID 是不变化的,即使在多个连环主从中也不会变。
例如：ServerA —>ServerB ---->ServerC
GTID从在ServerA ,ServerB,ServerC 中都是一样的。
4.2工作原理
1、master更新数据时，会在事务前产生GTID，一同记录到binlog日志中。
2、slave端的i/o 线程将变更的binlog，写入到本地的relay log中。
3、sql线程从relay log中获取GTID，然后对比slave端的binlog是否有记录。
4、如果有记录，说明该GTID的事务已经执行，slave会忽略。
5、如果没有记录，slave就会从relay log中执行该GTID的事务，并记录到binlog。
6、在解析过程中会判断是否有主键，如果没有就用二级索引，如果没有就用全部扫描。
4.3相比传统模式的优点
1.一个事务对应一个唯一ID，一个GTID在一个服务器上只会执行一次
2.GTID是用来代替传统复制的方法，GTID复制与普通复制模式的最大不同就是不需要指定二进制文件名和位置
3.减少手工干预和降低服务故障时间，当主机挂了之后通过软件从众多的备机中提升一台备机为主机
那么GTID复制是怎么实现自动同步，自动对应位置的呢？
例如：ServerC <-----ServerA ----> ServerB
主机ServerA
备机:ServerB,ServerC
当主机ServerA 挂了之后 ，此时ServerB执行完了所有从ServerA 传过来的事务，
ServerC 延时一点。这个时候需要把 ServerB 提升为主机 ，Server C 继续为备机。
当ServerC 链接ServerC 之后,首先在自己的二进制文件中找到从ServerA 传过来的最新的GTID，
然后将这个GTID 发送到ServerB ,ServerB 获得这个GTID之后,就开始从这个GTID的下一个GTID
开始发送事务给ServerC。这种自我寻找复制位置的模式减少事务丢失的可能性以及故障恢复的时间。
4.4GTID的限制
1.不支持非事务引擎
2.不支持create table … select 语句复制(主库直接报错)
原理：（ 会生成两个sql，一个是DDL创建表SQL，一个是insert into 插入数据的sql。
由于DDL会导致自动提交，所以这个sql至少需要两个GTID，但是GTID模式下，只能给这个sql生成一个GTID ）
3.不允许一个SQL同时更新一个事务引擎表和非事务引擎表
4.在一个复制组中，必须要求统一开启GTID或者是关闭GTID
5.开启GTID需要重启（5.7除外）
6.开启GTID后，就不再使用原来的传统复制方式
7.对于create temporary table 和 drop temporary table语句不支持
8.不支持sql_slave_skip_counter
4.5配置
将enforce_gtid_consistency设置为warn
如果出现GTID不兼容的语句用法，会在error log记录相关信息
mysql> set global enforce_gtid_consistency=warn;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%enforce_gtid_consistency%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | WARN |
+--------------------------+-------+
1 row in set (0.10 sec)
调整enforce_gtid_consistency=on
mysql> set global enforce_gtid_consistency=on;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%enforce_gtid_consistency%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.00 sec)
设置 global gtid_mode=on;
mysql> set global gtid_mode=off_permissive;
Query OK, 0 rows affected (0.12 sec)
mysql> set global gtid_mode=on_permissive;
Query OK, 0 rows affected (0.13 sec)
mysql> set global gtid_mode=on;
Query OK, 0 rows affected (0.04 sec)
mysql> show variables like '%gtid_mode%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.01 sec)
从
mysql> set global enforce_gtid_consistency=warn;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%enforce_gtid_consistency%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | WARN |
+--------------------------+-------+
1 row in set (0.02 sec)
mysql> set global enforce_gtid_consistency=on;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%enforce_gtid_consistency%';
+--------------------------+-------+
| Variable_name | Value |
+--------------------------+-------+
| enforce_gtid_consistency | ON |
+--------------------------+-------+
1 row in set (0.01 sec)
mysql> set global gtid_mode=off_permissive;
Query OK, 0 rows affected (0.00 sec)
mysql> set global gtid_mode=on_permissive;
Query OK, 0 rows affected (0.00 sec)
mysql> set global gtid_mode=on;
Query OK, 0 rows affected (0.00 sec)
mysql> show variables like '%gtid_mode%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| gtid_mode | ON |
+---------------+-------+
1 row in set (0.01 sec)
查看从库的参数Ongoing_anonymous_transaction_count参数值是否为0，如果为0，意味着没有等待的事务，可以直接进行后面的操作
mysql> show global status like '%ongoing_anonymous%';
+-------------------------------------+-------+
| Variable_name | Value |
+-------------------------------------+-------+
| Ongoing_anonymous_transaction_count | 0 |
+-------------------------------------+-------+
1 row in set (0.00 sec)
核对GTID相关参数状态
mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| Variable_name | Value |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_next | AUTOMATIC |
| gtid_owned | |
| gtid_purged | |
| session_track_gtids | OFF |
+----------------------------------+-----------+
8 rows in set (0.00 sec)
执行stop slave，把传统的复制停掉;
mysql> stop slave
-> ;
Query OK, 0 rows affected (0.03 sec)
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: 192.168.43.129
Master_User: wml
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 4231210
Relay_Log_File: mysql-relay-bin.000012
Relay_Log_Pos: 320
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: No
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 4231210
Relay_Log_Space: 527
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'Cannot replicate anonymous transaction when @@GLOBAL.GTID_MODE = ON, at file ./mysql-bin.000004, position 4231210.; the first event 'mysql-bin.000004' at 4231210, the last event read from './mysql-bin.000004' at 4231275, the last byte read from './mysql-bin.000004' at 4231275.'
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: ec5b1b61-367c-11e9-91d1-000c293f9dbd
Master_Info_File: /opt/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp: 190227 18:57:29
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
然后执行change master to master_auto_position=1
mysql> change master to master_auto_position=1;
Query OK, 0 rows affected (0.09 sec)
开启复制状态
mysql> start slave;
Query OK, 0 rows affected (0.01 sec)
mysql> show slave status\G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.43.129
Master_User: wml
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000008
Read_Master_Log_Pos: 52994
Relay_Log_File: mysql-relay-bin.000005
Relay_Log_Pos: 53207
Relay_Master_Log_File: mysql-bin.000008
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 52994
Relay_Log_Space: 221268
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: ec5b1b61-367c-11e9-91d1-000c293f9dbd
Master_Info_File: /opt/data/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: ec5b1b61-367c-11e9-91d1-000c293f9dbd:1-688
Executed_Gtid_Set: ec5b1b61-367c-11e9-91d1-000c293f9dbd:1-688
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
主：
核对GTID相关参数状态
mysql> show variables like '%gtid%';
+----------------------------------+-----------+
| Variable_name | Value |
+----------------------------------+-----------+
| binlog_gtid_simple_recovery | ON |
| enforce_gtid_consistency | ON |
| gtid_executed_compression_period | 1000 |
| gtid_mode | ON |
| gtid_next | AUTOMATIC |
| gtid_owned | |
| gtid_purged | |
| session_track_gtids | OFF |
+----------------------------------+-----------+
8 rows in set (0.00 sec)
向表中插入新数据，验证是否切换成功
mysql> use classmate;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> insert class1 value('3','wml',26);
Query OK, 1 row affected (0.05 sec)
mysql> select * from class1;
+----+---------+------+
| id | name | age |
+----+---------+------+
| 1 | wind | 26 |
| 2 | fantasy | 25 |
| 3 | wml | 26 |
+----+---------+------+
3 rows in set (0.01 sec)
从：
发现从库的的数据同步更新，且
mysql> use classmate;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from classmate;
ERROR 1146 (42S02): Table 'classmate.classmate' doesn't exist
mysql> select * from class1;
+----+---------+------+
| id | name | age |
+----+---------+------+
| 1 | wind | 26 |
| 2 | fantasy | 25 |
| 3 | wml | 26 |
+----+---------+------+
3 rows in set (0.00 sec)