MySQL case combat-MySQL database log management

Preface

The MySQL service also has many types of logs. Logs are involved in database data backup, data recovery, troubleshooting, and server tuning. Next, we talk about MySQL log management in detail

This environment is based on Centos 7.8 system to build MySQL-5.7.14 for
specific construction, please refer to MySQL-5.7.14 environment construction

---

MySQL log

Log file	Type of information in the log file
Error log	Record problems that occur when starting, running, or stopping
Query log	Record established client connections and executed statements
Binary log	Record all change data statements. Mainly used for replication and point-in-time recovery
Slow log	Log all queries whose execution time exceeds log_query_time seconds or queries that must not use indexes
Transaction log	Record logs generated during execution of supported storage engines such as InnoDB

1. Error log

The error log mainly records the following types of logs

• Information during server startup and shutdown
• Error message during server operation
• Information generated by the event scheduler for a time
• Information generated when the slave server process is started on the slave server

Manage error logs

The MySQL error log is stored in the datadir (data directory) directory by default,
but we can also manually set the log file storage path

1. Log default storage path path

[root@mysql-yum ~]# vim /etc/my.cnf
log-error=/var/log/mysqld.log
[root@mysql-yum ~]# systemctl restart mysqld

mysql> show global variables like '%log_error%';
+---------------------+---------------------+
| Variable_name       | Value               |
+---------------------+---------------------+
| binlog_error_action | ABORT_SERVER        |
| log_error           | /var/log/mysqld.log |
| log_error_verbosity | 3                   |
+---------------------+---------------------+
3 rows in set (0.00 sec)

2. The log storage path is not specified

[root@mysql-yum ~]# vim /etc/my.cnf
log-error
[root@mysql-yum ~]# systemctl restart mysqld

mysql> show global variables like '%log_error%';
+---------------------+----------------------------+
| Variable_name       | Value                      |
+---------------------+----------------------------+
| binlog_error_action | ABORT_SERVER               |
| log_error           | /var/run/mysqld/mysqld.err |
| log_error_verbosity | 3                          |
+---------------------+----------------------------+
3 rows in set (0.00 sec)

3. Set the storage path

[root@mysql-yum ~]# vim /etc/my.cn
log-error=mysql-yum
[root@mysql-yum ~]# systemctl restart mysqld

mysql> show global variables like '%log_error%';
+---------------------+-----------------+
| Variable_name       | Value           |
+---------------------+-----------------+
| binlog_error_action | ABORT_SERVER    |
| log_error           | ./mysql-yum.err |
| log_error_verbosity | 3               |
+---------------------+-----------------+
3 rows in set (0.00 sec)

Note: MySQL error log is stored in the data directory

2. Query log

Query log: record established client connections and executed statements

Open query log

[root@mysql-yum ~]# vim /etc/my.cnf
general_log=on
[root@mysql-yum ~]# systemctl restart mysqld

View log status

mysql> show global variables like '%general_log%';
+------------------+------------------------------+
| Variable_name    | Value                        |
+------------------+------------------------------+
| general_log      | ON                           |
| general_log_file | /var/lib/mysql/mysql-yum.log |
+------------------+------------------------------+
2 rows in set (0.00 sec)

mysql> show global variables like '%log_output%';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| log_output    | FILE  |
+---------------+-------+
1 row in set (0.00 sec)

# 执行查询语句
mysql> show databases;
mysql> select database();

View log records

Three, slow query log

Slow log: record all queries whose execution time exceeds log_query_time seconds or queries that must not use the index
Function: mainly used for server tuning

Set slow log start and query time

mysql> set global slow_query_log=on;
Query OK, 0 rows affected (0.22 sec)

mysql> set global long_query_time=3;
Query OK, 0 rows affected (0.00 sec)


# 查看日志设定
mysql> show global variables like '%slow_query_log%';
+---------------------+-----------------------------------+
| Variable_name       | Value                             |
+---------------------+-----------------------------------+
| slow_query_log      | ON                                |
| slow_query_log_file | /var/lib/mysql/mysql-yum-slow.log |
+---------------------+-----------------------------------+
2 rows in set (0.00 sec)

mysql> show global variables like '%long_query_time%';
+-----------------+----------+
| Variable_name   | Value    |
+-----------------+----------+
| long_query_time | 3.000000 |
+-----------------+----------+
1 row in set (0.00 sec)

Fourth, the binary log

What is the binary log?

• The binary log contains all statements that have updated data or have potentially updated data (for example, a DELETE that does not match any row)
• Statements are stored in the form of "events", which describe data changes. The binary log also contains information about the execution time of each statement that updates the database. It does not contain statements that do not modify any data.

The role of the binary log?

• The main purpose of the binary log is to update the database as much as possible (point-in-time recovery) when the database is faulty, because the binary log contains all the updates made after the backup
• The binary log is also used to record all the statements that will be sent to the slave server on the master replication server

In the production environment, we often adopt: execute statements and record execution results, record binary logs

Open log_bin

[root@mysql-yum ~]# systemctl restart mysqld
log_bin=mysql-bin
server_id=10
[root@mysql-yum ~]# systemctl restart mysqld

View log_bin open status

mysql> show global variables like '%log_bin%';
+---------------------------------+--------------------------------+
| Variable_name                   | Value                          |
+---------------------------------+--------------------------------+
| log_bin                         | ON                             |
| log_bin_basename                | /var/lib/mysql/mysql-bin       |
| log_bin_index                   | /var/lib/mysql/mysql-bin.index |
| log_bin_trust_function_creators | OFF                            |
| log_bin_use_v1_row_events       | OFF                            |
+---------------------------------+--------------------------------+
5 rows in set (0.00 sec)

# 查看二进制日志文件
mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |       154 |
+------------------+-----------+
1 row in set (0.00 sec)

# 查看二进制日志文件内容
mysql> mysql> show binlog events in 'mysql-bin.000001';
+------------------+-----+----------------+-----------+-------------+---------------------------------------+
| Log_name         | Pos | Event_type     | Server_id | End_log_pos | Info                                  |
+------------------+-----+----------------+-----------+-------------+---------------------------------------+
| mysql-bin.000001 |   4 | Format_desc    |        10 |         123 | Server ver: 5.7.14-log, Binlog ver: 4 |
| mysql-bin.000001 | 123 | Previous_gtids |        10 |         154 |                                       |
+------------------+-----+----------------+-----------+-------------+---------------------------------------+
2 rows in set (0.00 sec)

View log_bin content

[root@mysql-yum ~]# mysqlbinlog /var/lib/mysql/mysql-bin.000001
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=1*/;
/*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
DELIMITER /*!*/;
# at 4
#210129 11:06:37 server id 10  end_log_pos 123 CRC32 0x323c980c 	Start: binlog v 4, server v 5.7.14-log created 210129 11:06:37 at startup
# Warning: this binlog is either in use or was not closed properly.
ROLLBACK/*!*/;
BINLOG '
vXsTYA8KAAAAdwAAAHsAAAABAAQANS43LjE0LWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAC9exNgEzgNAAgAEgAEBAQEEgAAXwAEGggAAAAICAgCAAAACgoKKioAEjQA
AQyYPDI=
'/*!*/;
# at 123
#210129 11:06:37 server id 10  end_log_pos 154 CRC32 0xb271a2b4 	Previous-GTIDs
# [empty]
SET @@SESSION.GTID_NEXT= 'AUTOMATIC' /* added by mysqlbinlog */ /*!*/;
DELIMITER ;
# End of log file
/*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
/*!50530 SET @@SESSION.PSEUDO_SLAVE_MODE=0*/;

Main: The higher version of MySQL, the statement in log_bin has been encrypted, and it needs to be decoded to view
When log_bin is enabled, the server_id number needs to be specified, and this value is unique in the MySQL server cluster

Delete log_bin

# 多西刷新log_bin
mysql> flush logs;
Query OK, 0 rows affected (0.00 sec)

# 查看所有的log_bin
mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |       201 |
| mysql-bin.000002 |       201 |
| mysql-bin.000003 |       201 |
| mysql-bin.000004 |       201 |
| mysql-bin.000005 |       154 |
+------------------+-----------+
5 rows in set (0.00 sec)


# 删除日志
---方法一：按照日志名字删除
mysql> purge binary logs to 'mysql-bin.000003';
Query OK, 0 rows affected (0.00 sec)

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000003 |       201 |
| mysql-bin.000004 |       201 |
| mysql-bin.000005 |       154 |
+------------------+-----------+
3 rows in set (0.00 sec)

---方法二：按照日志删除
mysql> purge binary logs before '2021-01-29 11:24:37';
Query OK, 0 rows affected (0.00 sec)

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000004 |       201 |
| mysql-bin.000005 |       154 |
+------------------+-----------+
2 rows in set (0.00 sec)

---方法三：删除所有日志，重新从第一编号开始，重新记录日志
mysql> reset master;
Query OK, 0 rows affected (0.00 sec)

mysql> show binary logs;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000001 |       154 |
+------------------+-----------+
1 row in set (0.01 sec)

Note: Binary log files cannot be deleted directly. If you use commands such as rm to delete log files directly, the database may crash.
1. Delete with purge command 2. Delete
with reset command