Data verification tool pt-table-checksum

background

pt-table-checksum 是 Percona-Toolkit 的组件之一,用于检测 MySQL 主、从库的数据是否一致。

Write in front
内容一部分来自 作者:耿进 爱可生 DBA 团队成员,负责公司 DMP 产品的运维和客户 MySQL 问题的处理。对数据库技术有着浓厚的兴趣。你见过凌晨四点 MySQL 的 error 吗?
link

principle

在主库执行基于 statement 的 SQL 语句来生成主库数据块的checksum,
把相同的 SQL 语句传递到从库执行,并在从库上计算相同数据块的 checksum,
最后,比较主从库上相同数据块的 checksum 值,由此判断主从数据是否一致。

复制筛选器是危险的,因为 pt-table-checksum 执行的查询可能与它们发生冲突,并导致复制失败。

**参数innodb_lock_wait_timeout **

For example, pt-table-checksum sets its session-level innodb_lock_wait_timeout to 1 second, so that if there is a lock wait, it will become a victim instead of causing other queries to time out.

–set-vars innodb_lock_wait_timeout=120

By default, if there are more than 25 concurrently executing queries, pt-table-checksum will pause. You should probably use the --max-load option to set a reasonable value for the server.

若无主键做校验和修复对性能影响非常重,数据校验和修复最重要的约束便是主健,无主键或唯一索引,将导致修复不成功。

operating

user

必须保证一个账号能同时在主库和从库都有对应的访问权限
mysql> GRANT SELECT, PROCESS, SUPER, REPLICATION SLAVE ON *.* TO 'root'@'%' IDENTIFIED BY '******';
mysql> flush privileges;

--Nocheck-binlog-format: Do not check the log format. This option is very important for replication in ROW mode, because pt-table-checksum will set binlog_format=STATEMENT on the Master and Slave (to ensure that the checksum SQL is also executed from the database). MySQL limits the slave database cannot be set, so if the row is replicated When the slave library is used as the master library to copy a new slave library (A->B->C), the checksums data of B will not be transmitted.

--Replicate= Specify which database table the checksum calculation result is stored in, if not specified, the default is percona.checksums

–nocheck-replication-filters 默认值:是
–recursion-method=processlist
–replicate=test.checksums
–databases=test
–tables=t01

Additional analysis:
--no-check-binlog-format does not check the copied binlog mode.
--Nocheck-replication-filters Do not check replication filters, it is recommended to enable.
--Replicate=test.checksums The check results are written into the checksums table of the test library.
--Databases=db1 --tables=tb1 Check the tb1 table in the db1 library, if there is no parameter, check the whole database and the whole table.
-h 192.168.XXX.XX -P 3306 Main library IP address and 3306 port.
-u'hangxing' -p'PASSOWRD' verify account password.
--Recursion-method="processlist" Use the processlist method to discover slave libraries.

The processlist method is the default method because it is not reliable. However, if the server uses a non-standard port (not 3306), this method will become the default method because it will work better in this case. SHOW SLAVE HOSTShosts
The hosts method needs to be configured with the copy report_host, report_port, etc.

–Ask-pass prompt for password
–quiet only for connection verification, no output

–[No]check-slave-tables
default value: yes; group: security
Check whether the table on the slave server exists and has all checksums –columns. Missing tables or tables without all checksums on the slave server-columns may cause the tool to interrupt replication while trying to check for differences. Disable this check only if you are aware of the risk and ensure that all tables on all slave servers exist and are the same as the master server.

Perform master-slave data consistency check

检查整个test库
pt-table-checksum --nocheck-binlog-format --nocheck-replication-filters --recursion-method=processlist --replicate=test.checksums  --databases=test  --host=192.168.66.133 --user=msandbox --password=123456 --port=17261

检查test110表
pt-table-checksum --nocheck-binlog-format --nocheck-replication-filters --recursion-method=processlist --replicate=test.checksums  --databases=test  --tables=test110 --host=192.168.66.133 --user=msandbox --password=123456 --port=17261
检查所有表
pt-table-checksum --nocheck-binlog-format --nocheck-plan --nocheck-replication-filters --replicate=test.checksums  --defaults-file=/usr/local/sandboxes/msb_5_7_26/master/my.sandbox.cnf --socket=/tmp/mysql_sandbox17261.sock --user=msandbox --password='123456' --recursion-method="processlist"

以上操作会在主库建立--replicate=test.checksums 表,可以根据需要清理。

verification

DIFFS列 表示主从不同的行数
Insert picture description here
Insert picture description here
This article explains that the main technical content comes from the sharing of Internet technology giants, as well as some self-processing (only for the role of annotations). If related questions, please leave a message after the confirmation, the implementation of infringement will be deleted

Guess you like

Origin blog.csdn.net/baidu_34007305/article/details/112004728