1. Scene description:
In many cases, the data cannot be accessed because the MySQL database cannot be started, but the application data is usually not lost, but other files such as the system table space are damaged, or MySQL bugs are encountered.
If there is no backup at this time, many people think that the data is lost, but in fact the data can be saved most of the time.
For the table space of the MyISAM engine, just copy the corresponding data file directly to a new database, and the data can be restored.
For the database tablespace of the InnoDB engine, the data can be rescued by transferring the tablespace. The
premise is that MySQL has enabled the parameter innodb_file_per_table = 1 independent tablespace file
When the MySQL data tablespace file ibdata file is damaged or is not modified or deleted, the MySQL service fails to restart. At the same time, the MySQL data is not backed up in time. How to retrieve as much MySQL data as possible at this time? ?
2. Case demonstration:
2.1. Before confirming MySQL failure, how many records does each table in the library have?
for n in `mysql -e "use db_bbs;show tables;"|sed '1d'`;do echo $n; mysql -e "use db_bbs;select count(*) from $n;";done >test.txtBefore confirming the MySQL failure, how many tables are there in the library:
一共是39张表：
[root@10-10-127-11 ~]# mysql -e "use db_bbs;show tables;"|sed '1d'|wc -l
392.2. Simulate delete ibdata failure:
Delete the ibdata file (this is prohibited in the production environment)
innodb_force_recovery = 6 Use MySQL's mandatory startup parameters to start the MySQL service at this time, but it is no longer helpful. Because it is the data tablespace file ibdata file is deleted. So the MySQL service cannot be started at this time.
Because of the innodb engine used by MySQL. And the independent table space parameter innodb_file_per_table = 1 is turned on. So at this time, you can use the transmission table space to rescue the data.
2.3. Ways to retrieve data:
First, create the missing table structure:
First install mysql-utilities on the failed MySQL server.
yum -y install mysql-utilities
Use mysqlfrm to retrieve the table creation statement from the .frm file:
Analyze a .frm file to generate a statement to build a table
mysqlfrm --diagnostic
[root@test02 db_bbs]# mysqlfrm --diagnostic /data/mysql/data/db_bbs/t_admin.frm |grep -v "^#"
CREATE TABLE `db_bbs`.`t_admin` (
`f_id` int(4) NOT NULL AUTO_INCREMENT,
`f_type` tinyint(1) NOT NULL,
`f_username` varchar(80) NOT NULL,
`f_password` varchar(80) NOT NULL,
`f_nick_name` varchar(80) NOT NULL,
`f_real_name` varchar(80) NOT NULL,
`f_create_time` bigint(4) NOT NULL,
`f_update_time` bigint(4) NOT NULL,
`f_last_login_time` bigint(4) DEFAULT NULL,
`f_last_login_ip` varchar(80) DEFAULT NULL,
`f_status` tinyint(1) NOT NULL,
PRIMARY KEY `PRIMARY` (`f_id`) USING BTREE
) ENGINE=InnoDB ROW_FORMAT = 2;Import all the table creation statements into the /tmp/create.sql file:
[root@test02 ~]# cd /data/mysql/data/db_bbs/
[root@test02 db_bbs]# for n in `ls -l /data/mysql/data/db_bbs/*.frm|awk -F '/' '{print $NF}'|xargs -n 40`;do mysqlfrm --diagnostic $n|grep -v "^#" >>/tmp/create.sql;done Import the production table creation statement into the new MySQL instance database:
[root@10-10-127-11 ~]# mysql db_bbs < create.sql
ERROR 1064 (42000) at line 2: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '5' at line 10
原因是获取到的建表sql中包含了ROW_FORMAT = 2 这样的参数导致的###Remove the characters that contain ROW_FORMAT = 2, ROW_FORMAT = 5 in the table building statement. Re-import the table building statement###
The batch replacement commands are as follows:
cat /tmp/create.sql|sed -e 's/ENGINE=InnoDB ROW_FORMAT = 2;/ENGINE=InnoDB ;/g'|grep ROW_FORMAT |uniq -c
cat /tmp/create.sql|sed -e 's/ENGINE=InnoDB ROW_FORMAT = 5;/ENGINE=InnoDB ;/g'|grep ROW_FORMAT |uniq -c
sed -i 's/ENGINE=InnoDB ROW_FORMAT = 2;/ENGINE=InnoDB ;/g' /tmp/create.sql
sed -i 's/ENGINE=InnoDB ROW_FORMAT = 5;/ENGINE=InnoDB ;/g' /tmp/create.sql
cat /tmp/create.sql|grep ROW_FORMAT |uniq -c Export the table creation statement to the new MySQL instance db_bbs library and the error field is too long:
[root@10-10-127-11 ~]# mysql db_bbs -f < create.sql
ERROR 1074 (42000) at line 232: Column length too big for column 'f_content' (max = 16383); use BLOB or TEXT instead
ERROR 1074 (42000) at line 299: Column length too big for column 'f_desc' (max = 16383); use BLOB or TEXT instead
ERROR 1074 (42000) at line 365: Column length too big for column 'f_body_image' (max = 16383); use BLOB or TEXT instead
ERROR 1074 (42000) at line 406: Column length too big for column 'f_content' (max = 16383); use BLOB or TEXT instead
ERROR 1074 (42000) at line 433: Column length too big for column 'f_summary' (max = 16383); use BLOB or TEXT insteadAfter modifying the field length type, re-import the table-building sql into the new MySQL database
[root@10-10-127-11 ~]# mysql db_bbs -f < create.sql
[root@10-10-127-11 ~]#
[root@10-10-127-11 ~]#
[root@10-10-127-11 ~]# mysql -e "use db_bbs;show tables;"|sed '1d'|wc -l
39###Discard the .ibd file that does not include data of the newly created MySQL instance, and then import the .idb file of the failed database ###
Discard the data .ibd file of the newly created library:
mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a DISCARD TABLESPACE" ;done[root@10-10-127-11 db_bbs]# ll *.ibd|wc -l
39
[root@10-10-127-11 db_bbs]# mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a DISCARD TABLESPACE" ;done
[root@10-10-127-11 db_bbs]# ll *.ibd|wc -l
ls: cannot access *.ibd: No such file or directory0
* You can see that all .idb files have been discarded. Then copy the old .ibd files with data to the ./data/db_bbs/ directory of this new MySQL instance, don’t forget to change the owner: chown mysql. , and then import these data files to the database** .
[root@test02 db_bbs]# scp *.ibd [email protected]:/data/mysql/data/db_bbs/
[email protected]'s password:
browse_record.ibd 100% 100MB 50.0MB/s 00:02
t_admin.ibd
........
........[root@10-10-127-11 db_bbs]# ll *.ibd|wc -l
39
[root@10-10-127-11 db_bbs]# ll *.ibd
-rw-r----- 1 root root 104857600 Mar 14 21:56 browse_record.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_admin.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_anonymous_code.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_apply.ibd
-rw-r----- 1 root root 9437184 Mar 14 21:56 t_attach.ibd
-rw-r----- 1 root root 147456 Mar 14 21:56 t_banner.ibd
-rw-r----- 1 root root 163840 Mar 14 21:56 t_banner_log.ibd
-rw-r----- 1 root root 114688 Mar 14 21:56 t_black_ip.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_black_user.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_block_userbaseinfo.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_collect.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_country_code.ibd
-rw-r----- 1 root root 163840 Mar 14 21:56 t_ct_goods.ibd
-rw-r----- 1 root root 131072 Mar 14 21:56 t_ct_goods_record.ibd
-rw-r----- 1 root root 9437184 Mar 14 21:56 t_ct_integral.ibd
-rw-r----- 1 root root 46137344 Mar 14 21:56 t_ct_integral_record.ibd
-rw-r----- 1 root root 27262976 Mar 14 21:56 t_ct_news.ibd
-rw-r----- 1 root root 9437184 Mar 14 21:56 t_ct_order.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_feedback.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_lexicon.ibd
-rw-r----- 1 root root 327680 Mar 14 21:56 t_logs.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_manage.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_module.ibd
-rw-r----- 1 root root 9437184 Mar 14 21:56 t_post_extend.ibd
-rw-r----- 1 root root 12582912 Mar 14 21:56 t_post.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_post_video.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_realtime_message.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_recommend.ibd
-rw-r----- 1 root root 46137344 Mar 14 21:56 t_reply.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_reward.ibd
-rw-r----- 1 root root 196608 Mar 14 21:56 t_sensitive_word.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_system_message.ibd
-rw-r----- 1 root root 9437184 Mar 14 21:56 t_userbaseinfo.ibd
-rw-r----- 1 root root 344064 Mar 14 21:56 t_userextendinfo.ibd
-rw-r----- 1 root root 12582912 Mar 14 21:56 t_user_health.ibd
-rw-r----- 1 root root 98304 Mar 14 21:56 t_user_message.ibd
-rw-r----- 1 root root 442368 Mar 14 21:56 t_user_read_module_log.ibd
-rw-r----- 1 root root 17825792 Mar 14 21:56 t_viewpoint.ibd
-rw-r----- 1 root root 114688 Mar 14 21:56 t_white_ip.ibd
[root@10-10-127-11 db_bbs]# chown mysql.mysql *.ibd
mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a import TABLESPACE" ;doneWhen importing the table space of each table, an individual table error occurred:
[root@10-10-127-11 db_bbs]# mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a import TABLESPACE" ;done
ERROR 1808 (HY000) at line 1: Schema mismatch (Table has ROW_TYPE_DYNAMIC row format, .ibd file has ROW_TYPE_COMPACT row format.)Check the table file, it is found that only the browse_record table is imported into the independent table space and an error is reported, which causes the data recovery of this table to fail
[root@10-10-127-11 db_bbs]# mysqlcheck -c db_bbs
db_bbs.browse_record
Warning : InnoDB: Tablespace has been discarded for table 'browse_record'
Error : Tablespace has been discarded for table 'browse_record'
error : Corrupt
db_bbs.t_admin OK
db_bbs.t_anonymous_code OK
db_bbs.t_apply OK
db_bbs.t_attach OK
db_bbs.t_banner OK
db_bbs.t_banner_log OK
db_bbs.t_black_ip OK
db_bbs.t_black_user OK
db_bbs.t_block_userbaseinfo OK
db_bbs.t_collect OK
db_bbs.t_country_code OK
db_bbs.t_ct_goods OK
db_bbs.t_ct_goods_record OK
db_bbs.t_ct_integral OK
db_bbs.t_ct_integral_record OK
db_bbs.t_ct_news OK
db_bbs.t_ct_order OK
db_bbs.t_feedback OK
db_bbs.t_lexicon OK
db_bbs.t_logs OK
db_bbs.t_manage OK
db_bbs.t_module OK
db_bbs.t_post OK
db_bbs.t_post_extend OK
db_bbs.t_post_video OK
db_bbs.t_realtime_message OK
db_bbs.t_recommend OK
db_bbs.t_reply OK
db_bbs.t_reward OK
db_bbs.t_sensitive_word OK
db_bbs.t_system_message OK
db_bbs.t_user_health OK
db_bbs.t_user_message OK
db_bbs.t_user_read_module_log OK
db_bbs.t_userbaseinfo OK
db_bbs.t_userextendinfo OK
db_bbs.t_viewpoint OK
db_bbs.t_white_ip OKThe above browse_record table restore failure error report solution is as follows:
Reference: https://blog.csdn.net/weixin_30607659/article/details/94987901
Delete the browse_record table imported into the new MySQL instance, and then execute the following table creation statement to create a new browse_record table:
CREATE TABLE `browse_record` (
`id` int(4) unsigned NOT NULL AUTO_INCREMENT,
`post_id` int(4) unsigned NOT NULL,
`user_id` int(4) unsigned NOT NULL,
`status` tinyint(1) unsigned NOT NULL,
`update_time` bigint(4) unsigned NOT NULL,
`create_time` bigint(4) unsigned NOT NULL,
PRIMARY KEY (`id`) USING BTREE,
KEY `browse` (`post_id`,`user_id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 row_format=compact;Tip: If you delete the browse_record table separately in the new MySQL instance, if the deletion fails, drop the db_bbs library on the new MySQL instance directly, re-import the table creation statement of all tables in db_bbs, and then execute the following command:
mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a DISCARD TABLESPACE" ;doneRe-import the tablespace into the new instance MySQL:
[root@10-10-127-11 db_bbs]# mysql -e "show tables from db_bbs" | grep -v Tables_in_db_bbs| while read a; do mysql -e "ALTER TABLE db_bbs.$a import TABLESPACE" ;done
[root@10-10-127-11 db_bbs]# Data repair is complete here
Verify the tables in the MySQL db_bbs library:
[root@10-10-127-11 db_bbs]# mysqlcheck -c db_bbs
db_bbs.browse_record OK
db_bbs.t_admin OK
db_bbs.t_anonymous_code OK
db_bbs.t_apply OK
db_bbs.t_attach OK
db_bbs.t_banner OK
db_bbs.t_banner_log OK
db_bbs.t_black_ip OK
db_bbs.t_black_user OK
db_bbs.t_block_userbaseinfo OK
db_bbs.t_collect OK
db_bbs.t_country_code OK
db_bbs.t_ct_goods OK
db_bbs.t_ct_goods_record OK
db_bbs.t_ct_integral OK
db_bbs.t_ct_integral_record OK
db_bbs.t_ct_news OK
db_bbs.t_ct_order OK
db_bbs.t_feedback OK
db_bbs.t_lexicon OK
db_bbs.t_logs OK
db_bbs.t_manage OK
db_bbs.t_module OK
db_bbs.t_post OK
db_bbs.t_post_extend OK
db_bbs.t_post_video OK
db_bbs.t_realtime_message OK
db_bbs.t_recommend OK
db_bbs.t_reply OK
db_bbs.t_reward OK
db_bbs.t_sensitive_word OK
db_bbs.t_system_message OK
db_bbs.t_user_health OK
db_bbs.t_user_message OK
db_bbs.t_user_read_module_log OK
db_bbs.t_userbaseinfo OK
db_bbs.t_userextendinfo OK
db_bbs.t_viewpoint OK
db_bbs.t_white_ip OK2.4. Get the table records imported into the db_bbs library of the new MySQL instance and compare it with the original library test.txt table record file
[root@10-10-127-11 ~]# for n in `mysql -e "use db_bbs;show tables;"|sed '1d'`;do echo $n; mysql -e "use db_bbs;select count(*) from $n;";done >test.txt11
和原来的库test.txt表记录文件对比。
[root@test02 ~]# vimdiff test.txt11 test.txt
表记录完全一致So far, the MySQL data is repaired.
Reference materials:
https://mp.weixin.qq.com/s/r3KTPsFay292JnO0lgTLUg
https://www.cnblogs.com/jiangxu67/p/4744283.html
https://blog.csdn.net/Sonny_alice/article/details /80198200
https://www.cnblogs.com/jiangxu67/p/4744283.html