我是如何稳如狗删除360w数据的？

root@localhost[(none)]> 
root@localhost[(none)]> use ecrmdb;
Database changed
root@localhost[ecrmdb]> 
root@localhost[ecrmdb]> 
root@localhost[ecrmdb]> select count(*) from api_goldenf_gflc_gameinfo where bettime BETWEEN '2019-11-13 00:00:00' and '2019-11-14 00:00:00';
+----------+
| count(*) |
+----------+
|  3380000 |
+----------+
1 row in set (0.82 sec)


删除 3380000数据

1：mysqldump备份，从库执行：


mysqldump -uroot -p -P3388  -S /data/db/mysql3388/mysql3388.sock ecrmdb api_goldenf_gflc_gameinfo -w "bettime BETWEEN '2019-11-13 00:00:00' and '2019-11-14 00:00:00'" --skip-opt --set-gtid-purged=off > bak_api_goldenf_gflc_gameinfo_20191115.sql


2:删除。主库执行：

pt-archiver --source h=192.168.30.41,P=3388,u=localadmin,p=xxx,D=ecrmdb,t=api_goldenf_gflc_gameinfo --where 'bettime BETWEEN  "2019-11-13 00:00:00" and "2019-11-14 00:00:00"' --progress 10000 --limit 5000 --txn-size 5000  --no-check-charset   --sleep=1 --purge --statistics

 

1:备份1（慢稳）：
elasticdump --input=http://192.168.31.4:9200/game_record_2019_11 --output=api_goldenf_gflc_gameinfo3380000_2019115.json --searchBody '{ "query": { "bool": { "must": [ { "term": { "api_name.keyword": "GFLC" } }, { "term": { "flag": 1 } },{"range": { "betTime": { "gte": "2019-11-13T00:00:00+08:00", "lte": "2019-11-13T23:59:59+08:00" } } } ] } } }' 
  备份2（快但可能出现慢查询）

input {
elasticsearch {
hosts => ["192.168.31.5:9200"]
index => "game_record_2019_11"
query => '{ "query": { "bool": { "must": [ { "term": { "api_name.keyword": "GFLC" } }, { "term": { "flag": 1 } },{"range": { "betTime": { "gte": "2019-11-13T00:00:00+08:00", "lte": "2019-11-13T23:59:59+08:00" } } } ] } } }'
size => 15000
scroll => "2m"
codec => "json"
docinfo => true
}
}

output {
stdout { codec => rubydebug }
file {
codec => "json"
path => ["/data/backup/3380000.json"]
}
}

2: 删除：


GET game_record_2019_11/_delete_by_query
{
  "query": {
    "bool": {
      "must": [
        {
          "term": {
            "api_name.keyword": "GFLC"
          }
        },
        {
          "term": {
            "flag": 1
          }
        },
       
        {"range": {
          "betTime": {
            "gte": "2019-11-13T00:00:00+08:00",
            "lte": "2019-11-13T23:59:59+08:00"
          }
        }
        }
      ]
    }
  }
}