ElasticSearch 升级过程中给涉及到的术语-1、同步刷新synced flush

Es 会跟踪每个活动分片的索引，分片在五分钟内如果没有收到任何有关索引的操作，则认为分片不活动了；这将给Es机会去减小分片的资源，同时执行一种特殊刷新，成为：同步刷新；同步刷新也是执行普通的刷新，然后给所有的分片添加一个不同的标记；

当没有索引操作时，id 标记会添加到分片上。标记可以作为一个快速的方式来检查两个分片的lucene索引一致是否一致；这种快速的id 比较主要用于数据恢复或者重启后跳过第一个也是成本最高的阶段；这种情况下，segment 不需要copy，事务日志的重演会立即执行；当id 标记和flush 一起用的时候，事物日志很可能为空，更加加速了数据的恢复；

这对于具有很多很少更新或者不变的索引的情况(如基于时间的数据)，将会更有用；查看分片是否被id 标记，用如下命令：

curl -X GET "dmp9:9201/wzp_test/_stats/commit?level=shards"

{"_shards":{"total":10,"successful":10,"failed":0},"_all":{"primaries":{},"total":{}},"indices":{"wzp_test":{"primaries":{},"total":{},"shards":{"0":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQg==","generation":12,"user_data":{"translog_uuid":"OHqJo56NQoqD0vivoRE_-w","sync_id":"AWdPVMGxhBFTwa1Fywjs","translog_generation":"1543159976461"},"num_docs":411901},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQg==","generation":12,"user_data":{"translog_uuid":"OHqJo56NQoqD0vivoRE_-w","sync_id":"AWdPVMGxhBFTwa1Fywjs","translog_generation":"1543159976461"},"num_docs":411901},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"1":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQQ==","generation":13,"user_data":
{"translog_uuid":"r5RMt3ZDS5ip1Il2cldOKA","sync_id":"AWdPVMGxhBFTwa1Fywjr","translog_generation":"1543160040431"},"num_docs":383147},"shard_path":
{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQQ==","generation":13,"user_data":{"translog_uuid":"r5RMt3ZDS5ip1Il2cldOKA","sync_id":"AWdPVMGxhBFTwa1Fywjr","translog_generation":"1543160040431"},"num_docs":383147},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"2":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAPw==","generation":13,"user_data":{"translog_uuid":"EYkaGyrTTPCtgKArFf0Qyw","sync_id":"AWdPVMGxhBFTwa1Fywjp","translog_generation":"1543159976463"},"num_docs":383110},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAPw==","generation":13,"user_data":{"translog_uuid":"EYkaGyrTTPCtgKArFf0Qyw","sync_id":"AWdPVMGxhBFTwa1Fywjp","translog_generation":"1543159976463"},"num_docs":383110},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"3":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQw==","generation":13,"user_data":{"translog_uuid":"livjpLxlRSakLFnCp8n9Jw","sync_id":"AWdPVMGxhBFTwa1Fywjo","translog_generation":"1543160040432"},"num_docs":410859},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQw==","generation":13,"user_data":{"translog_uuid":"livjpLxlRSakLFnCp8n9Jw","sync_id":"AWdPVMGxhBFTwa1Fywjo","translog_generation":"1543160040432"},"num_docs":410859},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}],"4":[{"routing":{"state":"STARTED","primary":true,"node":"8hkDNjDQRrGMWmhUFbhPrA","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQA==","generation":13,"user_data":{"translog_uuid":"US1IS1LaR5qkwlJwLLs7Vw","sync_id":"AWdPVMGxhBFTwa1Fywjq","translog_generation":"1543159976461"},"num_docs":416518},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}},{"routing":{"state":"STARTED","primary":false,"node":"aagBamD6QZWOm-GF5QPNdg","relocating_node":null},"commit":{"id":"uSW9jUn1xqqvWjl0/SiAQA==","generation":13,"user_data":{"translog_uuid":"US1IS1LaR5qkwlJwLLs7Vw","sync_id":"AWdPVMGxhBFTwa1Fywjq","translog_generation":"1543159976461"},"num_docs":416518},"shard_path":{"state_path":"/data2/es_test/dmp_test/nodes/0","data_path":"/data2/es_test/dmp_test/nodes/0","is_custom_data_path":false}}]}}}}

官方文档给的结果示例：

curl -X GET "localhost:9200/twitter/_stats/commit?level=shards"

{
   ...

   "indices": {

      "twitter": {

         "primaries": {},

         "total": {},

         "shards": {

            "0": [

               {

                  "routing": {

                     ...
                  },

                  "commit": {

                     "id": "te7zF7C4UsirqvL6jp/vUg==",

                     "generation": 2,

                     "user_data": {

                        "sync_id": "AU2VU0meX-VX2aNbEUsD" ,
                        ...
                     },

                     "num_docs": 0
                  }
               }
               ...
            ],
            ...
         }
     }
   }
}

同步刷新 API

同步刷新API 允许管理员手动刷新。这对于计划的(回滚)集群重启特别有用，您可以停止索引，并且不希望等待默认的5分钟来自动同步刷新空闲索引；

尽管很方便，但是有一些警告:

同步刷新是个尽最大努力的操作：任何正在执行索引的操作都会造成同步刷新失败；这就一位置有些分片可能刷新成功，而有些分片则刷新失败；
同步id 的标记在再次刷新是会移除掉；

这是因为刷新会替换低级别的储存标记的lucene 提交点；事物日志中未提交的操作不会移除id 标记；现实中，在任何时间，应该把索引操作视触发的移除标记为刷新一样；

note：有索引操作时，请求同步刷新是无害的；当索引空闲时同步刷新成功，否则失败；任何请求成功的同步刷新在数据恢复时更加迅速；

curl -X POST "localhost:9200/twitter/_flush/synced"返回值将显示有多少分片同步刷新成功，及失败原因；

1）、以下是同步刷新成功的情况；


{
    "_shards": {
    "total": 4,
    "successful": 4,
    "failed": 0
    },
    "twitter": {
    "total": 4,
    "successful": 4,
    "failed": 0
    }
}

2)、以下是失败情况及其原因；

{

   "_shards": {
      "total": 4,
      "successful": 2,
      "failed": 2
   },
   "twitter": {
      "total": 4,
      "successful": 2,
      "failed": 2,
      "failures": [
         {
            "shard": 1,
            "reason": "[2] ongoing operations on primary"
         }
      ]
   }
}

note:当并行索引操作时，以上错误会出现，且http 状态码将是 409 ;

3)、有时候分片copy的时候错误很特殊；copy失败对于数据的快速恢复没有用，但是如果成功了依然有用；具体错误如下：

{
   "_shards": {
      "total": 4,
      "successful": 1,
      "failed": 1
   },
   "twitter": {
      "total": 4,
      "successful": 3,
      "failed": 1,
      "failures": [
         {
            "shard": 1,
            "reason": "unexpected error",
            "routing": {
               "state": "STARTED",
               "primary": false,
               "node": "SZNr2J_ORxKTLUCydGX4zA",
               "relocating_node": null,
               "shard": 1,
               "index": "twitter"
            }
         }
      ]
   }
}

同步刷新时copy失败，http 返回码将是409 ;

4)、同步刷新可以再单一索引上用，也可以在全部索引上用；

单一索引：

POST /kimchy,elasticsearch/_flush/synced

全部索引：

POST /_flush/synced

大师兄你家猴跑啦

发布了32 篇原创文章 · 获赞 8 · 访问量 8万+

私信关注

ElasticSearch 升级过程中给涉及到的术语-1、同步刷新synced flush

同步刷新 API

猜你喜欢