elasticsearch 批处理

介绍：

        最近有个需求：1：索引A 转移到索引B,进行copy操作      2：索引A复制到索引A,进行增量添加

                                3：优化查询，多个查询使用一个API进行批量取    4：调用API，执行批量索引插入(基于restful 接口)

实现：

    1： 索引复制 A -> B     _reindex

POST _reindex
{
"source": {
"index": "twitter"
},
"dest": {
"index": "new_twitter"
}

}

----------使用远程接口

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://oldhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "source",
    "query": {
         "match_all": {}
    }
  },
  "dest": {
    "index": "dest"
  }
}

--注意： 需要本地的elasticsearch.yml里添加远程 ip:port , 如下

reindex.remote.whitelist: oldhost:9200

    2 ： 索引 A -> A*   _reindex

实现思路: 使用 _reindex接口  把 A 索引复制 到 A-1索引 , 再用A* 进行检索

-----------复制索引

POST _reindex

{
"source": {
"index": "test"
},
"dest": {
"index": "test-1"
}

}

-----------查询索引

GET test*/_search

     3：批量查询索引,优化查询,提高查询效率   _msearch

语法：
POST {_index}/{_type}/_msearch
header\n
body\n
header\n

body\n

-----示例
GET /_msearch
{"index" : "my-test"}
{"query" : {"match_all" : {}}, "from" : 0, "size" : 3}
{"index" : "my-test"}
{"query" : {"match_all" : {}}}
{"index" : "my-test1"}
{"query" : {"match_all" : {}}}

--返回多个查询结果,可以根据不同的业务进行处理

     4： 执行批量索引插入 _bulk

 语法规则：

        POST   {ndex}/{type}/_bulk

        option

        fileddata

        option

        fielddata

----------使用_bulk接口实现批量插入

POST  /_bulk

{ "create" : { "_index" : "my-test", "_type" : "default" , "_id" : 1} }
{ "name": "gbasp", "age": 15  }
{ "create" : { "_index" : "my-test", "_type" : "default" ,"_id" :  2 } }

{ "name": "gbasp", "age": 15  }

----------自增序列插入

POST _bulk
{ "index" : { "_index" : "my-test", "_type" : "default" } }
{ "name": "gbaspa", "age": 15  }
{ "index" : { "_index" : "my-test", "_type" : "default" } }
{ "name": "gbaspa", "age": 15  }


POST /my-test/default/_bulk
{ "index" : {  } }
{ "name": "gbaspa", "age": 15  }
{ "index" : {  } }
{ "name": "gbaspa", "age": 15  }

----------官网案例

POST _bulk
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "_doc", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "_doc", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "test"} }

{ "doc" : {"field2" : "value2"} }

    5：批量获取索引  _mget
--------获取多个id

GET my-test/default/_mget
{
  "ids" : [1 , 3 , 4]

}

--------方法同上

GET my-test/default/_mget
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "3"

        },

        {

            "_id" : "4"

         }

    ]
}

-------指定返回字段

GET /_mget
{
    "docs" : [
        {
            "_id" : "1",
            "_source" : false
        },
        {
            "_id" : "2",
            "_source" : ["name"]
        },
        {
            "_id" : "3",
            "_source" : ["age"]
        }
    ]

}

--------------------------------测试数据

POST my-test/default/_bulk
{"index":{}}
{"name":"search1","age":21}
{"index":{}}
{"name":"search2","age":22}
{"index":{}}
{"name":"search4","age":24}
{"index":{}}
{"name":"search5","age":25}
{"index":{}}
{"name":"search6","age":26}
{"index":{}}
{"name":"search8","age":28}
{"index":{}}
{"name":"search0","age":20}


PUT my-test/default/1
{"name":"xiaomi","age":10}


PUT my-test/default/2
{"name":"bob","age":16}


PUT my-test/default/3
{"name":"alice","age":14}


PUT my-test/default/4
{"name":"yuhen","age":22}


PUT my-test1/default/1
{"name":"xiaomi","age":10}


PUT my-test1/default/2
{"name":"bob","age":16}


PUT my-test1/default/3
{"name":"alice","age":14}


PUT my-test1/default/4
{"name":"yuhen","age":22}