1. Preparation
reference document " Introduction to the use of high-performance elasticsearch ORM development library " Import and configure es client bboss
2. Define scroll retrieval dsl
First define a simple scroll dsl retrieval script
<properties> <property name="scrollQuery"> <![CDATA[ { ## Here are all operations with constants, and variables can be parameterized in actual scenarios "size":1000, "query": { "term" : { "gc.jvmGcOldCount" : 3 ##Parameter values can be defined as variables and passed in through parameters } } } ]]> </property> </properties>
3.Scroll retrieval code
@Test public void testScroll(){ ClientInterface clientUtil = ElasticSearchHelper.getConfigRestClientUtil("esmapper/scroll.xml"); //scroll paging retrieval, map the retrieval result to a Map object, or map it to a custom entity object ESDatas<Map> response = clientUtil.searchList("agentstat-*/_search?scroll=1m", "scrollQuery",//For the dsl script name, configure in the esmapper/scroll.xml file Map.class); List<Map> datas = response.getDatas();//The first page data List<String > scrollIds = new ArrayList<>();//It is used to record the scrollid of each scroll, which is convenient for clearing after retrieval long totalSize = response.getTotalSize();//Number of total records String scrollId = response.getScrollId(); if(scrollId != null) scrollIds.add(scrollId); System.out.println("totalSize:"+totalSize); System.out.println("scrollId:"+scrollId); if(datas != null && datas.size() > 0) {//1000 records per page, traverse scroll paging results by iterating scrollid do { response = clientUtil.searchScroll("1m",scrollId,Map.class); scrollId = response.getScrollId();//scrollid of each page if(scrollId != null) scrollIds.add(scrollId); datas = response.getDatas();//Number of records per page if(datas == null || datas.size() == 0){ break; } } while (true); } //Query and print the scroll context information that exists on the es server String scrolls = clientUtil.executeHttp("_nodes/stats/indices/search", ClientUtil.HTTP_GET); System.out.println(scrolls); //Clear the scroll context information, although the scrollid will automatically expire after more than 1 minute, it is a good habit to manually delete the unused scrollid and release es resources if(scrollIds.size() > 0) { scrolls = clientUtil.deleteScrolls(scrollIds); System.out.println(scrolls); } //View the scroll context information after cleaning scrolls = clientUtil.executeHttp("_nodes/stats/indices/search", ClientUtil.HTTP_GET); System.out.println(scrolls); }
4. Scroll case project address and code file
project address:
https://gitee.com/bboss/elasticsearchdemo/
scroll to retrieve the corresponding code and script files:
https://gitee.com/bboss/elasticsearchdemo/blob/master/src /test/resources/esmapper/scroll.xml
https://gitee.com/bboss/elasticsearchdemo/blob/master/src/test/java/org/frameworkset/elasticsearch/TestScrollQuery.java