5. ElasticSearch application
1. Two ways of ES Java API
Elasticsearch's API is divided into
two types: REST Client API (http request form) and transportClient API. In comparison, the transportClient API is more efficient. TransportClient
is requested through Elasticsearch's internal RPC. The connection can be a long connection, which is equivalent to treating the client's request as
A node of the Elasticsearch cluster. Of course, the REST Client API also supports
long connections in the form of http keepAlive, but not in the internal RPC form. But transportClient will be removed from Elasticsearch 7
. The main reason is that transportClient is difficult to be backward compatible with versions.
1.1 9300[TCP]
The one that uses port 9300 is spring-data-elasticsearch:transport-api.jar. However, because the corresponding SpringBoot versions are inconsistent with this method, the corresponding transport-api.jar is also different and cannot adapt to the es version, and ElasticSearch7.x It is no longer recommended to use in ElasticSearch 8, and it is even more abandoned after ElasticSearch 8, so we will not introduce it too much.
1.2 9200[HTTP]
There are also many methods based on port 9200
- JsetClient: Unofficial, slow to update
- RestTemplate: simulates sending Http requests. Many ES operations need to be encapsulated by ourselves, which is inefficient.
- HttpClient: Same as above
- ElasticSearch-Rest-Client: The official RestClient encapsulates ES operations. The API is clearly layered and easy to use.
- JavaAPIClient is recommended after version 7.15
2.ElasticSearch-Rest-Client integration
2.1 Create a retrieval service
We create a retrieval SpringBoot service in the mall service
Add corresponding dependencies: Official address: https://www.elastic.co/guide/en/elasticsearch/client/java-rest/current/java-rest-high-getting-started-maven.html#java-rest- high-getting-started-maven-maven
Don’t forget the public dependencies. At the same time, we depend on MyBatisPlus in the public dependencies, so we need to exclude the data source in the search service, otherwise an error will be reported at startup.
Then we need to register this service in the Nacos registration center. This operation has been done many times and will not be repeated.
Add the corresponding ElasticSearch configuration class
/**
* ElasticSearch的配置类
*/
@Configuration
public class MallElasticSearchConfiguration {
@Bean
public RestHighLevelClient restHighLevelClient(){
RestClientBuilder builder = RestClient.builder(new HttpHost("192.168.56.100", 9200, "http"));
RestHighLevelClient client = new RestHighLevelClient(builder);
return client;
}
}
test:
2.2 Test the saved document
SetRequestOptions
We set it in the ElasticSearch configuration file
save data
Then you can combine it with official documents to realize the storage of document data.
package com.msb.mall.mallsearch;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.util.JSONPObject;
import com.msb.mall.mallsearch.config.MallElasticSearchConfiguration;
import lombok.Data;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.json.JSONObject;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
@SpringBootTest
class MallSearchApplicationTests {
@Autowired
private RestHighLevelClient client;
@Test
void contextLoads() {
System.out.println("--->"+client);
}
/**
* 测试保存文档
*/
@Test
void saveIndex() throws Exception {
IndexRequest indexRequest = new IndexRequest("system");
indexRequest.id("1");
// indexRequest.source("name","bobokaoya","age",18,"gender","男");
User user = new User();
user.setName("bobo");
user.setAge(22);
user.setGender("男");
// 用Jackson中的对象转json数据
ObjectMapper objectMapper = new ObjectMapper();
String json = objectMapper.writeValueAsString(user);
indexRequest.source(json, XContentType.JSON);
// 执行操作
IndexResponse index = client.index(indexRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 提取有用的返回信息
System.out.println(index);
}
@Data
class User{
private String name;
private Integer age;
private String gender;
}
}
Success later
2.3 Search operation
Refer to the official documentation to get the API for handling various retrieval situations.
Case 1: Retrieve all documents indexed by all banks
@Test
void searchIndexAll() throws IOException {
// 1.创建一个 SearchRequest 对象
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank"); // 设置我们要检索的数据对应的索引库
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
/*sourceBuilder.query();
sourceBuilder.from();
sourceBuilder.size();
sourceBuilder.aggregation();*/
searchRequest.source(sourceBuilder);
// 2.如何执行检索操作
SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 3.获取检索后的响应对象，我们需要解析出我们关心的数据
System.out.println("ElasticSearch检索的信息："+response);
}
Case 2: Full text search based on address
@Test
void searchIndexByAddress() throws IOException {
// 1.创建一个 SearchRequest 对象
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank"); // 设置我们要检索的数据对应的索引库
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 查询出bank下 address 中包含 mill的记录
sourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
searchRequest.source(sourceBuilder);
// System.out.println(searchRequest);
// 2.如何执行检索操作
SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 3.获取检索后的响应对象，我们需要解析出我们关心的数据
System.out.println("ElasticSearch检索的信息："+response);
}
Case 3: Nested aggregation operation: retrieve the age distribution under the bank and the average salary of each age group
/**
* 聚合：嵌套聚合
* @throws IOException
*/
@Test
void searchIndexAggregation() throws IOException {
// 1.创建一个 SearchRequest 对象
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank"); // 设置我们要检索的数据对应的索引库
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 查询出bank下 所有的文档
sourceBuilder.query(QueryBuilders.matchAllQuery());
// 聚合 aggregation
// 聚合bank下年龄的分布和每个年龄段的平均薪资
AggregationBuilder aggregationBuiler = AggregationBuilders.terms("ageAgg")
.field("age")
.size(10);
// 嵌套聚合
aggregationBuiler.subAggregation(AggregationBuilders.avg("balanceAvg").field("balance"));
sourceBuilder.aggregation(aggregationBuiler);
sourceBuilder.size(0); // 聚合的时候就不用显示满足条件的文档内容了
searchRequest.source(sourceBuilder);
System.out.println(sourceBuilder);
// 2.如何执行检索操作
SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 3.获取检索后的响应对象，我们需要解析出我们关心的数据
System.out.println(response);
}
Case 4: Parallel aggregation operation: Query the distribution of age groups and the total average salary under the bank
/**
* 聚合
* @throws IOException
*/
@Test
void searchIndexAggregation1() throws IOException {
// 1.创建一个 SearchRequest 对象
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank"); // 设置我们要检索的数据对应的索引库
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 查询出bank下 所有的文档
sourceBuilder.query(QueryBuilders.matchAllQuery());
// 聚合 aggregation
// 聚合bank下年龄的分布和平均薪资
AggregationBuilder aggregationBuiler = AggregationBuilders.terms("ageAgg")
.field("age")
.size(10);
sourceBuilder.aggregation(aggregationBuiler);
// 聚合平均年龄
AvgAggregationBuilder balanceAggBuilder = AggregationBuilders.avg("balanceAgg").field("age");
sourceBuilder.aggregation(balanceAggBuilder);
sourceBuilder.size(0); // 聚合的时候就不用显示满足条件的文档内容了
searchRequest.source(sourceBuilder);
System.out.println(sourceBuilder);
// 2.如何执行检索操作
SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 3.获取检索后的响应对象，我们需要解析出我们关心的数据
System.out.println(response);
}
Case 5: Processing results after retrieval
@Test
void searchIndexResponse() throws IOException {
// 1.创建一个 SearchRequest 对象
SearchRequest searchRequest = new SearchRequest();
searchRequest.indices("bank"); // 设置我们要检索的数据对应的索引库
SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
// 查询出bank下 address 中包含 mill的记录
sourceBuilder.query(QueryBuilders.matchQuery("address","mill"));
searchRequest.source(sourceBuilder);
// System.out.println(searchRequest);
// 2.如何执行检索操作
SearchResponse response = client.search(searchRequest, MallElasticSearchConfiguration.COMMON_OPTIONS);
// 3.获取检索后的响应对象，我们需要解析出我们关心的数据
// System.out.println("ElasticSearch检索的信息："+response);
RestStatus status = response.status();
TimeValue took = response.getTook();
SearchHits hits = response.getHits();
TotalHits totalHits = hits.getTotalHits();
TotalHits.Relation relation = totalHits.relation;
long value = totalHits.value;
float maxScore = hits.getMaxScore(); // 相关性的最高分
SearchHit[] hits1 = hits.getHits();
for (SearchHit documentFields : hits1) {
/*"_index" : "bank",
"_type" : "account",
"_id" : "970",
"_score" : 5.4032025*/
//documentFields.getIndex(),documentFields.getType(),documentFields.getId(),documentFields.getScore();
String json = documentFields.getSourceAsString();
//System.out.println(json);
// JSON字符串转换为 Object对象
ObjectMapper mapper = new ObjectMapper();
Account account = mapper.readValue(json, Account.class);
System.out.println("account = " + account);
}
//System.out.println(relation.toString()+"--->" + value + "--->" + status);
}
@ToString
@Data
static class Account {
private int account_number;
private int balance;
private String firstname;
private String lastname;
private int age;
private String gender;
private String address;
private String employer;
private String email;
private String city;
private String state;
}
Data results:
