Elastic Stack简称ELK，用于数据分析和收集，它是由多个开源软件组成。包括：Elasticsearch、Logstash、Kibana、Beats。
Elasticsearch：核心中的核心组件，基于著名的全文检索引擎lucence的一个分布式版本。由于扩展成分布式，容量和性能得到极大的提升，使得Elasticsearch得以成为目前许多大数据产品和大数据架构的核心组件。
Kibana：展示组件，基于angularjs。从Elasticsearch中读取数据并展示。具有强大而且灵活的界面配置。
Logstash： 一个灵活的数据传输和处理系统，在beats出来之前，还负责进行数据收集。Logstash的任务，就是将各种各样的数据，经过配置转化规则，统一化存入Elasticsearch。使用Ruby开发的Logstash在灵活性上，确实非常出色。不过性能一直是被诟病的问题。
Beats：beats是一组轻量级采集程序的统称，这些采集程序包括并不限于：
- filebeat: 进行文件和目录采集，主要用于收集日志数据。
- metricbeat: 进行指标采集，指标可以是系统的，也可以是众多中间件产品的，主要用于监控系统和软件的性能。
- packetbeat: 通过网络抓包、协议分析，对一些请求响应式的系统通信进行监控和数据收集，可以收集到很多常规方式无法收集到的信息。
- Winlogbeat: 专门针对windows的event log进行的数据采集。
- Heartbeat: 系统间连通性检测，比如icmp, tcp, http等系统的连通性监控。
安装ES
安装步骤
- 官网下载安装包。
- 进行解压 tar -vxf elasticsearch-7.2.0-linux-x86_64.tar.gz
- 在Elasticsearch/config/elasticsearch.yml加入以下参数:
action.auto_create_index: .security,.monitoring*,.watches,.triggered_watches,.watcher-history*
xpack.security.enabled: false
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.ml.enabled: false - nohup sh ./bin/elasticsearch &
配置文件
配置文件在config目录中：
- elasticsearch.yml es的配置文件。
- jvm.options jvm相关配置。
- log4j2.properties 日志相关配置。
ES安装-分布式安装
安装说明，安装三个节点，一个master ，两个slave。
集群名称 ip-端口
myEsCluster （master） 127.0.0.1:9500
myEsCluster（slave） 127.0.0.1:9600
myEsCluster（slave） 127.0.0.1:9700
| 集群名称 | ip-端口 |
|---|---|
| myEsCluster （master） | 127.0.0.1:9500 |
| myEsCluster（slave） | 127.0.0.1:9600 |
| myEsCluster（slave） | 127.0.0.1:9600 |
Es安装包解压出三份ES，修改每个 elasticsearch安装目录 /config/elasticsearch.yml 这个配置文件。
master配置说明:
# 设置支持elasticsearch-head
http.cors.enabled: true
http.cors.allow-origin: "*"
# 设置集群master配置信息
# 首先要指定 集群的名字 ,名字随便起，符合常规命名规则
cluster.name: myEsCluster
# 节点的名字，一般为master 或者 slave
node.name: master
# 节点是否为master，设置为true的话，说明此节点为master节点
node.master: true
# 设置网络,如果是本机的话就是127.0.0.1，其他服务器配置对应的IP地址即可
network.host: 127.0.0.1
# 默认端口为 9200,可以修改默认设置
http.port: 9500
slave配置说明:
# 设置集群slave配置信息
# 首先要指定 集群的名字 ,名字随便起，符合常规命名规则
cluster.name: myEsCluster
# 节点的名字，一般为master 或者 slave
node.name: slave1
# 节点是否为master，设置为true的话，说明此节点为master节点
node.master: false
# 默认端口为 9200,可以修改默认设置
http.port: 9600
# 设置网络,如果是本机的话就是127.0.0.1，其他服务器配置对应的IP地址即可
network.host: 127.0.0.1
# 设置集群中master节点的初始列表，可以通过这些节点来自动发现新加入集群的节点。可以是一个数组。
# #discovery.zen.ping.unicast.hosts: ["host1", "host2"]
discovery.zen.ping.unicast.hosts: ["127.0.0.1"]
两个slave配置只需要改相应的端口号即可！一个slave1：9600 ，一个slave2：9700。
从上面的安装来看，Es的安装相对还是比较简单，并且在安装分布式的时候，通过简单的配置就可以快速扩展多个ES节点，能够快速的进行扩容，非常灵活，方便！
ElasticSearch 基础概念
基本概念
集群和节点
集群：是由一个或者多个ES节点组成的集合。 每一个集群都有一个唯一的名字，默认是ElasticSearch。每个节点也都有唯一名字。
索引
索引： 含有相同属性的文档集合，比如产品的索引，用户的索引等。
类型：索引可以定义一个或多个类型，文档必须属于一个类型（一般会定义有相同字段的文档为一个类型）。
文档：文档是可以被所有的基本数据单位，是整个ES中最小的存储单位。
索引在Es中是通过一个名字来识别的，而且它的名字必须是 英文字母小写，且不能有中划线。
分片和备份
分片： 每个索引都有多个分片，每个分片是一个Lucene索引
分片的好处：假如数据量很大的话，就会造成硬盘的压力很大，同时搜索速度也会出现瓶颈，将索引分成多个分片，从而分摊压力提供效率。
备份：拷贝一份分片就完成了分片的备份，提供可用性。
ES默认在创建索引的时候，会创建5个分片，1个备份。这个默认的数量是可以修改的。
number_of_shards： 每个索引的主分片数，默认值是 5 。这个配置在索引创建后不能修改。
number_of_replicas：每个主分片的副本数，默认值是 1 。对于活动的索引库，这个配置可以随时修改。
注意：索引的分片只能在创建索引的时候指定，而不能再后期进行修改。备份是可以动态修改的。
ElasticSearch 基本用法
ES是以Restful API的风格来命名的。
-
API 的基本格式
http://:/<索引>/<类型>/<文档id>
-
常用HTTP动词
GET/PUT/POST/DELETE
索引的创建
在Elasticsearch 6.0.0或更高版本中创建的索引只包含一个mapping type。 在5.x中使用multiple mapping types创建的索引将继续像以前一样在Elasticsearch 6.x中运行。 Mapping types将在Elasticsearch 7.0.0中完全删除。
PUT http://127.0.0.1:9200/people
7.X版本
{
"settings":{
"number_of_shards":1,
"number_of_replicas":1
},
"mappings":{
"properties":{
"name":{
"type":"text"
},
"age":{
"type":"integer"
},
"birth":{
"type":"date",
"format":"yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
"country":{
"type":"keyword"
}
}
}
}
6.X版本
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
},
"mappings": {
"book": {
"properties": {
"name": {
"type": "text"
},
"age": {
"type": "integer"
},
"birth": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
"country": {
"type": "keyword"
}
}
}
}
}
返回结果：
{
"acknowledged": true,
"shards_acknowledged": true,
"index": "people"
}
插入
文档ID是一个唯一索引值，指向我们的文档数据。
指定文档ID插入
PUT localhost:9200/people/_doc/1
{
"name":"dufy",
"age":27,
"birth":"1992-09-28",
"country":"China"
}
返回结果
{
"_index": "people",
"_type": "_doc",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1
}
自动产生文档ID插入
自动产生ID和上面指定ID插入不同的两个地方是:
（1）、在请求地址上不需要指定 ID，让ES自动帮我们创建ID!
（2）、请求方式为POST方式
POST localhost:9200/people/_doc
{
"name":"dufy_auto_id",
"age":37,
"birth":"1982-09-28",
"country":"China"
}
返回结果
{
"_index": "people",
"_type": "_doc",
"_id": "C70VMmwBZVZWY6HUsxOQ",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 1,
"_primary_term": 1
}
修改
直接修改文档
POST localhost:9200/people/_doc/1/_update
{
"doc":{
"name":"dufy_update"
}
}
返回结果：
{
"_index": "people",
"_type": "_doc",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 2,
"_primary_term": 1
}
删除
删除文档
DELETE localhost:9200/people/_doc/1
返回结果
{
"_index": "people",
"_type": "_doc",
"_id": "1",
"_version": 3,
"result": "deleted",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 3,
"_primary_term": 1
}
删除索引
删除操作是一个比较危险的操作，因为索引一旦删除，对应的所有数据全部被删除了。
DETELE localhost:9200/people
返回结果
{
"acknowledged": true
}
查询
查询功能是使用过操作比较频繁的。
首先进行数据初始化，创建一个book 的索引，然后插入一些数据。具体的创建索引和插入数据，查看之前内容。
PUT localhost:9200/book
7.X版本
{
"mappings": {
"properties": {
"author": {
"type": "keyword"
},
"word_count": {
"type": "integer"
},
"publish_date": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
"title": {
"type": "text"
}
}
}
}
6.X版本
{
"settings": {
"number_of_shards": 1,
"number_of_replicas": 1
},
"mappings": {
"book": {
"properties": {
"author": {
"type": "keyword"
},
"word_count": {
"type": "integer"
},
"publish_date": {
"type": "date",
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
},
"title": {
"type": "text"
}
}
}
}
}
简单查询
根据id查询
GET localhost:9200/book/_doc/1
返回结果
{
"_index": "book",
"_type": "_doc",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"found": true,
"_source": {
"title": "移魂大法",
"word_count": 1000,
"publish_date": "2000-10-01",
"author": "张三"
}
}
条件查询
POST localhost:9200/book/_search
查询所有数据
{
"query":{
"match_all":{}
}
}
返回结果
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 11,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "book",
"_type": "_doc",
"_id": "1",
"_score": 1,
"_source": {
"title": "移魂大法",
"word_count": 1000,
"publish_date": "2000-10-01",
"author": "张三"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "2",
"_score": 1,
"_source": {
"title": "Java入门",
"word_count": 2000,
"publish_date": "2000-10-01",
"author": "李三"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "3",
"_score": 1,
"_source": {
"title": "Python入门",
"word_count": 2000,
"publish_date": "2005-10-01",
"author": "张四"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "4",
"_score": 1,
"_source": {
"title": "ElasticSearch大法好",
"word_count": 1000,
"publish_date": "2017-08-01",
"author": "李四"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "5",
"_score": 1,
"_source": {
"title": "菜谱",
"word_count": 5000,
"publish_date": "2002-10-01",
"author": "王五"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "6",
"_score": 1,
"_source": {
"title": "剑谱",
"word_count": 10000,
"publish_date": "1997-01-01",
"author": "赵六"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "7",
"_score": 1,
"_source": {
"title": "太极拳",
"word_count": 1000,
"publish_date": "1997-01-01",
"author": "张三丰"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "8",
"_score": 1,
"_source": {
"title": "太极拳",
"word_count": 1000,
"publish_date": "1997-01-01",
"author": "张三丰"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "9",
"_score": 1,
"_source": {
"title": "ElasticSearch精通",
"word_count": 3000,
"publish_date": "2017-08-15",
"author": "很胖的瓦力"
}
},
{
"_index": "book",
"_type": "_doc",
"_id": "10",
"_score": 1,
"_source": {
"title": "芭蕉扇",
"word_count": 1000,
"publish_date": "2000-10-01",
"author": "牛魔王"
}
}
]
}
}
修改 hits默认返回数据的总数。
{
"query":{
"match_all":{}
},
"from": 0,
"size": 1
}
关键词查询
POST localhost:9200/book/_search
根据text格式的字段进行搜索，会进行分词处理。
{
"query": {
"match": {
"title": "ElasticSearch"
}
},
# 默认 排序 使用 _score
"sort": [
{
# 如何查询数据较多，可以进行排序操作
"publish_date": {
"order": "desc"
}
}
]
}
返回结果
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": "book",
"_type": "_doc",
"_id": "9",
"_score": null,
"_source": {
"title": "ElasticSearch精通",
"word_count": 3000,
"publish_date": "2017-08-15",
"author": "很胖的瓦力"
},
"sort": [
1502755200000
]
},
{
"_index": "book",
"_type": "_doc",
"_id": "4",
"_score": null,
"_source": {
"title": "ElasticSearch大法好",
"word_count": 1000,
"publish_date": "2017-08-01",
"author": "李四"
},
"sort": [
1501545600000
]
}
]
}
}
聚合查询
POST localhost:9200/book/_search
对书籍的字数进行聚合
{
"aggs":{
"groud_word_count":{ # 自定义聚合名称
"terms":{
"field":"word_count"
}
}
}
}
返回结果
# 其他数据省略
"aggregations": {
"groud_word_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 1000,
"doc_count": 6
},
{
"key": 2000,
"doc_count": 2
},
{
"key": 3000,
"doc_count": 1
},
{
"key": 5000,
"doc_count": 1
},
{
"key": 10000,
"doc_count": 1
}
]
}
}
上面这个是单个聚合，多个聚合同理
{
"aggs":{
"groud_word_count":{
"terms":{
"field":"word_count"
}
},
"groud_publish_date":{
"terms":{
"field":"publish_date"
}
}
}
}
返回结果
# 其他数据省略
"aggregations": {
"groud_publish_date": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 970358400000,
"key_as_string": "2000-10-01 00:00:00",
"doc_count": 4
},
{
"key": 852076800000,
"key_as_string": "1997-01-01 00:00:00",
"doc_count": 3
},
{
"key": 1033430400000,
"key_as_string": "2002-10-01 00:00:00",
"doc_count": 1
},
{
"key": 1128124800000,
"key_as_string": "2005-10-01 00:00:00",
"doc_count": 1
},
{
"key": 1501545600000,
"key_as_string": "2017-08-01 00:00:00",
"doc_count": 1
},
{
"key": 1502755200000,
"key_as_string": "2017-08-15 00:00:00",
"doc_count": 1
}
]
},
"groud_word_count": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": 1000,
"doc_count": 6
},
{
"key": 2000,
"doc_count": 2
},
{
"key": 3000,
"doc_count": 1
},
{
"key": 5000,
"doc_count": 1
},
{
"key": 10000,
"doc_count": 1
}
]
}
进行函数计算
status 会调用所有函数，也可以指定具体函数：min 、max、avg、sum
{
"aggs":{
"grades_word_count":{
"stats":{ #统计数据 min 、max、avg、sum
"field":"word_count"
}
}
}
}
返回结果
"aggregations": {
"grades_word_count": {
"count": 11,
"min": 1000,
"max": 10000,
"avg": 2545.4545454545455,
"sum": 28000
}
}
ElasticSearch 高级查询
子条件查询
特定字段查询 所指特定值
Query Context
在查询过程中，除了判断文档是否满足查询条件外，ES还会计算一个**_score来标识匹配的程度，旨在判断目标文档和查询条件匹配的有多好**（匹配度）！
全文本查询 —— 针对文本类型数据
-
模糊匹配查询
查询书名是ElasticSearch入门。由于title是text对类型，es会进行分词处理，包含ElasticSearch和入门的数据都会被查出来。
POST localhost:9200/book/_search
{ "query":{ "match":{ "title": "ElasticSearch入门" } } }返回结果
{ "took": 432, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 4, "relation": "eq" }, "max_score": 3.175439, "hits": [ { "_index": "book", "_type": "_doc", "_id": "2", "_score": 3.175439, "_source": { "title": "Java入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "李三" } }, { "_index": "book", "_type": "_doc", "_id": "3", "_score": 3.175439, "_source": { "title": "Python入门", "word_count": 2000, "publish_date": "2005-10-01", "author": "张四" } }, { "_index": "book", "_type": "_doc", "_id": "9", "_score": 1.5877194, "_source": { "title": "ElasticSearch精通", "word_count": 3000, "publish_date": "2017-08-15", "author": "很胖的瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "4", "_score": 1.4001489, "_source": { "title": "ElasticSearch大法好", "word_count": 1000, "publish_date": "2017-08-01", "author": "李四" } } ] } } -
精准匹配
将 ElasticSearch入门 看作是一个整体。
POST localhost:9200/book/_search
{ "query":{ "match_phrase":{ "title": "ElasticSearch入门" } } }返回结果
{ "took": 1219, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 1, "relation": "eq" }, "max_score": 3.98057, "hits": [ { "_index": "book", "_type": "_doc", "_id": "12", "_score": 3.98057, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } } ] } } -
匹配多个字段
fields 中只要有一个包含要查询的内容就会被查出来。
POST localhost:9200/book/_search
{ "query":{ "multi_match":{ "query": "瓦力", "fields": ["title", "author"] } } }返回结果
{ "took": 68, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 3, "relation": "eq" }, "max_score": 3.3231916, "hits": [ { "_index": "book", "_type": "_doc", "_id": "14", "_score": 3.3231916, "_source": { "title": "瓦力跳舞", "word_count": 2000, "publish_date": "2000-10-01", "author": "海波" } }, { "_index": "book", "_type": "_doc", "_id": "13", "_score": 2.7105768, "_source": { "title": "瓦力叫我们ElasticSearch", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "12", "_score": 1.7917595, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } } ] } }
字段级别查询——针对结构化数据，如数字、日期等
-
字段查询
POST localhost:9200/book/_search
作者是瓦力的。
{ "query":{ "term":{ "author": "瓦力" } } }返回结果
{ "took": 9, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 2, "relation": "eq" }, "max_score": 1.7917595, "hits": [ { "_index": "book", "_type": "_doc", "_id": "12", "_score": 1.7917595, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "13", "_score": 1.7917595, "_source": { "title": "瓦力叫我们ElasticSearch", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } } ] } } -
范围查询
POST localhost:9200/book/_search
按时间范围查询
{ "query": { "range": { "publish_date": { "gt": "2017-01-01 10:00:00", "lte": "2019-07-22 11:00:11" } } } }返回结果
{ "took": 22, "timed_out": false, "_shards": { "total": 1, "successful": 1, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 2, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "book", "_type": "_doc", "_id": "4", "_score": 1, "_source": { "title": "ElasticSearch大法好", "word_count": 1000, "publish_date": "2017-08-01", "author": "李四" } }, { "_index": "book", "_type": "_doc", "_id": "9", "_score": 1, "_source": { "title": "ElasticSearch精通", "word_count": 3000, "publish_date": "2017-08-15", "author": "很胖的瓦力" } } ] } }
Filter Context
Filter 只关注是否匹配 不关注匹配程度 不会算分。
POST localhost:9200/book/_search
查询字数等于5000
{
"query": {
"bool": {
"filter": {
"term": {
"word_count": 5000
}
}
}
}
}
返回结果
{
"took": 137,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0,
"hits": [
{
"_index": "book",
"_type": "_doc",
"_id": "5",
"_score": 0,
"_source": {
"title": "菜谱",
"word_count": 5000,
"publish_date": "2002-10-01",
"author": "王五"
}
}
]
}
}
复合条件查询
-
固定分数查询
text文本格式，返回固定的分数。boost 设置固定分数。
POST localhost:9200/book/_search
{ "query": { "constant_score": { "filter": { "match": { "title": "ElasticSearch" } }, "boost": 2 } } }返回结果
{ "took": 20, "timed_out": false, "_shards": { "total": 7, "successful": 7, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 5, "relation": "eq" }, "max_score": 2, "hits": [ { "_index": "book", "_type": "_doc", "_id": "4", "_score": 2, "_source": { "title": "ElasticSearch大法好", "word_count": 1000, "publish_date": "2017-08-01", "author": "李四" } }, { "_index": "book", "_type": "_doc", "_id": "9", "_score": 2, "_source": { "title": "ElasticSearch精通", "word_count": 3000, "publish_date": "2017-08-15", "author": "很胖的瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "12", "_score": 2, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "13", "_score": 2, "_source": { "title": "瓦力叫我们ElasticSearch", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "accounts", "_type": "_doc", "_id": "0OslAGwB8tcczbmqUeti", "_score": 2, "_source": { "title": "ElasticSearch入门", "word_count": 3000, "publish_date": "2017-08-20", "author": "瓦力" } } ] } } -
布尔查询
POST localhost:9200/book/_search
should 或的关系。
{ "query": { "bool": { "should": [ { "term": { "author": "瓦力" } }, { "match": { "title": "ElasticSearch" } } ] } } }返回结果
{ "took": 26, "timed_out": false, "_shards": { "total": 7, "successful": 7, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 5, "relation": "eq" }, "max_score": 3.0505137, "hits": [ { "_index": "book", "_type": "_doc", "_id": "12", "_score": 3.0505137, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "13", "_score": 2.7024455, "_source": { "title": "瓦力叫我们ElasticSearch", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "9", "_score": 1.2587543, "_source": { "title": "ElasticSearch精通", "word_count": 3000, "publish_date": "2017-08-15", "author": "很胖的瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "4", "_score": 1.1165094, "_source": { "title": "ElasticSearch大法好", "word_count": 1000, "publish_date": "2017-08-01", "author": "李四" } }, { "_index": "accounts", "_type": "_doc", "_id": "0OslAGwB8tcczbmqUeti", "_score": 0.9331132, "_source": { "title": "ElasticSearch入门", "word_count": 3000, "publish_date": "2017-08-20", "author": "瓦力" } } ] } }POST localhost:9200/book/_search
must 与的关系。 还有must_not 代表必须不满足，不做介绍了。
{ "query": { "bool": { "must": [ { "term": { "author": "瓦力" } }, { "match": { "title": "ElasticSearch" } } ], "filter": { "term": { "word_count": 2000 } } } } }返回结果
{ "took": 92, "timed_out": false, "_shards": { "total": 7, "successful": 7, "skipped": 0, "failed": 0 }, "hits": { "total": { "value": 2, "relation": "eq" }, "max_score": 3.0505137, "hits": [ { "_index": "book", "_type": "_doc", "_id": "12", "_score": 3.0505137, "_source": { "title": "ElasticSearch入门", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } }, { "_index": "book", "_type": "_doc", "_id": "13", "_score": 2.7024455, "_source": { "title": "瓦力叫我们ElasticSearch", "word_count": 2000, "publish_date": "2000-10-01", "author": "瓦力" } } ] } }
es整合spring-boot
使用 spring-boot-starter-data-elasticsearch 实现了对索引的简单的增删查改。
pom文件中引入spring-boot-starter-data-elasticsearch。
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.1.6.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example.es</groupId>
<artifactId>es</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>es</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<version>1.16.10</version>
</dependency>
<!-- 构建Restful API -->
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger2</artifactId>
<version>2.6.0</version>
</dependency>
<dependency>
<groupId>io.springfox</groupId>
<artifactId>springfox-swagger-ui</artifactId>
<version>2.6.0</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application.yml
server:
port: 9999
spring:
data:
elasticsearch:
cluster-nodes: 127.0.0.1:9300 #配置es节点信息，逗号分隔，如果没有指定，则启动ClientNode（9200端口是http查询使用的。9300集群使用。这里使用9300.）
repositories: true
es索引对象。
@Data
@Accessors(chain = true)
@EqualsAndHashCode(callSuper = false)
@Document(indexName = "book", type = "book")
public class EsBook implements Serializable {
@Id
private String id;
@Field(type = FieldType.Text)
private String title;
@Field(type = FieldType.Keyword)
private String author;
@Field(type = FieldType.Integer)
private Integer wordCount;
@JsonFormat(pattern = "yyyy-MM-dd\'T\'HH:mm:ss")
@JsonSerialize(using = LocalDateTimeSerializer.class)
@JsonDeserialize(using = LocalDateTimeDeserializer.class)
@Field(type = FieldType.Date, format = DateFormat.date_hour_minute_second)
private LocalDateTime publishDate;
}
EsBookRepository
public interface EsBookRepository extends ElasticsearchRepository<EsBook, String> {
}
EsBookService
@Component
public class EsBookService {
@Autowired
private EsBookRepository esBookRepository;
public EsBook save(EsBook esBook) {
if(Objects.isNull(esBook)) {
return null;
}
return esBookRepository.index(esBook);
}
public EsBook update(EsBook esBook) {
if(Objects.isNull(esBook) || StringUtils.isEmpty(esBook.getId())) {
return null;
}
return esBookRepository.save(esBook);
}
public void delete(String id) {
if(!StringUtils.isEmpty(id)) {
esBookRepository.deleteById(id);
}
}
public EsBook getById(String id) {
if(StringUtils.isEmpty(id)) {
return null;
}
return esBookRepository.findById(id).get();
}
public Page<EsBook> getByParams(EsBookParams esBookParams) {
BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();
//作者 =
if(!StringUtils.isEmpty(esBookParams.getAuthor())) {
boolQueryBuilder.filter(QueryBuilders.termQuery("author", esBookParams.getAuthor()));
}
//标题like
if(!StringUtils.isEmpty(esBookParams.getTitle())) {
boolQueryBuilder.filter(QueryBuilders.matchPhraseQuery("title", esBookParams.getTitle()));
}
//字数 between
RangeQueryBuilder rangeQuery = QueryBuilders.rangeQuery("wordCount");
Integer wordCountFrom = esBookParams.getWordCountFrom();
rangeQuery.from(Objects.isNull(wordCountFrom)?0:wordCountFrom);
if(Objects.nonNull(esBookParams.getWordCountTo())) {
rangeQuery.to(esBookParams.getWordCountTo());
}
boolQueryBuilder.filter(rangeQuery);
//时间排序
Sort sort = Sort.by(new Order(Sort.Direction.DESC, "publishDate"));
PageRequest pageRequest = PageRequest.of(esBookParams.getPageNum() - 1, esBookParams.getPageSize(), sort);
return esBookRepository.search(boolQueryBuilder ,pageRequest);
}
}
EsBookController
@RequestMapping("/order")
@RestController
public class EsBookController {
@Autowired
private EsBookService esBookService;
@GetMapping("/save")
@ResponseBody
public EsBook save(@RequestParam(required = false) String title,
@RequestParam(required = false) String author,
@RequestParam(required = false) Integer wordCount,
@RequestParam(required = false) String publishDate) {
DateTimeFormatter df = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
EsBook esBook = new EsBook();
esBook.setId(UUID.randomUUID().toString())
.setAuthor(author)
.setTitle(title)
.setWordCount(wordCount)
.setPublishDate(LocalDateTime.parse(publishDate, df));
return esBookService.save(esBook);
}
@PostMapping("/update")
@ResponseBody
public EsBook update(@RequestBody EsBookArgs esBookArgs) {
EsBook esBook = new EsBook();
BeanUtils.copyProperties(esBookArgs, esBook);
DateTimeFormatter df = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
esBook.setPublishDate(LocalDateTime.parse(esBookArgs.getPublishDate(), df));
return esBookService.update(esBook);
}
@PostMapping("/delete")
@ResponseBody
public void delete(@RequestParam String id) {
esBookService.delete(id);
}
@GetMapping("/getById")
@ResponseBody
public EsBook getById(@RequestParam String id) {
return esBookService.getById(id);
}
@GetMapping("/getPage")
@ResponseBody
public Page<EsBook> getPage(@RequestParam(required = false) String title,
@RequestParam(required = false) String author,
@RequestParam(required = false) Integer wordCountFrom,
@RequestParam(required = false) Integer wordCountTo,
@RequestParam(required = false) String publishDate,
@RequestParam Integer pageNum,
@RequestParam Integer pageSize) {
EsBookParams params = new EsBookParams();
params.setTitle(title);
params.setAuthor(author);
params.setWordCountFrom(wordCountFrom);
params.setWordCountTo(wordCountTo);
params.setPublishDate(publishDate);
params.setPageNum(pageNum);
params.setPageSize(pageSize);
return esBookService.getByParams(params);
}
}
安装Kibana
- 官网下载安装包
- 进行解压 tar -vxf kibana-7.2.0.tar.gz
- 在kibana.yml 中增加es配置。elasticsearch.url: “http://localhost:9200”
- nohup sh ./bin/kibana &
