Today, I will continue to introduce the relevant knowledge of Linux operation and maintenance. The main content of this article is the basic principle of ELK.
1. Introduction to ELK
ELK is the collective name of three software, namely the abbreviation of Elasticsearch, Logstash and Kibana three open source software. These three software are all open source software, usually used together, and they are all under the name of Elastic.co, so they are referred to as the ELK protocol stack. ELK is mainly used to deploy in enterprise architecture, collect log information of multiple services on multiple devices, and provide it to users after unified integration. The official website of ELK is as follows: https://www.elastic.co/cn/ , and its page is as follows:
2. ELK architecture
In the ELK architecture, the functions of Elasticsearch, Logstash and Kibana are as follows:
1. Elasticsearch
Elasticsearch is a highly scalable full-text search and analysis engine based on Apache Lucence (in fact, Lucence is also the search engine used by Baidu), Ability to store, search, and analyze large volumes of data in near real-time.
2. Logstash
Logstash is a data collection engine that can dynamically collect data from various data sources, filter, analyze and unify the data, and store the output results in a specified location. Logstash supports common log files and log parsing in custom Json format.
3. Kibana
Kibana is a data analysis and visualization platform, usually used in conjunction with Elasticsearch, to search, analyze, and display the data in it in the form of statistical icons.
The architecture of ELK is as follows:
As shown in the figure above, Logstash is installed on each device to collect log information. The collected log information is aggregated to Elasticsearch, and then Kibana is responsible for the web-side display. Among them, if there are too many terminal devices, Elasticsearch will be overloaded. At this time, we can use a Redis device as a message queue to temporarily cache data and avoid sudden pressure on Elasticsearch.
3. Advantages of ELK
The advantages of the ELK architecture are as follows:
1. Flexible processing methods. Elasticsearch is a full-text index with powerful search capabilities.
2, the configuration is relatively simple. The configuration of Kibana is very simple, while Elasticsearch all uses the Json interface, and the configuration is not complicated. The configuration of Logstash uses the module method, and the configuration is relatively simple.
3. High retrieval performance. The ELK architecture can usually achieve second-level responses to queries of tens of billions of data.
4. Cluster linear expansion. Elasticsearch itself does not have the concept of a single point, the automatic default cluster mode, Elasticsearch and Logstash can be flexibly expanded.
5, the page is beautiful. The front end of Kibana is beautifully designed and easy to operate.
Originality is not easy, please indicate the source for reprinting: https://blog.csdn.net/weixin_40228200