elk 入门

安装

---

准备

• 3台机器

172.16.240.60  db01  elasticsearch 
172.16.240.70  db02  kibana
172.16.240.80  db03  filebeat nginx
172.16.240.90  db04  logstash

• 安装阿里yum和java

curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo
yum install java-1.8.0-openjdk.x86_64 -y

• 更新时间

yum install ntpdate -y
ntpdate time1.aliyun.com

elasticsearch

172.16.240.60

rpm下载

mkdir -p /data/soft
cd /data/soft
rpm -ivh elasticsearch-6.6.0.rpm 
vim  /etc/elasticsearch/elasticsearch.yml 
  node.name: node-1
  path.data: /var/lib/elasticsearch
  path.logs: /var/log/elasticsearch
  network.host: 0.0.0.0 
  http.port: 9200
  
vim /usr/lib/systemd/system/elasticsearch.service
  [Service]
  LimitMEMLOCK=infinity
  
systemctl daemon-reload
systemctl enable elasticsearch.service
systemctl start elasticsearch.service

• 验证

  curl 172.16.240.60:9200

Kibana

172.16.240.70

安装kibana

mkdir -p /data/soft
cd /data/soft
rpm -ivh kibana-6.6.0-x86_64.rpm 
rpm -qc kibana
    /etc/kibana/kibana.yml

vim /etc/kibana/kibana.yml
  server.port: 5601
  server.host: "172.16.240.70"
  server.name: "db02"
  elasticsearch.hosts: ["http://172.16.240.60:9200/"]
  kibana.index: ".kibana"
  
systemctl start kibana
systemctl enable kibana

• 验证

  访问 http://172.16.240.70:5601/

Filebeat + nginx

172.16.240.80

安装nginx

• 配置nginx的yum源

  文档

yum install yum-utils -y
vim /etc/yum.repos.d/nginx.repo
  [nginx-stable]
  name=nginx stable repo
  baseurl=http://nginx.org/packages/centos/$releasever/$basearch/
  gpgcheck=1
  enabled=1
  gpgkey=https://nginx.org/keys/nginx_signing.key
  module_hotfixes=true

  [nginx-mainline]
  name=nginx mainline repo
  baseurl=http://nginx.org/packages/mainline/centos/$releasever/$basearch/
  gpgcheck=1
  enabled=1
  gpgkey=https://nginx.org/keys/nginx_signing.key
  module_hotfixes=true
  
yum-config-manager --enable nginx-mainline
yum -y install httpd-tools nginx
systemctl start nginx
systemctl enable nginx

安装filebeat

mkdir -p /data/soft
cd /data/soft/
rpm -ivh filebeat-6.6.0-x86_64.rpm

• 查看filebeat的配置文件

rpm -qc filebeat

配置filebeat

vim /etc/filebeat/filebeat.yml 
  filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /var/log/nginx/access.log
  filebeat.config.modules:
    path: ${path.config}/modules.d/*.yml
    reload.enabled: false
  setup.template.settings:
    index.number_of_shards: 3
  setup.kibana:
  output.elasticsearch:
    hosts: ["172.16.240.60:9200"]
  processors:
    - add_host_metadata: ~
    - add_cloud_metadata: ~
    
systemctl start filebeat
systemctl enable filebeat

验证日志

• 通过chrom插件 elasticsearch head 查看

• 通过kibana 查看

  输入 http://172.16.240.70:5601/

收集JSON日志

---