ELK - Logstash - 过滤日志输出

通过Filebeat + ELK 可以部署一个日志过滤的程序。

可以将过滤出来的日志记录保存到任何地方，做保存、计算等用途。

实验开始了！我们过滤出错误日志，再输出

==============================================================================

日志格式：

2018-12-27 21:48:50,512 [Test-ELK] ERROR id=42, myname=dbwtest03bc.daodao.com, myaddr=192.168.4.17, c1=84, c2=d, c3=f, c4=aHikMq

logstash命令配置：

./logstash -e 'input { beats { port => 5044 } } filter { grok { match => { "message" => "%{DATESTAMP:time}%{SPACE}\[%{DATA:title}\]%{SPACE}%{LOGLEVEL:level}%{SPACE}%{GREEDYDATA:message}" } } } output { if [level]== "ERROR" { stdout { } } }'

输出结果：

{
           "host" => {
        "containerized" => true,
                   "os" => {
            "platform" => "centos",
             "version" => "7 (Core)",
              "family" => "redhat",
            "codename" => "Core"
        },
                 "name" => "dbwtest03bc.daodao.com",
         "architecture" => "x86_64",
                   "id" => "6787d9310dd84654ab8871f64df6f6d7"
    },
     "@timestamp" => 2018-12-28T02:48:53.335Z,
           "time" => "18-12-27 21:48:50,512",
         "offset" => 390397,
         "source" => "/root/test_elk/test_elk.log",
     "prospector" => {
        "type" => "log"
    },
        "message" => "2018-12-27 21:48:50,512 [Test-ELK] ERROR id=42, myname=dbwtest03bc.daodao.com, myaddr=192.168.4.17, c1=84, c2=d, c3=f, c4=aHikMq",
          "level" => "ERROR",
          "input" => {
        "type" => "log"
    },
           "beat" => {
         "version" => "6.5.4",
            "name" => "dbwtest03bc.daodao.com",
        "hostname" => "dbwtest03bc.daodao.com"
    },
          "title" => "Test-ELK",
    "log_message" => "id=42, myname=dbwtest03bc.daodao.com, myaddr=192.168.4.17, c1=84, c2=d, c3=f, c4=aHikMq",
       "@version" => "1",
           "tags" => [
        [0] "beats_input_codec_plain_applied"
    ]
}