记录logback + ELK日志

logback + ELK日志

安装es

略

安装logstash

https://www.elastic.co/guide/en/logstash/current/index.html

• 新建一个简单的配置文件，只设置logstash的输入和输出，输入为tcp监听的端口，输出为es

input {
    tcp {
        host => "10.1.1.1"
        port => 4567
        mode => "server"
        codec => json_lines
    }
}

output {
     elasticsearch {
        action => "index"
        hosts => ["10.1.1.1:9200"]
        index => "%{[appName]}"
     }
}

• 使用命令行指定配置文件的方式启动logstash，可以nohup后台启动，我这里示例前台启动，启动参数详细介绍：https://www.elastic.co/guide/en/logstash/current/running-logstash-command-line.html

  ~/su/logstash-7.3.0# ./bin/logstash -f ./config/logstash-es.conf 
  

  看到如下日志打印就是启动成功

  [2020-08-21T14:23:05,840][INFO ][logstash.inputs.tcp      ] Starting tcp input listener {:address=>"10.35.96.110:4567", :ssl_enable=>"false"}
  [2020-08-21T14:23:05,986][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
  [2020-08-21T14:23:06,946][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
  

安装kibana

• kibana和的版本和es的版本必须一样，否则启动报错

• 修改配置文件，打开必要的配置项的注释

  # Kibana is served by a back end server. This setting specifies the port to use.
  server.port: 5601
  
  # Specifies the address to which the Kibana server will bind. IP addresses and host names are both valid values.
  # The default is 'localhost', which usually means remote machines will not be able to connect.
  # To allow connections from remote users, set this parameter to a non-loopback address.
  server.host: "10.1.1.1"
  
  # The URLs of the Elasticsearch instances to use for all your queries.
  elasticsearch.hosts: ["http://10.1.1.1:9200"]
  
  # Kibana uses an index in Elasticsearch to store saved searches, visualizations and
  # dashboards. Kibana creates a new index if the index doesn't already exist.
  kibana.index: ".kibana"
  

• 启动kibana

  ~/su/kibana-6.8.1-linux-x86_64/bin# ./kibana
  

  看到如下日志就是启动成功

    log   [10:55:32.074] [info][migrations] Creating index .kibana_1.
    log   [10:55:32.230] [info][migrations] Pointing alias .kibana to .kibana_1.
    log   [10:55:32.355] [info][migrations] Finished in 281ms.
    log   [10:55:32.359] [info][listening] Server running at http://10.35.96.110:5601
    log   [10:55:32.664] [info][status][plugin:[email protected]] Status changed from yellow to green - Ready
  

  到浏览器访问即可打开kibana的页面http://10.1.1.1:5601。

服务logback配置

<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="false" scan="true" scanPeriod="10 seconds">

	<property name="LogHome" value="logs/" />
	<property name="LogPattern" value="[%d{yyyy-MM-dd HH:mm:ss.SSS}][%p][App:%t][%C{0}:%M:%L]%X{requestId}%X{errorId}, %m%n" />

	<!-- 控制台 -->
	<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
		<encoder>
			<pattern>${LogPattern}</pattern>
			<charset>utf-8</charset>
		</encoder>
	</appender>

	<springProperty scop="context" name="destination" source="log.destination" defaultValue="127.0.0.1:4567"/>
	<springProperty scop="context" name="appName" source="spring.application.name" defaultValue="appLog"/>

	<appender name="logstash" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
		<param name="Encoding" value="UTF-8"/>
		<destination>${destination}</destination>
		<!-- <filter class="com.program.interceptor.ELKFilter"/>-->
		<!-- encoder is required -->
		<encoder charset="UTF-8" class="net.logstash.logback.encoder.LogstashEncoder" >
			<customFields>{"appName":"${appName}"}</customFields>
		</encoder>
	</appender>

	<root level="INFO" additivity="false">
		<appender-ref ref="stdout" />
		<appender-ref ref="infoAppender" />
		<appender-ref ref="logstash" />
	</root>
</configuration>

其中的 是在application.properties文件中获取的

server.port=8080
# ......
spring.application.name=es-log
log.destination=10.1.1.1:4567

查看日志

• 启动应用让应用打印日志后，通过head插件查看es，成功创建了记录日志的索引，这里我们设置的是es-log，并且记录了日志。
• 也可直接到kibana页面Management, Create index pattern, 可以找到我们记录日志的索引，添加后可在Discover菜单查看日志
• 还可使用 字段：值 的方式搜索过滤日志，e.g. status:200 AND extension:PHP)