This page is a glossary of common log formats that can be parsed with the Tail input plugin.
Apache Access Log
Use
format apache2
as shown below:<source> @type tail format apache2 tag apache.access path /var/log/apache2/access.log </source>
Apache Error Log
Use a regular expression. See the
format
field in the following sample configuration.<source> @type tail format /^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\] \[pid (?<pid>[^\]]*)\] \[client (?<client>[^\]]*)\] (?<message>.*)$/ tag apache.error path /var/log/apache2/error.log </source>
Depending on your particular error log format, you may need to adjust the regular expression above. You can test your format using fluentd-ui’s in_tail editor or Fluentular.
Maillog
Use a regular expression. See the
format
field in the following sample configuration.<source> @type tail format /^(?<time>[^ ]+) (?<host>[^ ]+) (?<process>[^:]+): (?<message>((?<key>[^ :]+)[ :])? ?((to|from)=<(?<address>[^>]+)>)?.*)$/ tag postfix.maillog path /var/log/maillog </source>
Nginx Access Log
Use
format nginx
as shown below:<source> @type tail format nginx tag nginx.access path /var/log/nginx/access.log </source>
Nginx Error Log
Use the
format*
andmultiline_flush_interval
fields in the following sample configuration. Applications running under Nginx can output multi-line errors including stack traces, so the multiline mode is a good fit.<source> @type tail tag nginx.error path /var/log/nginx/error.log format multiline format_firstline /^\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2} \[\w+\] (?<pid>\d+).(?<tid>\d+): / format1 /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)/ multiline_flush_interval 3s </source>
If you know your error log will only contain single lines, you can use the below simpler configuration with just a
format
.<source> @type tail format /^(?<time>\d{4}/\d{2}/\d{2} \d{2}:\d{2}:\d{2}) \[(?<log_level>\w+)\] (?<pid>\d+).(?<tid>\d+): (?<message>.*)$/ tag nginx.error path /var/log/nginx/error.log </source>
GlusterFS Logs
Use the GlusterFS input plugin.