Linux中利用logrotate来对log文件进行转储

使用logrotate对日志文件转储，按时或者按大小。

logrotate简介

NAME
       logrotate - rotates, compresses, and mails system logs

SYNOPSIS
       logrotate [-dv] [-f|--force] [-s|--state file] config_file ..

DESCRIPTION
       logrotate  is  designed  to  ease  administration of systems that generate large numbers of log files.  It allows automatic
       rotation, compression, removal, and mailing of log files.  Each log file may be handled daily, weekly, monthly, or when  it
       grows too large.

       Normally,  logrotate  is  run as a daily cron job.  It will not modify a log multiple times in one day unless the criterium
       for that log is based on the log's size and logrotate is being run multiple times each day, or  unless  the  -f  or  -force
       option is used.

       Any  number  of config files may be given on the command line. Later config files may override the options given in earlier
       files, so the order in which the logrotate config files are listed is important.  Normally,  a  single  config  file  which
       includes  any other config files which are needed should be used.  See below for more information on how to use the include
       directive to accomplish this.  If a directory is given on the command line, every file in that directory is used as a  con-
       fig file.

       If  no  command  line arguments are given, logrotate will print version and copyright information, along with a short usage
       summary.  If any errors occur while rotating logs, logrotate will exit with non-zero status.

使用需求

在启动flume时候会将启动日志和接下来的运行日志收集到日志文件中/var/log/flume/agent.out，注意这里的这个日志文件不是log4j配置的日志文件。

/usr/hdp/current/flume-server/bin/flume-ng agent --name agent --conf /etc/flume/conf/agent --conf-file /etc/flume/conf/agent/flume.conf -Dflume.monitoring.type=org.apache.hadoop.metrics2.sink.flume.FlumeTimelineMetricsSink -Dflume.monitoring.node=10.254.100.198:6188 > /var/log/flume/agent.out 2>&1

遇到异常情况该日志文件会变的非常大，产生系统隐患，那么我们就需要控制这个文件的大小，最好可以像log4j一样可以定时的或者按照大小来“转储”日志文件，这就可以使用logrotate这个命令实现。

主流Linux发行版上都默认安装有logrotate包，如果出于某种原因，logrotate没有出现在里头，可以使用apt-get或yum命令来安装。

样例配置

基于上面的需求，我们在/etc/logrotate.d/下面创建自定义的配置文件：

vi /etc/logrotate.d/flume

/var/log/flume/agent.out {
    size 100M
    create
    start 1
    rotate 5
    compress
    copytruncate
    missingok
}

说明一下（详情请参见man logrotate）：

1. size：Log files are rotated when they grow bigger than size bytes.
2. create：Immediately after rotation (before the postrotate script is run) the log file is created (with the same name as the log file just rotated).
3. start：This is the number to use as the base for rotation. For example, if you specify 0, the logs will be created with a.0 extension as they are rotated from the original log files.
4. rotate：Log files are rotated count times before being removed or mailed to the address specified in a mail directive. If count is 0, old versions are removed rather than rotated.
5. compress：Old versions of log files are compressed with gzip(1) by default. See also nocompress.
6. copytruncate：Truncate the original log file in place after creating a copy, instead of moving the old log file and optionally creating a new one.
7. missingok：If the log file is missing, go on to the next one without issuing an error message. See also nomissingok.

效果

默认情况下，logrotate会每天自动执行一次：
在/etc/cron.daily目录下有logrotate执行的脚本（通过crontab程序每天执行一次）

vi /etc/cron.daily/logrotate

如果配置完想立即测试看一下效果，可以使用logrotate的强制执行的命令：

/usr/sbin/logrotate -vf /etc/logrotate.conf