Logrotate

Introduction

As a Linux sysadmin, you know that logs are part of the tools you have to solve problems, when there an error in the server, the only way to know what is happening are the logs.

As important as they are, you need to keep copies of them on your server, but not for ever, you have to decide how much information you need and keep the esential logs, and discard the rest, now, this should be done automatically. Here is where logrotate comes into play, logrotate keeps your log files in disk only for the days, months or whatever time you decide, and delete older log files.

Logrotate is a unix-like utility that manage the log files created by other processes, it can compress old log files, delete older ones, and create the empty new ones when rotating them.

Configuration

As with most Linux tools, the configuration file is at /etc/ folder, the specific file is /etc/logrotate.conf

Configuration options

The most used options in logrotate configuration file are:

size

Size is used to specify the threshold at which the log file will be rotated, when logrotate is run, it will the size of the file, and if is equal or greater than specified size it will be rotated. It can us k, M, or G to set the size.

size 100M

Will set the threshold to 100 Mbytes

daily, weekly, monthly

These options (only one can be used at a time), defines when the log file will be rotated, it is used instead of size, so, no matter the size the log file will be rotated once a day, a week, or a month.

weekly

Will rotate logs weekly

rotate

This is used together with a number that specifies the number of copies to be held on disk, so each time logrotate is run, the oldest copy is deleted, a new one is created and all others are rotated to the new position.

rotate 4

Will keep four copies of the log files on disk

compress

Compress the log files with gzip command

compress

create

Creates the new lof file with specified permissions and group and owner

create 0644 www-data www-data

Creates the log file owned by www-data and with 0644 permissions.

postrotate

All commands between postrotate and endscript will be executed after rotation.

postrotate 
   some commands
endscript

prerotate

All commands between prerotate and endscrip will be executed befor rotation.

prerotate
   some commands
endscript

missingok

Will instruct logrotate to not write an error if the log file is missing

notifempty

Do not rotate the file if it is empty, this is to be used when instead of size you use daily, weekly or monthly.

delaycompress

When some program cannot be told to immediately close it's logfile it delays the compression process

sharedscripts

This options indicates that there are different instances for the same server, Apache in this case, I have the same commands for different directories, and the options indicates that the postrotate section should be run just once per server, not per directory.

Configuration example

Now that we have some of the most used options let us see some examples:

In Ubuntu logrotate config files are organized in two parts: The main file, and a directory with specific sections for each server that generate log files

The main file is located at: /etc/logrotate.conf and looks like this in Ubuntu:

# General options unless overwritten later
weekly
rotate 4
create
include /etc/logrotate.d # Include all files in this directory, server specific files should be there
/var/log/wtmp {
    missingok
    monthly
    create 0664 root utmp
    rotate 1
}
/var/log/btmp {
    missingok
    monthly
    create 0660 root utmp
    rotate 1
}

These are the options I have for my Apache server log files, this is my /etc/logrotate.d/apache file.

/var/www/html/garron.me/log/*.log {

daily                     # Rotate the files daily
missingok                 # If there is no log file, is ok
rotate 14                 # Keep two weeks of info before deleting files
compress                  # Compress rotated log files
delaycompress             # Will delay the compress of file to give time to the server (Apache in this case) to stop writting
notifempty                # Will not rotate the log file if there is no data in it
create 640 root adm       # Will create the new file as root and on the adm group with 0640 permissions
sharedscripts             # Will run postrotate section just once, no matter how many times it appears on the config file
postrotate                # Will run the postorate commands, in this case it test if Apache is running reload it the server
            if /etc/init.d/apache2 status > /dev/null ; then \
                /etc/init.d/apache2 reload > /dev/null; \
            fi;
endscript
} If your site gets to few visits, you may change from daily to weekly, or set a size, in that case this might be the example.

/var/www/html/garron.me/log/*.log {

size 100M                 # Rotate once the file size is 100 Mega Bytes or bigger
missingok                 # If there is no log file, is ok
rotate 14                 # Keep 14 files before deleting
compress                  # Compress rotated log files
delaycompress             # Will delay the compress of file to give time to the server (Apache in this case) to stop writting
create 640 root adm       # Will create the new file as root and on the adm group with 0640 permissions
sharedscripts             # Will run postrotate section just once, no matter how many times it appears on the config file
postrotate                # Will run the postorate commands, in this case it test if Apache is running reload it the server
            if /etc/init.d/apache2 status > /dev/null ; then \
                /etc/init.d/apache2 reload > /dev/null; \
            fi;
endscript
}

Configure logrotate to run periodically with cronjob

The final step is to be sure that logrotate will run every day, in Ubuntu you will surely find a logrotate file in the /etc/cron.daily/ directory, and it will contain this data:

#!/bin/sh

# Clean non existent log file entries from status file
cd /var/lib/logrotate
test -e status || touch status
head -1 status > status.clean
sed 's/"//g' status | while read logfile date
do
    [ -e "$logfile" ] && echo "\"$logfile\" $date"	
done >> status.clean
mv status.clean status

test -x /usr/sbin/logrotate || exit 0
/usr/sbin/logrotate /etc/logrotate.conf

If you need to manuall add logrotate to crontab, you can read this and add this line to cronjob

59 23 * * * /usr/sbin/logrotate /etc/logrotate.conf