Varnish logs analized with awstats | Multiple virtual servers

Written by
Date: 2013-01-06 23:13:35 00:00

Varnish is getting quite popular these days. If you run Varnish in front of Apache and PHP you will unload Apache a lot, and will improve you server hundreds of times.

But this have another effect, because now varnish is handling the load, Apache does not see the hits to your server, therefore it can not log the visits.

If you want to log them, and analyze later you will have to make Varnish do the job instead of Apache.

At the end of this tutorial you will have awstats installed, and analyzing logs for all your virtual hosts.

Install Awstats on Ubuntu

First things first, lets install awstats (I will assume you already have Apache and Varnish configured and running)

sudo apt-get install awstats

I will assume you have two virtual hosts. www.domain1.com and www.domain2.com

Create one configuration file for each of them:

sudo cp /etc/awstats/awstats.conf /etc/awstats/awstats.www.domain1.com.conf

And:

sudo cp /etc/awstats/awstats.conf /etc/awstats/awstats.www.domain2.com.conf

Now edit each one of them, and change this lines (I will show you only domain1.com data)

LogFile="/var/log/varnish/www.domain1.com.log"
SiteDomain="www.domain1.com"
HostAliases="www.domain1.com localhost 127.0.0.1"
LogFormat=1
DNSLookup=0
LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat"

Locate those lines and make the looks like the above ones.

Now we need to install the GeoIP perl module.

Split Log files per virtual server

Varnish uses an independent daemon to log visits, it is called varnishncsa and it will aggregated by default all hits to any virtual server on the same file. Even though awstats can manage combined logs, it is better if the are splitted in one file per virtual server. Thanks to my friend Riccardo I know how to split logs.

Run this command, one per virtual server.

varnishncsa -m "RxHeader:^Host: www.domain1.com$" -a -w /var/log/varnish/www.domain1.com -D

Those lines should go in the /etc/rc.local file.

You will need to make logrotate to rotate those logs.

Run awstats for the first time

Now that we have all prepared, it is time to run awstats for the first time:

sudo /usr/lib/cgi-bin/awstats.pl -config=www.domain1.com -update

You will need to add that line to the cronjob

sudo crontab -e

And add this line for domain1.com, and one per domain.

0 */6 * * * /usr/lib/cgi-bin/awstats.pl -config=www.domain1.com -update > /dev/null

Make awstast visible from the browser

Finally to be able to see the stats, add these lines for every virtual host you have, probably on each file on /etc/apache2/sites-enabled/ folder. These lines goes right before the closing VirtuaHost tag.

Alias /awstatsclasses "/usr/share/awstats/lib/"
Alias /awstats-icon/ "/usr/share/awstats/icon/"
Alias /awstatscss "/usr/share/doc/awstats/examples/css"
ScriptAlias /awstats/ /usr/lib/cgi-bin/
Options ExecCGI -MultiViews +SymLinksIfOwnerMatch

You can now go to http://www.domain1.com/awstats/awstats.pl and see your info. Remember that everybody else can also see your stats, unless you prohibit in the htaccess file for example.