Benchmarks Online

Skip Navigation Links


Page One

Campus Computing News

Changes in Authentication Services Greet the New Academic Year

They're Outta Here!

What the General Access Lab Staff Did During Your Summer Vacation

Computer Buying Tips for Students

Free Software Licenses: A General Overview

Today's Cartoon

RSS Matters

The Network Connection

Link of the Month

WWW@UNT.EDU

Short Courses

IRC News

Staff Activities

Subscribe to Benchmarks Online
    

WWW@UNT.EDU

Logging Your Web Traffic

By Shane Jester, Campus Web Administrator

Do you run an Apache Webserver?  If so, are you tired of the consistent hassle of managing log files and generating statistics?  I recently began playing with an Apache module that may make your life much easier.  It is called mod_log_sql and you can read about it and download it from here: http://www.grubbybaby.com/mod_log_sql/

Mod_log_sql is an Apache module that allows you to log your Apache traffic to an SQL server.  It used to be called mod_log_mysql, but it is no longer limited to just the MySQL database.  To get it running you simply need to have an sql server running (it can be on the localhost or a completely separate machine) and install the module in your Apache server.  The module comes with scripts that allow you to setup the tables on the fly for your server, or for those who like to get their hands dirty, there are specific directions on manually setting up the tables.

What are the advantages to using this module?

There are many advantages. The first advantage is the ease of managing the apache log files.  The module allows you to log traffic simultaneously to the standard log files as well as the sql server.  This means that I can more regularly rotate my apache log files to keep them under a reasonable file size without the headache of maintaining the data for end-of-month statistics.  All the data is now in the sql database.  This point leads to a second advantage.  Since the log traffic is located in a database, I can setup scripts that generate simple or complex statistical reports for my end-users on the fly.  They won't have to wait for me each month to get their web traffic information.  Additionally, as a system administrator I can make more complex and exact searches on the database when I need to look for specific log entries than I could just using "grep".  Also, the module comes with some perl scripts that allow you to extract the data from the database in a Combined Log format that you can pipe to your favorite log analysis tools for more complicated statistical information.

What about reliability? 

What it the SQL server goes down?  What about very high traffic loads?  Well, the module has fail-over built-in.  If the Apache server can't get access to the SQL server, it will start logging the sql insert statements to a text file that you designate.  You can then either automatically or manually insert the data into the SQL server once you have reestablished a connection to the SQL server.  As far as traffic goes, I have been running it on my system which generates approximately thirty-six million hits monthly for over month now with no problems.  I do get occasional entries in the failover file when the sql server gets very busy, but a simple UNIX cron job inserts them every night and resets the failover file.

The Bottom Line

There are many other uses and advantages to using this module, and the bottom line is it gives you greater flexibility with minimal overhead.  Oh yeah, and it's free!  So visit the website and give it a try.  If nothing else, you might learn something.