Awstats MySQL access_log Export

Alright, so I thought I would start posting some of my code here since this blog gets a fair bit of traffic. Anyway, I thought I would kick things off with a small script I wrote just last night and “perfected” this morning. I was a bit disappointed that Awstats doesn’t have a way to pull logs from a MySQL database. I have been logging my Apache access_log information to a MySQL database for quite some time now and last night it hit me that I could actually just export that data into a flat file for Awstats.  The script, though quite simple in design, does that job and does it well.  Simply set a few configuration options at the top and you are good to go.

You can also have the script run the awstats.pl file to generate the stats after the data has been exported.  This is definable in the configuration section at the top of the script.  Everything should be pretty straightforward really, just define the username/password/database/table at the top, set the location you are exporting to, and you are good to go.  Just make sure the user you run it as has permissions to write to wherever you are exporting the log.  It can be called from command line using “php /path/to/generate_awstats_log.php” or as a cron job similar to this “0 0  * * *  root php /path/to/generate_awstats_log.php > /dev/null 2>&1” which would run every night at midnight.

It’s also very quick, it can export tens of thousands of records in seconds, you will actually notice that it takes Awstats longer to generate your stats then it does for this script to create the file.  If you have any questions or problems you can post them in the comments below or in the forums over at SourceForge.  I’m not going to post the script here as I don’t want to keep it updated in two places but you can grab a copy over at SourceForge.  Enjoy!!

3 comments on “Awstats MySQL access_log Export

  1. What about moving data the other way? Do you have, or know of, a way to push the stats that AWStats produces into a MySQL database?

    And what about Apache’s error log?

    • Hi there Ted!

      I do not currently know of a way to do that, no. I don’t know that awstats even produces any data that you could push over. It might just generate the web page and call it a day. I can check into it though and see what I can find.

      Since the error log was still a flat file I didn’t have to worry about trying to export it from MySQL. Having your error log in a MySQL database seems like a bad idea anyways because if it can’t connect it can’t log the error.

      If you want to chat feel free to shoot me an email to dan@dansrandomness.com or you can post back here.

      Thanks for the feedback,
      Dan

  2. Hi Ted,

    AWStats produces a AWSTATS DATA FILE that hold the calculated value for the configured domain.

    The configuration point for this file is located in your AWSTATS DATA FILE.
    vim /etc/awstats/awstats.yourdomain.com.conf

    When AWStats updates its statistics, it stores results of its analysis in files (AWStats database). All those files are written in the directory, defined by the “DirData” parameter. Set this value to the directory where you want AWStats to save its database and working files into.

    Hope helped.

Leave a Reply

Your email address will not be published. Required fields are marked *