This article describes how to set up AWStats for Web Farm of servers working in Windows 2008 and IIS 7.0. I havent found anything helpfull in the Internet so I decided to post my solution here.

 I have a site working in Network Load Balancing on 2 Web servers. Lets call them Webserver1 and Webserver2. There is also third server which runs SQL Server and Domain Controller. Lets call him Sqlserver.

The site logs are collected on 2 machines as they work in Network Load Balancing. What we need to do is to merge logs and generate awstats from those merged logs.

 In my solution I merge logs from webservers and put them on shared folder which is located on sqlserver. Then I generate logs on one of the Webservers by using standard awstats.pl script. Thanks to that I keep oryginal log files on every server for other analytics ( for example I can query each server' logs to all http 500 errors )

 Step 1:

Add domain user IISshare which will have access to all shared folders with logs.  

 Step 2:

In Windows 2008 logs are collected in C:\inetpub\logs\LogFiles in subfolders like: W3SVC4. What we need to do is to share this folder. Remember to add "Change" share permission for IISShare domain user and "Modify" NTFS permission for C:\inetpub\logs\LogFiles folder on both srvers.

Then we need to add one more shared folder on Sqlserver where all merged logs will be storred. Please note that some disk space is needed for this storage. Lets say merged logs will be storred in F:\IISLogFiles folder on Sqlserver machine. Again we need to share this folder with "Change" permission and add NTFS "Modify" permission to IISShare domain user. 

Ok now we have 3 shared folders with read/write access for IISShare user.

Step 3:

I spent some time to find what is the log folder name for IIS website. I havent found it in the IIS Manager so I exported IIS configuration to file ( IIS Manager, click on webserver name, then shared configuration from features view and click export on the right pane ). Then edit applicationHost and find lines like:

Folder for site.com will be called W3SVC8 as the id=8 for this site. Please note that the best rutine is to use shared configuration for webservers working in network load balancing. All IIS websites have the same ids in both servers what makes life easier.

Step 4:

I wrote perl script for merging IIS logs from both sites and moving them to shared folder on Sqlserver machine. Below is perl script code which must be saved in MergeLogs.pl file on Sqlserver disk. I use logresolvemerge.pl script which is by default provided in AWStats installation. Please copy this script to F:\IISLogFiles folder on Sqlserver machine.

#! perl

# Specify function library
use Time::Local;
use File::Copy;

my $argc;   # Declare variable $argc. This represents
            # the number of commandline parameters entered.
 
$argc = @ARGV; # Save the number of commandline parameters.
if (@ARGV==0)
{
  # The number of commandline parameters is 0,
  # so print an Usage message.
  #
  usage();  # Call subroutine usage()
  exit();   # When usage() has completed execution,
            # exit the program.
}

# Convert today to epoch seconds
$today = timelocal(localtime);

# Subtract 1 day
@yesterday = localtime($today - (24 * 60 * 60));

# Extract year, month and day, and correct base offsets

$year  = (@yesterday)[5] + 1900;
$month = (@yesterday)[4] + 1;
$day   = (@yesterday)[3];

# Display the result, correctly formated
$data = sprintf("%02d%02d%02d", $year, $month, $day);

$data = substr($data,2,6);
# running logresolvemerge on files u_ex$data from both Webserver1 and Webserver2 machines

# path to the destination folder
$targetPath = "\\\\Sqlserver\\\\IISLogFiles";

# path to log source folder on server 1
$sourcePath1 = "\\\\webserver1\\LogFiles\\@ARGV[0]";

# path to log source folder on server 2
$sourcePath2 = "\\\\webserver2\\LogFiles\\@ARGV[0]";

if ((-e "$sourcePath1\\u_ex$data.log") && (-e "$sourcePath2\\u_ex$data.log"))
{
 $command = "perl $targetPath\\logresolvemerge.pl $sourcePath1\\u_ex$data.log $sourcePath2\\u_ex$data.log > $targetPath\\@ARGV[1]\\u_ex$data.log"; 
 
 system($command);
}
else
{
 if (-e "$sourcePath1\\u_ex$data.log")
 {
  # only log file on server 1 exists
  # printf("Only log file on server 1 exists");
  copy("$sourcePath1\\u_ex$data.log","$targetPath\\@ARGV[1]\\u_ex$data.log");
 } 
 if (-e "$sourcePath2\\u_ex$data.log") 
 {
  # only log file on server 2 exists
  # printf("Only log file on server 2 exists");
  copy("$sourcePath2\\u_ex$data.log","$targetPath\\@ARGV[1]\\u_ex$data.log");

 }
}
# Adding first line in the log file to allow AWStats processing. The line is sometimes missing after running logresolvemerge

# rename processing log file
system("rename f:\\IISLogFiles\\@ARGV[0]\\u_ex$data.log tmp.txt");
# merge files
system("copy line.txt+\"f:\\IISLogFiles\\@ARGV[0]\\tmp.txt\" \"f:\\IISLogFiles\\@ARGV[0]\\u_ex$data.log\"");
# go to parent direcotry
system("del f:\\IISLogFiles\\@ARGV[0]\\tmp.txt");
exit(0);

sub usage
{
  print "Usage: perl mergeIISLogs.pl LocalFolderName DomainName\n";
  print "Example:\n";
  print "perl mergIISLogs.pl \\\\sqlserver\\IISLogFiles\\logresolvemerge.pl \\\\webserver1\\LogFiles\\[LogFolder]\\u_ex$data.log \\\\webserver2\\LogFiles\\[LogFolder]\\u_ex$data.log > \\\\Sqlserver\\IISLogFiles\\[MergedLogFolder]\\u_ex$data.log";
}

As you can see above the MergeLogs.pl script takes 2 input arguments. First argument is the W3SVC folder name where the site logs are collected on webservers. Secong argument is destination folder where merged logs will be storred. On my servers I have the folowing folders structure:

On Sqlserver:

F:\IISLogFiles\site1.com
F:\IISLogFiles\anothersite.com

Please remember to set SourcePath1, SourcePath2 and targetPath in the above script.

Ok so now we have script for merging logs. Lets go to the next step.

Step 5:

Now we need to write batch file which will be runned in Task Scheduler. Ok so lets create c:\awstats.bat file with the folowing content:

C:\Perl64\bin\perl.exe MergeLogs.pl W3SVC2 Site.com

C:\Perl64\bin\perl.exe MergeLogs.pl W3SVC3 Site1.com

Then add task in Windows Task Scheduler ( it is in Administration tools ) and run it daily.

So every day we will have merged IIS Log files merged in F:\IISLogFiles folder on Sqlserver from now on. One more thing needs explonation in my script. The few folowing lines of code from MergeLogs.pl add at the beggining of each merged log file the content of "line.txt" file.

# rename processing log file
system("rename f:\\IISLogFiles\\@ARGV[0]\\u_ex$data.log tmp.txt");
# merge files
system("copy line.txt+\"f:\\IISLogFiles\\@ARGV[0]\\tmp.txt\" \"f:\\IISLogFiles\\@ARGV[0]\\u_ex$data.log\"");
# go to parent direcotry
system("del f:\\IISLogFiles\\@ARGV[0]\\tmp.txt");

Sometimes logresolvemerge cut first few comments lines from log files. Without those lines awstats.pl script can not parse log files.There should be line.txt file placed in \\Sqlserver\IISLogFiles folder with the folowing content:

#Software: Microsoft Internet Information Services 7.0
#Version: 1.0
#Date: 2008-07-17 08:41:13
#Fields: date time cs-method cs-uri-stem cs-uri-query cs-username c-ip cs-version cs(User-Agent) cs(Referer) sc-status sc-bytes

Step 6:

Ok so now it is time for running awstats.pl script. You can run this script on any of the servers because merged log files are accessible on the \\sqlsevrver\IISLogFiles. I run awstats.pl script on sqlserver machine.

For every website we need config file created. You can edit awstats.model.conf. Make sure log format is IIS log formach choosen and there is proper path to the log files set.It should be similar to the one below:

LogFile="\\Sqlserver\IISLogFiles\site.com\u_ex%YY-24%MM-24%DD-24.log"

Now we need to create awstats_generate.bat file which will be runned by Taks Scheduler. The content should be:

perl awstats.pl -config=site.com -update

Please remember to run awstats.bat ( merging log files ) before awstats_generate.bat ( generating awstats from merged logs ) in task scheduler.

Ok  AWStats are done