Message-ID: <4BDA5F79.6050504@gmail.com> Date: Thu, 29 Apr 2010 21:41:29 -0700 From: Ken Neighbors User-Agent: Mozilla-Thunderbird 2.0.0.24 (X11/20100328) MIME-Version: 1.0 To: awstats@packages.debian.org Subject: Re: Trying to get /usr/share/awstats/tools/update.sh working for multiple sites References: <4BD5F598.6080201@gmail.com> In-Reply-To: <4BD5F598.6080201@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit I have created a patch to fix this problem I reported earlier (it is patch number 0006 below). I have also included a couple of patches I posted to bugs.debian.org, and I included some patches for additional features and some suggestions. Please feel free to take any of the patches or give me some feedback if you'd like something done differently, or if you'd like me to post these to bugs.debian.org. I hope this is helpful, and I really appreciate the work your team has put into making this Debian package. You can access the patchset here: http://nsds.com/software/dist/awstats/awstats-patches.mbox.txt Or see them individually here: http://nsds.com/software/dist/awstats/ Here is a summary of my proposed changes: ------------------------------------------------------------------------------- 0001 Add patch to report error when config file exists but cannot be read. /usr/lib/cgi-bin/awstats.pl http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=572353 ------------------------------------------------------------------------------- 0002 Add patch to fix truncated nested included config files. /usr/lib/cgi-bin/awstats.pl http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=575545 ------------------------------------------------------------------------------- 0003 Show error messages from cron jobs (update.sh and buildstatic.sh) /usr/share/awstats/tools/buildstatic.sh /usr/share/awstats/tools/update.sh TMPFILE is used to record the output of awstats/awstats_buildstaticpages.pl (from both stdout and stderr). If the return status of awstats/awstats_buildstaticpages.pl is nonzero, then an error occurred and the content of TMPFILE is printed to stderr to show up in a cron email report. Note that a failure of awstats.pl/awstats_buildstaticpages.pl due to a bad config file will now no longer stop it from processing additional config files; update.sh/buildstatic.sh will continue to run awstats.pl/awstats_buildstaticpages.pl on other config files. This feature has both good and bad consequences: an error in a single config file will not interrupt processing of the remaining config files (good); but an error in awstats.local.conf (which is included in all config files) will generate a repeated error message for multiple config files (bad). I think the good outweighs the bad, but if you think that update.sh/buildstatic.sh should stop after the first error, then you can add "exit 1" after "cat $TMPFILE >&2" in both scripts. ------------------------------------------------------------------------------- 0004 Add option to disable nightly generation of static html reports. /etc/default/awstats /usr/share/awstats/tools/buildstatic.sh /usr/share/docs/awstats/README.Debian With inspiration from /etc/default/rsync and /etc/init.d/rsync, I set up an option in /etc/default/awstat called AWSTATS_ENABLE_BUILDSTATICPAGES. If set to "yes" then buildstatic.sh will run every night to build static pages. If set to "no" (which I think should be the default, but feel free to make up your own mind), then buildstatic.sh will not build static pages. I placed a short note in README.Debian about it. ------------------------------------------------------------------------------- 0005 Use "nice" to lower the priority of the AWStats processes when updating databases and generating static html reports from cron. /etc/default/awstats /usr/share/awstats/tools/buildstatic.sh /usr/share/awstats/tools/update.sh Again with inspiration from /etc/default/rsync and /etc/init.d/rsync, I set up an option in /etc/default/awstat called AWSTATS_NICE. It sets the niceness level for awstats.pl and awstats_buildstaticpages.pl run from update.sh and buildstatic.sh. Default value is 10. Note that unlike /etc/init.d/rsync, I did not include error checking for this value. Here is some code from /etc/init.d/rsync you might consider (or just let me know if you'd like me to create a patch that includes this type of error-checking on AWSTATS_NICE): case "x$RSYNC_NICE" in x[0-9]) RSYNC_NICE_PARM="--nicelevel $RSYNC_NICE";; x[1-9][0-9]) RSYNC_NICE_PARM="--nicelevel $RSYNC_NICE";; x) ;; *) log_warning_msg "Value of RSYNC_NICE in $RSYNC_DEFAULTS_FILE must be a value between 0 and 19 (inclusive);" log_warning_msg "ignoring RSYNC_NICE now." ;; esac ------------------------------------------------------------------------------- 0006 Do not use implied default config file /etc/awstats/awstats.conf but instead require sysadmin to install one or more explicit config file(s) so that update.sh and buildstatic.sh will work for either single-site or multi-site installations. This fixes the problem where awstats.pl complains about undefined SiteDomain when using separate config files as described in README.Debian for multiple stats. /usr/share/awstats/tools/buildstatic.sh /usr/share/awstats/tools/update.sh /usr/share/docs/awstats/README.Debian /usr/share/docs/awstats/examples/awstats.default.conf debian/rules I removed "/etc/conf/awstats.conf" from the list of config files to process because it always generates an error message when trying to set up awstats for multiple sites. I think it is better to treat the single-site case as a special case of multiple sites where there is only one site, served by a single config file "awstats.default.conf" which contains one line: Include "awstats.conf". ------------------------------------------------------------------------------- 0007 Improve documentation of logfile permissions and suggest EnableLockForUpdate=1. /usr/share/docs/awstats/README.Debian ------------------------------------------------------------------------------- 0008 Cosmetic improvements to cron scripts (update.sh and buildstatic.sh). /usr/share/awstats/tools/buildstatic.sh /usr/share/awstats/tools/update.sh Feel free to take or leave any of these changes. I used a different method to initialize the values read from /etc/default/awstats (again with inspiration from /etc/init.d/rsync and other init files). Specifically, instead of: [ -r "$DEFAULT" ] && . "$DEFAULT" -NICE=${AWSTATS_NICE:-10} I did this: AWSTATS_NICE=10 [ -r "$DEFAULT" ] && . "$DEFAULT" This method reduces the need for another environment variable ("NICE") because it can use the existing environment variable ("AWSTATS_NICE") defined in /etc/default/awstats (similarly for "AWSTATS_LANG" and "AWSTATS_ENABLE_BUILDSTATICPAGES"). I moved AWSTATS_LANG, Y, and m (the date calls) out of the buildstatic.sh loop since they should remain constant for all config files. I used an alternate method to extract the config file name using posix shell parameter expansions (without calling the external programs ls and sed). ------------------------------------------------------------------------------- Thanks again, Ken