Web log analysis on OpenBSD using GoAccess

Feb 13, 2019, updated Feb 15, 2019 openbsd logs goaccess stats htpasswd crontab

GoAccess is an open source real-time web log analyzer and interactive viewer that runs in a terminal in *nix systems or through your browser.

Requirements

This only works with httpd, tested on OpenBSD 6.4
Assumes your log style is set to common (default)
Install the GoAccess package

$ doas pkg_add goaccess

Run from the command line

Read access.log, remove the domain from beginning of each line and save as stats.log

grep example.com /var/www/logs/access.log | cut -d' ' -f2- > stats.log

Run GoAccess excluding local IP’s and specifing common style

goaccess -e 192.168.1.1-192.168.1.255 --log-format=COMMON stats.log

Update HTML report every hour

Make a folder in your website for stats.html

doas mkdir /var/www/example.com/stats

Create script to be run by root’s crontab, then paste in the two commands below

doas vi /etc/hourly.local  
zgrep example.com /var/www/logs/access.log | cut -d' ' -f2- > stats.log  
/usr/local/bin/goaccess >/dev/null 2>&1 stats.log -o /var/www/htdocs/chevybeef.com/stats/stats.html --log-format=COMMON -e 192.168.1.1-192.168.1.255

Change to the root user

su -

Edit root’s crontab with

# crontab -e

Paste in the following line at the bottom

@hourly /bin/sh /etc/hourly.local

Save and exit
Now, every hour on the hour the stats report HTML will be updated in:

https://www.example.com/stats/stats.html

Secure the stats folder with htpasswd

Create a user with these properties

stats:*:1002:1002:Stats Viewer:/home/stats:/sbin/nologin  
doas htpasswd /var/www/htpasswd stats  

Enter a strong password for the stats user and confirm
Change ownership of the htpasswd file

doas chown www:daemon /var/www/htpasswd

Modify /etc/httpd.conf and include

location "/stats*" {  
	authenticate with "/htpasswd"  
}  

Reload the configuration

doas rcctl reload httpd