Last modified August 11,
2024
under libmaxminddb-1.10.0 we finally type
./configure
make
and as root make install
We return to the goaccess-19.3 directory and type
./configure --enable-utf8 --enable-geoip=mmdb
here the result
Your build configuration:
Prefix :
/usr/local
Package :
goaccess
Version : 1.9.3
Compiler flags : -pthread
Linker flags : -lncursesw -lmaxminddb
-lpthread -lintl -lintl
UTF-8 support : yes
Dynamic buffer : no
ASan
: no
Geolocation : GeoIP2
Storage method : In-Memory with On-Disk Persistent
Storage
TLS/SSL : no
Bugs
: hello@goaccess.io
type make then as root make install
Everything happens in the file /usr/local/etc/goaccess/goaccess.conf
I only changed three
parameters which can be found below
######################################
# Time Format Options (required)
######################################
#
# The hour (24-hour clock) [00,23]; leading zeros are permitted
but not required.
# The minute [00,59]; leading zeros are permitted but not
required.
# The seconds [00,60]; leading zeros are permitted but not
required.
# See `man strftime` for more details
#
# The following time format works with any of the
# Apache/NGINX's log formats below.
#
time-format %H:%M:%S
(...)
######################################
# Date Format Options (required)
######################################
#
# The date-format variable followed by a space, specifies
# the log format date containing any combination of regular
# characters and special format specifiers. They all begin with
a
# percentage (%) sign. See `man strftime`
#
# The following date format works with any of the
# Apache/NGINX's log formats below.
#
date-format %d/%b/%Y
(...)
######################################
# Log Format Options (required)
######################################
log-format %v %h %^[%d:%t %^] (%T s) "%r" %s %b "%R" "%u"' --date-format=%d/%b/%Y --time-format=%T
This last option is probably the most important option, it is the format of the log file. The custom format that is indicated corresponds to my funix logs.
www.funix.org:80 74.125.217.134 - - [10/Aug/2024:02:04:33 +0200] "GET /images/style.css HTTP/1.0" 301 561 "http://www.funix.org/" "Mozilla/5.0 (Linux; Android 4.0.4; Galaxy Nexus Build/IMM76B) AppleWebKit/537.36 (KHTML, like Gecko; Mediapartners-Google) Chrome/126.0.6478.182 Mobile Safari/537.36"
Just type goaccess with the name of the log file after it, everything happens in a terminal
This site will give you an idea of all the information you can have. Now if you prefer a visualization via html, just type
goaccess www.funix.org.log -a -o funixlog.html
and this will give something more attractive, below two views
tar xvfz awstats-7.9.tar.gz
This will give us the awstats-7.9 directory . First we will have to slightly modify the apache httpd.conf configuration file in case you are using virtual hosts, here is an example configuration
<VirtualHost
192.168.13.11:1234>
ServerName
funix.homelinux.org
DocumentRoot
/roger/homepage/online
ErrorLog
logs/online-error_log
TransferLog
logs/online-access_log
CustomLog
logs/online-access_log combined
</VirtualHost>
We must put combined for the CustomLog parameter . Don't forget to restart apache in case of modification
systemctl restart
httpd.service
Now we need to create the
directory /etc/awstats
mkdir /etc/awstats
For the configuration of awstats we will go to the tools directory
cd awstats-7.9/tools
In the case of an update we will type
perl
awstats_updateall.pl now
-awstatsprog=/absolute-path/awstats-7.9/wwwroot/cgi-bin/awstats.pl
-configdir=/etc/awstats/
/etc/awstats is the directory where your servers configuration files are located. Here is the result
Running
'"/home/olivier/web/awstats-7.9/wwwroot/cgi-bin/awstats.pl"
-update -config=funix.kervao.fr -configdir="/etc/awstats/"'
to update config funix.kervao.fr
Failed to open blacklist:
No such file or directory
Running
'"/home/olivier/web/awstats-7.9/wwwroot/cgi-bin/awstats.pl"
-update -config=www.funix.org -configdir="/etc/awstats/"' to
update config www.funix.org
Create/Update database for
config "/etc/awstats/awstats.www.funix.org.conf" by AWStats
version 7.9 (build 20230108)
From data in log file
"/usr/local/apache2/logs/funix.org/www.funix.org.log"...
Phase 1: First bypass old
records, searching new record...
Direct access after last
parsed record (after line 6479)
Jumped lines in file: 6479
Found 6479 already
parsed records.
Parsed lines in file: 0
Found 0 dropped
records,
Found 0 comments,
Found 0 blank
records,
Found 0 corrupted
records,
Found 0 old records,
Found 0 new
qualified records.
Running
'"/home/olivier/web/awstats-7.9/wwwroot/cgi-bin/awstats.pl"
-update -config=www.hoarau.org -configdir="/etc/awstats/"'
to update config www.hoarau.org
Create/Update database for
config "/etc/awstats/awstats.www.hoarau.org.conf" by AWStats
version 7.9 (build 20230108)
From data in log file
"/usr/local/apache2/logs/hoarau.org/www.hoarau.org.log"...
Phase 1: First bypass old
records, searching new record...
Direct access after last
parsed record (after line 7827)
Jumped lines in file: 7827
Found 7827 already
parsed records.
Parsed lines in file: 0
Found 0 dropped
records,
Found 0 comments,
Found 0 blank
records,
Found 0 corrupted
records,
Found 0 old records,
Found 0 new
qualified records.
In the case of a first
installation, we will type (in the example below, I have two Apache servers ( version 1.3 and 2) with the virtual web servers http://funix.homelinux.org:1234 and http://ohoarau.homelinux.org:1234 ) as root
perl awstats_configure.pl
here is the result
----- AWStats awstats_configure
1.0 (build 20140126) (c) Laurent Destailleur -----
This tool will help you to configure AWStats to analyze
statistics for
one web server. You can try to use it to let it do all that is
possible
in AWStats setup, however following the step by step manual
setup
documentation (docs/index.html) is often a better idea. Above
all if:
- You are not an administrator user,
- You want to analyze downloaded log files without web server,
- You want to analyze mail or ftp log files instead of web log
files,
- You need to analyze load balanced servers log files,
- You want to 'understand' all possible ways to use AWStats...
Read the AWStats documentation (docs/index.html).
-----> Running OS detected: Linux, BSD or Unix
Warning: AWStats standard directory on Linux OS is
'/usr/local/awstats'.
If you want to use standard directory, you should first move all
content
of AWStats distribution from current directory:
/usr/local/linux/web/awstats-7.9
to standard directory:
/usr/local/awstats
And then, run configure.pl from this location.
Do you want to continue setup from this NON standard directory
[yN] ? y
here it finds the
configuration files of my two Apache servers
-----> Check for web server
install
Found Web server Apache
config file '/usr/local/apache/conf/httpd.conf'
Found Web server Apache
config file '/usr/local/apache2/conf/httpd.conf'
It modifies both configuration files, it also modifies files to update the log archiving format
-----> Check and complete web
server config file '/usr/local/apache2/conf/httpd.conf'
Warning: You Apache config file contains directives to write
'common' log files
This means that some features can't work (os, browsers and
keywords detection).
Do you want me to setup Apache to write 'combined' log files
[y/N] ? y
Add 'Alias /awstatsclasses
"/usr/local/linux/web/awstats-7.9/wwwroot/classes/"'
Add 'Alias /awstatscss
"/usr/local/linux/web/awstats-7.9/wwwroot/css/"'
Add 'Alias /awstatsicons
"/usr/local/linux/web/awstats-7.9/wwwroot/icon/"'
Add 'ScriptAlias /awstats/
"/usr/local/linux/web/awstats-7.9/wwwroot/cgi-bin/"'
Add '<Directory>' directive
AWStats directives added to Apache config file.
It updates its own configuration file
-----> Update model config
file
'/usr/local/linux/web/awstats-7.9/wwwroot/cgi-bin/awstats.model.conf'
File awstats.model.conf updated.
-----> Need to create a new config file ?
Do you want me to build a new AWStats config/profile
file (required if first install) [y/N] ? y
Here we indicate the URL of the web server to analyze (address:port)
-----> Define config file name to create
What is the name of your web site or profile analysis ?
Example: www.mysite.com
Example: demo
Your web site, virtual server or profile name:
> funix.homelinux.org:1234
Defining the path to store the configuration files for each server to be analyzed
-----> Define config file path
In which directory do you plan to store your config file(s) ?
Default: /etc/awstats
Directory path to store config file(s) (Enter for default):
>
Creating the server configuration file mentioned above
-----> Create config file
'/etc/awstats/awstats.funix.homelinux.org:1234.conf'
Config file
/etc/awstats/awstats.funix.homelinux.org:1234.conf created.
Apache is restarted automatically
-----> Restart Web server with '/sbin/service httpd
restart'
Redirecting to /bin/systemctl restart httpd.service
-----> Add update process inside a scheduler
Sorry, configure.pl does not support automatic add to cron yet.
You can do it manually by adding the following command to your
cron:
/usr/local/linux/web/awstats-7.9/wwwroot/cgi-bin/awstats.pl
-update -config=funix.homelinux.org:1234
Or if you have several config files and prefer having only one
command:
/usr/local/linux/web/awstats-7.9/tools/awstats_updateall.pl now
Press ENTER to continue...
A SIMPLE config file has been created:
/etc/awstats/awstats.funix.homelinux.org:1234.conf
You should have a look inside to check and change manually
main parameters.
You can then manually update your statistics for
'funix.homelinux.org:1234' with command:
> perl awstats.pl -update -config=funix.homelinux.org:1234
You can also read your statistics for
'funix.homelinux.org:1234' with URL:
>
http://localhost/awstats/awstats.pl?config=funix.homelinux.org:1234
Press ENTER to finish...
He created a configuration
file for the site concerned /etc/awstats/awstats.funix.homelinux.org:1234.conf in this file I modified the following parameters:
- The one which indicates
the path of the raw log file to analyze
LogFile="/usr/local/apache/logs/online-access_log"
- to enable DNS name
resolution
DNSLookup=1
- Directory where awstats will store its statistics
DirData="/usr/local/var/awstats"
- To indicate the index file
of your site
DefaultFile="index.php"
- To ignore the machines of
your domain
SkipHosts="127.0.0.1
REGEX[^192\.168\.13\.] REGEX[\.kervao\.fr$]"
- If your site is identified by another URL, you must add it here (put a space as a separator field)
HostAliases="funix.homelinux.org:1234"
To combat spam referrers , you need to enable the following variable
SkipReferrersBlackList="/etc/awstats/blacklist.txt"
with a blacklist.txt file that we will retrieve under awstats-7.9/wwwroot/cgi-bin/lib
A much more recent blacklist can be found here : https://perishablepress.com/blacklist/ultimate-referrer-blacklist.txt
in this file according to my
tests it would seem that the first part which begins with RewriteCond is useless for awstats, it is only useful if you filter
SPAM referrers directly on access to your site with a .htaccess be careful this method will result in slowing down
access to your site. See here an interesting
discussion on the subject.
It's only the second part that is actually useful and works with awstats , it starts with
# This is the
URL blacklist from the chongqed.org database
# it is available from
http://blacklist.chongqed.org/
# You can use each line
below as a regular expression
# that can be tested
against URLs on your wiki.
# The last spammer was
added on 2008-09-11 10:14:51
# Check
http://blacklist.chongqed.org/ for updates
I started to complete the list with the following lines:
https?:\/\/([^\/]*\.)?proxtrail\.ru
https?:\/\/([^\/]*\.)?denterum\.ru
https?:\/\/([^\/]*\.)?profeservice\.ru
https?:\/\/([^\/]*\.)?bazakanstovarov\.com
https?:\/\/([^\/]*\.)?balkanfarma\.ru
https?:\/\/([^\/]*\.)?sobervoditel\.ru
https?:\/\/([^\/]*\.)?avtokor-23\.ru
https?:\/\/([^\/]*\.)?avtokor23\.ru
https?:\/\/([^\/]*\.)?xn--j1at1a.xn--p1ai
https?:\/\/([^\/]*\.)?rupolitshow\.ru
https?:\/\/([^\/]*\.)?vyezd-viyezd\.ru
https?:\/\/([^\/]*\.)?igru-2015\.ru
https?:\/\/([^\/]*\.)?jeribetejewu\.c0\.pl
https?:\/\/([^\/]*\.)?creditservise\.ru
https?:\/\/([^\/]*\.)?megamashiny\.com
https?:\/\/([^\/]*\.)?seoxbeep\.ru
https?:\/\/([^\/]*\.)?woman3050\.ru
https?:\/\/([^\/]*\.)?www.vselgoty\.ru
https?:\/\/([^\/]*\.)?playpokeronline\.dk
https?:\/\/([^\/]*\.)?medical\.in\.ua
https?:\/\/([^\/]*\.)?viagralevitradzheneriki\.ru
https?:\/\/([^\/]*\.)?\.com\.ua
https?:\/\/([^\/]*\.)?tasgroup\.it
https?:\/\/([^\/]*\.)?\.co\.ua
https?:\/\/([^\/]*\.)?kiev\.ua
https?:\/\/([^\/]*\.)?bringtwo\.net
https?:\/\/([^\/]*\.)?cleaningservices\.kiev\.ua
https?:\/\/([^\/]*\.)?studio-topkapi\.eu
https?:\/\/([^\/]*\.)?kruchen\.com\.ua
https?:\/\/([^\/]*\.)?freedom\.co\.ua
https?:\/\/([^\/]*\.)?bringtwo\.net
https?:\/\/([^\/]*\.)?studio-topkapi\.eu
https?:\/\/([^\/]*\.)?c0\.pl
https?:\/\/([^\/]*\.)?bazakanstovarov\.com
https?:\/\/([^\/]*\.)?xikiz\.com
https?:\/\/([^\/]*\.)?lesbianmilf\.xblog\.in
https?:\/\/([^\/]*\.)?pamyatniki-in-kiev\.com\.ua
https?:\/\/([^\/]*\.)?carivka\.com\.ua
https?:\/\/([^\/]*\.)?agent-05\.su
https?:\/\/([^\/]*\.)?websolution\.com\.ua
https?:\/\/([^\/]*\.)?pamjatnik\.com\.ua
https?:\/\/([^\/]*\.)?startimes\.com
https?:\/\/([^\/]*\.)?arktech\.co
https?:\/\/([^\/]*\.)?goohey\.com
https?:\/\/([^\/]*\.)?himalayan-imports\.com
https?:\/\/([^\/]*\.)?pizza-imperia\.com
https?:\/\/([^\/]*\.)?nowellgroup\.com
https?:\/\/([^\/]*\.)?ddrgame\.com
https?:\/\/([^\/]*\.)?torrinomedica\.it
https?:\/\/([^\/]*\.)?oliveriobalcells\.com
https?:\/\/([^\/]*\.)?giocagiocagioca\.com
for this to be taken into account with awstats you have to delete what is under /usr/local/var/awstats/ and restart awstats for all the weekly log files
Remember to create this
directory beforehand
mkdir /usr/local/var/awstats
To see if everything works, we now place ourselves as root under awstats-7.9/wwwroot/cgi-bin and type
./awstats.pl -config=funix.homelinux.org:1234 -update
This is the result
Create/Update
database for config
"/etc/awstats/awstats.www.hoarau.org.conf" by AWStats
version 7.9 (build 20230108)
From data in log file
"/usr/local/apache2/logs/hoarau.org/www.hoarau.org.log"...
Phase 1: First bypass old
records, searching new record...
Direct access after last
parsed record (after line 7827)
Jumped lines in file: 7827
Found 7827 already
parsed records.
Parsed lines in file: 0
Found 0 dropped
records,
Found 0 comments,
Found 0 blank
records,
Found 0 corrupted
records,
Found 0 old records,
Found 0 new
qualified records.
On previous versions there was a slight bug, here is a typical log line
192.168.0.11
- - [19/Feb/2004:10:58:17 +0100] "GET /images/journal.gif
HTTP/1.1" 304 -
"http://funix.homelinux.org:1234/fr/linux/index.php?ref=tips&page=menu"
"Mozilla/5.0 (X11; U; Linux i686; fr-FR; rv:1.4) Gecko/20
030630"
Or sometimes Apache archives this
192.168.0.11
- - [19/Feb/2004:10:58:24 +0100] "GET / HTTP/1.1" 200 17493
In the absence of information, it leaves empty fields and awstats does not appreciate it (it does not analyze the logs...). I do not have enough perspective to know if it was corrected.
If you have a second virtual site, create another file /etc/awstats/awstats.my-virtual-domain.conf on the same model as before. In my case, just type
cp
/etc/awstats/awstats.funix.homelinux.org:1234.conf
/etc/awstats/awstats.ohoarau.homelinux.org:1234.conf
In this file remember to modify the parameters for the URL and the Apache log file to analyze.
Note that given the method
used (see next chapter) to use awstats , I deleted the lines that awstats added to
my Apache configuration files.
If you want to use awstats as an accessible cgi-bin from Apache, you must leave these lines.
if you have an error like
bash: /usr/local/linux/web/awstats-7.9/tools/awstats_buildstaticpages.pl: /usr/bin/perl^M: bad interpreter: No such file or directoryHere are some (non-exhaustive) overviews
[ Back to FUNIX home page ] | [ back to top of page ] |