I managed to get live direct logging working on a Centos 6 install. The huge obstickale was rsyslog is used instead of syslog-ng and nginx sysloging was crippiled and can not be used.
I"me just posting the relevent files. Most of the info was gathered from http://syshero.org/post/68174083489/nginx-syslog-ing-without-breaking-the-bank-or-patching[/url] and [url=https://github.com/piwik/piwik/tree/master/misc/log-analytics]https://github.com/piwik/piwik/tree/master/misc/log-analytics
First off, relevant lines from nginx: Note that the log file must be created first using mkfifo to create a named pipe.
# Log to a named pipe that rsyslog is reading and exec a shell script to record directly to piwik
log_format piwik '{"ip": "$remote_addr",'
'"host": "$host",'
'"path": "$request_uri",'
'"status": "$status",'
'"referrer": "$http_referer",'
'"user_agent": "$http_user_agent",'
'"length": $bytes_sent,'
'"generation_time_milli": $request_time,'
'"date": "$time_iso8601"}';
access_log /var/lib/nginx/access.log piwik
The relevant lines from rsyslog. Note I had to use the legasy format for the input module.
$ModLoad imfile # needs to be done just once
# File 1
$InputFileName /var/lib/nginx/access.log
$InputFileTag nginx_log:
$InputFileStateFile /var/lib/rsyslog/nginx_statefile1
$InputFileSeverity info
$InputFileFacility local7
$InputFilePollInterval 5
$InputRunFileMonitor
if $syslogtag contains 'nginx_log' and $syslogfacility-text == 'local7' then ^/path/to/piwik.sh;nginxlog
:syslogtag, contains, "nginx_log" ~
Now the helper script that calls the importer. Note that when using the importer, both index.php and piwik.php must be aviable as the importer will call both of those. The index.php to get a session variable (I think) then calls the piwik.php to record the log.
#!/bin/sh
echo "$1" | /path/to/piwik/misc/log-analytics/import_logs.py \
--url=http://statsapi.localhost.org/ --token-auth=<token> \
--idsite=1 --recorders=4 --enable-http-errors --enable-http-redirects --enable-static --enable-bots \
--log-format-name=nginx_json -
The echo “$1” is used because rsyslog calls the helper script using the %msg% as the one and only argument instead of syslog-ng of pipeing the %msg% on the standard input.
I give permission for anyone to use this info and create a nice tuturial post plus update the info at https://github.com/piwik/piwik/tree/master/misc/log-analytics