How can I force Archive to re-populate all stats (day, week, month, year) for all websites?

Knowledge base
1

Running 2.2.0 here, working on a log-analytics mode piwik siteid.

I’m running into an issue which I presume must be the fault of the archive routine. In a nutshell, although daily and monthly charts are populated (not enough data really to check yearly), the weekly charts are blank except for the most recent.

Daily:
[attachment 1710 daily.JPG]
Weekly:
[attachment 1711 weekly.JPG]
Monthly:
[attachment 1712 monthly.JPG]

During setup and test of PIwik I have purged and reloaded the log stats a number of times.

This is the import syntax we’re using:


/var/www/piwik/misc/log-analytics/import_logs.py \
        --url=https://example.com/piwik/ \
        --idsite=3 \
        --config=/var/www/piwik/config/config.ini.php \
        --show-progress \
        --show-progress-delay=60 \
        --enable-reverse-dns \
        --exclude-path-from=/var/www/util/piwik_excluded.paths \
        /var/log/apache2/access.log.1

(obviously, for initial import we use something like access.* to load the full history – the above is integrated into logrotate).

And the hourly archive syntax is this:


/usr/bin/php5 /var/www/piwik/console core:archive --url=http://example.com/piwik/

And, for the initial loads we’ve done (after purging the site), we add –force-all-websites switch:


/usr/bin/php5 /var/www/piwik/console core:archive --piwik-domain=http://example.com/piwik/ --verbose --force-all-websites

This looks like it should be the one we want when loading history like this (from the --help text):


--force-all-websites
If specified, the script will trigger archiving on all websites and all past dates.
You may use --force-all-periods=[seconds] to trigger archiving on those websites
that had visits in the last [seconds] seconds.

I might have played with --force-all-periods except the monthly reports seem to be fine.

So, can anyone shed any light on what’s going on here? Is there are an archive syntax that would force rebuilding the weeklies?

Thanks!

2

To re-process the data after your initial import of log in piwik, here is the command that will process for all websites, all periods, and also go back far in the past.


./console core:archive --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 --url=localhost
3

Yup, that worked. Thank-you, sir.

4

Good to hear! I’ve added it as FAQ: Log Analytics tool - Analytics Platform - Matomo

5

I’ve added this to the existing FAQ: How do I force the reports to be re-processed from the logs? - Analytics Platform - Matomo

6

Sorry for digging up an old topic, but I have a two questions strongly related to it.

  1. Is there a way to force rebuilding of the past statistics, but only for two or three specific days and for one site I’m missing?

  2. Is there any place, where the core:archive flags are described in some understandable manner?

Reason I ask is obvious – I have a huge DB to work on, I have data I need to rebuild for two or three days only and I want to avoid rebuilding/reprocessing of the whole data for all sites (hence these flags --force-all-websites --force-all-periods=315576000 --force-date-last-n=1000 are no go for me).

7

Hi there,

  1. see How do I reprocess all websites, all dates and all periods, after initial import of logs? - Analytics Platform - Matomo

  2. try:


./console help core:archive