Massive Problems after Upgrade

Support & Bugs
1

Hi,

I upgraded our (low traffic!) piwik installation from 1.11 (!) to the current version 2.11.1. For the upgrade I moved my old piwik dir to piwik.old and created a fresh piwik directory, into which I copied the config.ini.php file from the old installation.
Then I started the database upgrade from the command line. This took some hours and stopped with a segmentation fault :frowning:
So I started the database upgrade again, it took some more hours, but this time it finished with a success message :slight_smile:
Now piwik is running again, but it is extremely slow (every page takes some minutes). So I tried to disable real-time reports and called core:archive manually / per cron. This takes some more hours, but for some sites it produces errors like this:

INFO CoreConsole[2015-02-27 14:03:15] Error: Got invalid response from API request: http:///piwik/index.php?module=API&method=API.get&idSite=4&period=week&date=last260&format=php&token_auth=d0c5bd34d9faedb915e0adf402c9e361&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of ‘memory_limit’ in your php.ini file. Please check your Web server Error Log file for more details.

I already increased the number of CPUs for the (virtual) server running piwik from 1 to 4 and as well the vRAM from 1GB to 4GB. I also increased the “memory_limit” to now 2048 MB, but the latter just means, it takes some more time until the error is produced.

I guess, our database is somehow broken. Can I check this for sanity? Perhaps I just need to remove some buggy entries, but which ones? :slight_smile:

Any ideas ?

Olaf.

P.S.: Our installation is really not big. I checked the database:
mysql> select idsite,count(idvisit) from p_log_visit group by idsite;
±-------±---------------+
| idsite | count(idvisit) |
±-------±---------------+
| 1 | 411233 |
| 2 | 17674 |
| 3 | 2119 |
| 4 | 5476 |
| 5 | 70 |
| 6 | 441 |
| 7 | 1367 |
| 8 | 6313 |
| 9 | 15429 |
| 10 | 1112 |
| 11 | 804 |
| 12 | 308 |
| 13 | 674 |
| 14 | 430 |
| 15 | 19602 |
| 16 | 688 |
| 17 | 5621 |
| 18 | 8469 |
| 19 | 36719 |
| 20 | 4710 |
| 21 | 473 |
| 23 | 4344 |
| 24 | 11876 |
| 25 | 2717 |
| 26 | 257 |
| 27 | 834 |
±-------±---------------+
26 rows in set (1 min 0.56 sec)

2

ARRRRGGHHH!!!
I realized, that the server was running for more than 3 years now without rotating the apache2 log! So this was more than 1.2 GB. After renaming it and restarting apache2, the server is much faster :slight_smile:
Now I restarted core:archive with --force-all-websites --force-all-periods and it went through in less than two hours and produced only two errors (for unimportant websites). Now I think I can clean up the database.

Olaf.

P.S.: I wonder why the old Piwik version had no problems with speed, if it turns out, that the issue was an apache2 misconfiguration


3

Hi there,

Also in case you are using PHP 5.3 please upgrade to php 5.5 which brings many performance improvements

4

Hi,

after fixing the apache2 log problem, I still got errors like this one (from the archive cron job):

ERROR CoreConsole[2015-03-02 13:41:12] Got invalid response from API request: http:///piwik/index.php?module=API&method=API.getDefaultMetricTranslations&format=original&serialize=1&trigger=archivephp. The response was empty. This usually means a server error. This solution to this error is generally to increase the value of ‘memory_limit’ in your php.ini file. Please check your Web server Error Log file for more details.
ERROR CoreConsole[2015-03-02 13:41:12] The Piwik URL http:///piwik/index.php does not seem to be pointing to a Piwik server. Response was ‘’.
ERROR CoreConsole[2015-03-02 13:41:12] The Piwik URL http:///piwik/index.php does not seem to be pointing to a Piwik server. Response was ‘’.

But now I upgraded the PHP 5.3.17 to PHP 5.6.6. Since this update the piwik-archive runs in just 3 minutes for all periods and all 27 websites (took several hours before), and I don’t get any more errors like the above one.

Olaf.

P.S.: I think, you should mention, that Piwik is already incompatible with PHP 5.3 :-]