Documentation request

I’ve already met some dev on the irc channel.

Due to the very large usebase on the website that I mangage (a web adserving company) i found the piwik.php too slow…
I’ve profiled witch xdebug and callgrind but I’ve not found room for drammatic improvements (the output is here stats.wellnessadv.com/xdebug/cachegrind.out.19602)

Of course I’m not willing to rent other server…

I preferer to try to rewrite (at first) a smaller and whit lesser function C++ based tracker listener, the job now done by piwik.php visit.php and tracker.php.

I’m already lloking at it as a black box (the query it does… like http://stats.wellnessadv.com/swap/q2.txt)

And at a first glance seem quite a “reasonable” task.
I’m looking to found the documentation of the “inner” architecture and “logical” steps starting from piwik.php, the cookie format and some other stuff…

Essentially the script has only to move the get parameter inside the mysql table and smaller administrative stuff…

IF a dev is interested please contact me on irc (RoyBellingan) or send a pm trought the forum.

A good entry point would be this…

I don’t think the problem is PHP… the bottleneck is Mysql. You really want to improve the existing code instead of rewriting it wont help much and will be complicated as there’s a lot of logic.

I can provide all the benchmark
Those are just a few …
http://pastebin.com/KBM5JicX

The query was
http://stats.wellnessadv.com/swap/query.txt
Just a “big” select an Update and an Insert the other are very simple…

You can clearly see that mysql load was around 30%, disk usage level was low too…
And those benchmark was done with the xdebug extension on, removing xdebug increased the request to 315.
If you want I can post the metrics, but they are very similar.

I think most of the people having mysql bottleneck should change some parameter on the innodb to “lower” the crash safety and query transaction isolation.

http://dev.mysql.com/doc/refman/5.5/en/innodb-parameters.html#sysvar_innodb_flush_log_at_trx_commit
innodb_flush_log_at_trx_commit = 2 #data is flushed to disk every second

http://dev.mysql.com/doc/refman/5.5/en/set-transaction.html
transaction-isolation=READ-COMMITTED #

Actually I don’t have an SSD to make test, which could be interesting…

As a side note I’m willing to “buffer” the request and made a single commit of … X (in the range of thousand) query all at once, actually I’m still studing the inner PIWIK logic, but for returning visitor I think is feasible.

I’n the afternoon if I can i’ll post a benchmark of the exact same query done by a “early” C++ server just to see how much MySQL can handle…