Hi, I have a website where there are a few dedicated users and the rest mostly checking it out.
So when I look into Visits by days since last visit I see this: Screenshot by Lightshot
It seemed a bit strange that I have so relatively few visitors with values “1 day” and “2 days” so I did a bit of digging in the db.
It was not hard to find some data, that looks a bit suspicious. Here Screenshot by Lightshot I have selected all the visits from a particular visitor, but although there is a day when visitor has not visited the site, visitor_days_since_last always keeps value of 0 (check out lines 7375 and 10387).
disable hearbeat functionality (if you have it enabled) to make it easier to debug for you
set trust_visitors_cookies =1 in config.ini.php to make sure the db lookup is done by idvisitor only (easier to debug)
[optional] enable logging for the tracker, if you’d like to see more information in logs, than just the db entry update
visit your website (to create the initial cookie if you don’t have it yet)
use cookie editing browser plugin and find out your visitor id. Visitor id cookie’s name starts with _pk_id. For a cookie like this 396907bea57e1ef0.1458551132.5.1458638931.1458627971. visitor id is 396907bea57e1ef0
find your entry in the log_visit table with sql like SELECT * FROM log_visit where hex(idvisitor) = '396907bea57e1ef0'
now in order for server side part of the tracker to register a new visit, we either need to delete this row, or adjust the time of visit_last_action_time to make it earlier than 30 mins ago (default session length). I usually just delete the row.
use your cookie editor and edit the cookie so that last two timestamp values would be more than 12 hours ago, so that it would count as being in the previous day
delete the session cookie (it’s name starts with _pk_ses). This cookie expires within 30 mins and in order to reproduce a case when user revisits your website after a day or so, we need this cookie to be deleted.
refresh your website. Now theoretically tracker should create a new entry in log_visit table and visitor_days_since_last should be > 0.
repeat step 6. to find the latest entry in log_visit for yourself.
confirm that value of visitor_days_since_last is 0 although it should be >= 1.
The bug happens because of the code on the client side.
When we calculate the values we’re going to send to the tracker (in getRequest) we have a code like this:
if (!cookieSessionValue) {
// cookie 'ses' was not found: we consider this the start of a 'session'
// here we make sure that if 'ses' cookie is deleted few times within the visit
// and so this code path is triggered many times for one visit,
// we only increase visitCount once per Visit window (default 30min)
var visitDuration = configSessionCookieTimeout / 1000;
if (!cookieVisitorIdValues.lastVisitTs
|| (nowTs - cookieVisitorIdValues.lastVisitTs) > visitDuration) {
cookieVisitorIdValues.visitCount++;
cookieVisitorIdValues.lastVisitTs = cookieVisitorIdValues.currentVisitTs;
}
The idea is - if session cookie has been expired, then we must start a new session. So the timestamp of last visit now should be the value which in previous session we had as current visit timestamp.
A bit further down we set the updated lastVisitTs to the server.
Now the problem is at the point when we get to getRequest we have already overwritten currentVisitTs value in our cookie. Here is where it happens:
function setVisitorIdCookie(visitorIdCookieValues) {
if(!configTrackerSiteId) {
// when called before Site ID was set
return;
}
var now = new Date(),
nowTs = Math.round(now.getTime() / 1000);
if(!isDefined(visitorIdCookieValues)) {
visitorIdCookieValues = getValuesFromVisitorIdCookie();
}
var cookieValue = visitorIdCookieValues.uuid + '.' +
visitorIdCookieValues.createTs + '.' +
visitorIdCookieValues.visitCount + '.' +
nowTs + '.' +
visitorIdCookieValues.lastVisitTs + '.' +
visitorIdCookieValues.lastEcommerceOrderTs;
setCookie(getCookieName('id'), cookieValue, getRemainingVisitorCookieTimeout(), configCookiePath, configCookieDomain);
}
As you can see, we just pass nowTs for the value of current visit timestamp. The problem is - we call setVisitorIdCookie at the very beginning of Tracker initialization (for me it usually doesn’t complete there though because site id is not set yet, but we also call it from setSiteId and it completes then).
So this results in the situation when we overwrite currentVisitTs with a timestamp from new Date() before we call getRequest. So if the session has been expired, we will set lastVisitTs to the value of timestamp from new Date() and send it to the server. On the server we will calculate passed days since last visit timestamp and will see that the value is 0.
Thanks for your investigation. Would you please create a Pull request on piwik/piwik with some copy-paste of your findings above, so we can consider your fix for the next Piwik release?