What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

The last couple of days I've been trying to understand how the Stack Overflow page views counter works. I wish to add a views counter to a project of mine and investigating the options. Stack Overflow seems to be a good source of useful ideas.

Needless to say I've browsed through all of the related questions here on Stack Overflow and on Meta Stack Overflow. I can't say I've found an explanation but some tips (presumably false leading) were given in this answer: View counter in ASP.NET MVC

From what I can tell there is now a JavaScript section in each answer page (used to be CSS in 2008):

<script type="text/javascript" src="/posts/3590653/ivc/a947"></script>
<noscript>
    <div><img src="/posts/3590653/ivc/a947" class="dno" alt=""></div>
</noscript>

The last part of the URL - a947 - seems to be dynamically generated. It is changed every 15-20 minutes and is the same for all questions and presumably users (I tried anonymous access, changing my IP address, it stays the same during the interval).

In a comment to the answer Did anyone notice that some sites seem to be scraping/republishing SO's posts? Jeff Atwood gave a bit of a hint:

our view counts are very very strict -- more akin to visits as they are unique per IP per 15 minute interval.

Observing things with Firebug reveals that this URL always returns "204 No Content". So my first idea was that the browser is likely not to retrieve that counter URL on subsequent page requests until the random code changes, thus preventing duplicate counter hits already on the client side for that 15 minutes interval.

Watching the things with Firebug doesn't seem to confirm that theory:

enter image description here

As you can see the counter URL is retrieved again and again (even if I don't refresh the page with F5 but just click the page link in the title).

More to it. I tried to recreate the same scheme in a test project also returning 204, and I confirm that the counter URL is retrieved again and again. No browser caching.

More peculiar, Firefox for some reason calls the URL twice (the number is always even):

enter image description here

At this point the idea behind this counter URL is a mystery to me. What is the point of that last changing part of the URL if it's the same for all questions and users and does not prevent repeated requests from the client-side?

In that question a possible explanation is given:

View counter in ASP.NET MVC

I think I know the answer - they are analyzing the IIS log as Ope suggested.

Hidden image src is set to

http://stackoverflow.com/posts/3590653/ivc/[Random code]

[Random code] is needed because many people may share the same IP address (in a network, for example) and the code is used to distinguish users.

I'm not sure I understand this and I somehow feel it's off course.

Can somebody or perhaps the Stack Exchange Inc. folks explain what and how they are doing with the views counter?

I'm not trying to understand how it works in order to start gaming the system, I only wish to learn how these things are done. I hope Stack Exchange Inc. won't mind revealing some secrets.

UPDATE: I'm beginning to suspect this changing value at the end of the url is used to help aggregate the collected data by dividing time in 15-minute-slots. Still thinking in what way exactly...

UPDATE 2: So I've tried my test project with different browsers to see if 204 No Content would result in browser caching. It doesn't. The counter URL is retrieved again when the page is requested next time, but the value at the end of the counter URL is not changed. I tried with Internet Explorer 8 and the current versions of Firefox 3.6, Opera, Safari and Chrome. The behavior is identical and not surprising actually since the response for 204 doesn't contain the "Expires" header (neither does a live response from stackoverflow.com) so there is no reason for the browser to cache the response.

I kindly ask again the nice folks what is then the reason of that changing value of the counter URL? It doesn't force temporary browser caching so the reason must be elsewhere.

share|improve this question

migrated from stackoverflow.com Apr 12 '11 at 13:01

This question came from our site for professional and enthusiast programmers.

1  
FWIW, I like this as a main not meta post. –  NickC Apr 12 '11 at 21:12
    
"As you can see the counter url is retrieved again and again (even if I don't refresh the page with F5 but just click the page link in the title)." this is a bug in Firefox AFAIK, try in Chrome –  Jeff Atwood Apr 13 '11 at 9:28
    
@Jeff Atwood: Very interesting, I will check out Chrome. But still, could you please tell me what is the purpose of that changing value? To prevent repeated browser retrieval or something else? I almost can't sleep, it got me. :) –  user136634 Apr 13 '11 at 10:26
    
Care for a small award for your work? Then see meta.stackoverflow.com/q/36728 Sorry for the code; I figured that truly posting the URL here would also link it in that other question ;-) –  Arjan Apr 30 '11 at 20:17
    
@Arjan: Thank you for the hint. Not for the reward's sake, but I feel like I should help there with some info. –  user136634 May 6 '11 at 18:53
    
A small note on not tracking individual page views in Nick's answer for Add a “recently viewed” tab in the user account page. –  Arjan Dec 17 '11 at 10:24

1 Answer 1

All right, I have a theory... let's see what you think about it.

This changing value is meant as a time interval identifier (let's call it tid for brevity) in the following scheme:

There is a buffer in memory which accumulates counter hit data then periodically writes it to the database where it updates the [Views] column of each question involved. Since Jeff calls the mechanism in place a "buffered writing scheme", the theory fits the description.

The buffer has some fixed size. It's flushed when either it's filled up or when a predetermined time interval elapses. That's where the tid comes into play - it helps to distinguish the records pertaining to the previous interval from those belonging to the time interval still in progress. Only the hits from the previous interval are written to the database and erased from the buffer. The records from the active interval are kept until it elapses. This effectively achieves the model of a view as one hit per time interval.

The tid is generated somewhat like this:

14:00 > ca6f
14:15 > de42
14:30 > a590
14:45 > 3ab0
15:00 > 7ff1
...

Therefore it works in 15-minute-tact most of the time.

However at the times of traffic surge the buffer is going to be filled quicker. Suppose at peak hours it is filled up each 3 minutes and has to be flushed. When it happens the next tid is generated immediately with its 15-minute lifetime. The round times from the example above would obviously shift. Perhaps this is why the ids generated for Stack Overflow pages do not tact quite evenly. If the traffic surge ceases then the system works again in 15-minute intervals.

That however necessitates that the definition of a view depends on how much traffic the site is currently receiving. Makes sense actually. The more visitors there is, the more likelihood a question has changed significantly in a shorter period of time therefore quicker deprecating what the user has already seen. So one view in 3 minutes at peak hours and one view per 15 minutes during off hours increment the counter.

Do I make any sense? Your opinions?

share|improve this answer
2  
Architecturally this is extremely over engineered. Why do you need an in memory buffer on the web server? That is going to be extremely complicated in a server farm to keep in sync not to mention all the overhead of maintaining. The correct design is to have a database table to store unique visits with a stored procedure that executes every 15 minutes which counts the entries, updates the totals, and finally deletes all rows from said table. KISS. –  TugboatCaptain Jul 26 '13 at 21:15

You must log in to answer this question.