What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 133 Stack Exchange communities.

Is there a particular reason why vote times are omitted in the data dump?

IMO this could be a very useful data-set. Is this one of the steps that was done to anonymize the data, or does it simply not exist? This would be a very good addition to future dumps to help build time-based relationships, which are currently very rigid and full of assumptions.

share|improve this question
    
It must exist because without it they couldn't do rep recalcs. –  cletus Jun 30 '09 at 1:46
    
Well technically you could do a recount on the date alone, but it doesn't accurately describe the flow of events. –  Ian Elliott Jun 30 '09 at 2:15
    
Hi, I'm working with the Sept 2011 data dump. This includes a timestamp in the Votes table. Can you update on me on how this has been adjusted? Is there a systematic or random shift added? Thanks –  paulusm Apr 29 '12 at 20:38

2 Answers 2

up vote 11 down vote accepted

One of Jeff's requirements in releasing the data dump was that specific user voting data would not be available. The site goes to great lengths to keep voting data private, and I support that. Jeff strongly had in mind the AOL data dump debacle (google it if you're not familiar) in which AOL thought they had anonymised a search dataset but enterprising researchers were able to correlate data searched for with other information and actually identify real-world individuals. Like, down to their home address, just from what they typed into the AOL search box.

Stack Overflow obviously has less private information and less potentially invasive results if voting data were to be exposed, but if the online site keeps voting data private then the data dump should respect that privacy too.

If the millisecond-resolution vote timestamp were included in the dump, I believe the up/down voting patterns could strongly correlate with other activity on the site (questions, answers, comments). The more history available in the dump, the stronger the correlation can be. People use Stack Overflow during certain times of the day and not others, and the usage pattern will be distinct for each individual. There might be enough pattern information in there to identify who cast a given vote or votes.

I'm not completely certain that one could get useful information out of timestamp correlation in this way, but I think there's enough of a risk that I suggested truncating the timestamps. If somebody can present a convincing argument that there wouldn't be a way to discover user voting patterns, then the data dump can always be changed for future runs. It's certainly not set in stone.

share|improve this answer
2  
You are right! Never underestimate the power of statistic evaluation on long-term data. –  Ladybug Killer Jun 30 '09 at 8:22
1  
I'd agree with this in some cases, but due to the hundreds of thousands of users, and thousands of votes per hour there really is no way of correlating this information back to users. In the same statistical sense I could say that a nasty comment left will most likely correlate with the next down-vote, and a positive comment with an up vote. However anyone can assume this information already simply by looking at the comment itself. There's no real way for me to prove that every n comments y user posts, and that because of z's comment it is likely that this comment is also y's. –  Ian Elliott Jun 30 '09 at 13:16
1  
Without included post times you can build a fairly accurate picture of the times that votes were cast already, not quite milliseconds, but in the realm of minutes. By revealing this you're also not really giving us anything we can't already obtain. Any determined user can already track a user's behaviour through the site by following their recent activity and watching the reputation patterns against them. A users comments/posting times are more incriminating than vote times, because they not only implicate votes but expose behaviour. Also n,x,y,z example above should be votes not comments. –  Ian Elliott Jun 30 '09 at 13:26
    
I do agree with @Ian argument, providing a vote timestamp rounded to minute; –  Rubens Farias Jan 17 '10 at 11:11

This was done to anonymize the data

share|improve this answer
1  
I suspected this, but then I had a difficult time trying to figure out how knowledge of the vote time would give away who the voter was. –  Kyle Cronin Jun 30 '09 at 2:39
    
How would the time make the votes identifiable? I'm certain there's a good reason, I just can't think what it would be.. –  dbr Jun 30 '09 at 2:58
    
we need Greg Hewgill to comment on this, he suggested it, and we agreed.. –  Jeff Atwood Jun 30 '09 at 3:30
    
I would love that input, because out of everything I can foresee this data is very useful with no major downsides. No other information really alludes to way of correlating these with users. –  Ian Elliott Jun 30 '09 at 4:19
    
Would it be possible to "tamper" the timestamp with a little random jitter ? –  Steve Schnepp Jun 30 '09 at 9:44
    
The time is not necessary to hide since we already hide UserId to anonymize the data. Also see this comment by duplicate question's OP –  hims056 Oct 1 '13 at 11:22

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .