10

The publishing date on archive.org/details/stackexchange is 16th March 2015

Stack Exchange Data Dump by Stack Exchange, Inc.

Published March 16, 2015

Usage Attribution-Share Alike 3.0

However, the files appear to be newer than that, the time stamps from archive.org/download/stackexchange/ indicate August, except the Comments and Badges files which are April.

stackoverflow.com-Badges.7z                      09-Apr-2015 19:28     90.9M
stackoverflow.com-Comments.7z                    09-Apr-2015 19:42     2.0G
stackoverflow.com-PostHistory.7z                 18-Aug-2015 15:49     12.0G
stackoverflow.com-PostLinks.7z                   18-Aug-2015 15:50     35.6M
stackoverflow.com-Posts.7z                       18-Aug-2015 16:52     7.3G
stackoverflow.com-Tags.7z                        18-Aug-2015 16:52     573.1K
stackoverflow.com-Users.7z                       18-Aug-2015 16:54     153.0M
stackoverflow.com-Votes.7z                       18-Aug-2015 16:58     500.1M

I've imported all the data with the stack dump offline browser, which reported a large number of inconsistently errors, I'm guessing from the mixing an old comment file with newer post file.

Is this a mistake? Where can I find a consistent set of files? Unfortunately I cannot access bit-torrent from where I work.

  • Strange. It's now labelled as being from August, but when I downloaded the torrent file it was the same as the one released in March... – Jeremy Banks Oct 3 '15 at 4:28
  • I filed a bug in the internal archive.org JIRA for this issue. – Greg Lindahl Jan 16 '16 at 23:34
  • The problem is that the Internet Archive doesn't make .torrent files for any item bigger than 25 gigabytes. And, unfortunately, we don't delete any older .torrent when this happens, so that's how it ended up with an obsolete .torrent. – Greg Lindahl Feb 21 '16 at 20:44
6

A good way to estimate the age of the data (true, not the same as upload date, maybe some format changed, but likely what most people want), is to download Votes.xml ("only" 400M compressed) and:

tail Votes.xml

This shows to me that the latest dump that I've just downloaded was from 2014-09.

Then I've done:

grep 2014-10 Votes.xml

to confirm, and got no results as expected.

So this seems to be the one from this announcement: Stack Exchange September 2014 Data Dump is available

I've found a few questions mentioning that the data is outdated or that the date is not clearly documented:

3

In the data obtain via bittorrent from https://archive.org/download/stackexchange/stackexchange_archive.torrent the latest post from stackoverflow that I saw was from 2014-09-14 03:07:29 UTC.

However, the zip archive https://archive.org/compress/stackexchange/formats=7Z&file=/stackexchange.zip had posts from 2015/08.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .