The Stack Exchange Data Explorer (SEDE) is now updated on a weekly basis. It would be nice if all of that public data was made accessible off-line on a regular and consistent basis. The SEDE says that the release occurs every three months, but the last release was in August 2012 (almost four months ago), the release before that was in April 2012 (four months), and prior to that was December 2011 (five months).

Perhaps the process could be automated and made part of the SEDE updates. I think that weekly is probably too much for the data dump generation, but an automated script that runs regularly so we have consistent access to the public data would be nice. I'd really like to see this monthly, but every two or three months would be fine. The ClearBits page should also be updated to reflect the actual release schedule.

  • Huh...I always thought they were the same data set – Ben Brocka Dec 8 '12 at 14:54
  • It's my understanding (although I'm going off of unverified memory) that this is something being considered, but that at the moment it's fairly low priority. – Tim Stone Dec 8 '12 at 15:49
  • 1
    @BenBrocka That used to be the case a while time ago, but for performance reasons Data Explorer's load process doesn't rely on the data dump export anymore. – Tim Stone Dec 8 '12 at 15:49
  • +1 Because I didn't realize the data explorer was now weekly <3 <3 <3 !! – Michael Berkowski Jan 5 '13 at 13:50
  • 6
    It's on the todo...we've had problems uploading to ClearBits and we've had issues with the open-data hosting on S3...getting in touch to a reliable contact there is trying at best. – Nick Craver Jan 5 '13 at 14:49
  • @Nick Craver: So is it purely a hosting problem? What sort have problems have you had with ClearBits? – Bkkbrad Feb 7 '13 at 3:07
  • @Bkkbrad - the upload is very, very bad - we've had it drop numerous times and the client to do so is...let's say not great. Oded and I are working on automating the export this week (we're validating the xml now, then just need to zip and send it somewhere...so almost done). We'll do a post here in a bit asking for hosting suggestions. – Nick Craver Mar 6 '13 at 11:10
  • clearbits.net (i.e. where the Stack Exchange data dumps are stored) seems to be down since few days. Any idea about alternative ways of getting access to the dumps? Thanks – guido Dec 28 '13 at 9:44
  • See Is there a problem with the ClearBits data dump? for more on that. – Pops Jan 21 '14 at 20:00

You must log in to answer this question.

Browse other questions tagged .