17

I want to use the Stack Overflow database for a data classification project. How do I dump the Stack Overflow database?

0

3 Answers 3

23

You can download the data dump (produced monthly) at: https://blog.stackoverflow.com/category/cc-wiki-dump/

You will need a 7zip decompressor, XML parsing code, and probably a database to load the data into.

You may also be interested in https://data.stackexchange.com/ which has already done all this for you.

4
  • There is an RSS feed here: feed://www.clearbits.net/feeds/creator/146-stack-overflow-data-dump.rss
    – Ameer Deen
    Commented Jun 14, 2011 at 5:12
  • Stackql.net is no longer available. Do you know another site that provides this functionality? Commented Nov 15, 2012 at 16:26
  • 2
    @Marcel: SE now runs a regularly-updated interactive query thingamajig here: data.stackexchange.com
    – Shog9
    Commented Dec 30, 2012 at 3:38
  • @Shog9 thanks! the thingamajig is awesome! Is the (code.google.com/p/stack-exchange-data-explorer) also up-to-date? maybe we should link to that as well.
    – d-_-b
    Commented Dec 30, 2012 at 4:56
3

If you are using MySQL I have a schema and a script written in ruby that will load the data. Just grab the latest data dump, create a database, load this schema, modify this loader script to point to your database and then run the script. It only takes a few minutes to load all the data.

2
  • Your script and schema worked like a charm.
    – Snehal
    Commented Nov 5, 2012 at 2:27
  • Cannot access the database files anymore. Anyone know what's wrong? There are backups? Wanna map full site to search offline :)
    – m3nda
    Commented Aug 5, 2015 at 12:18
0

I just made a slow Python script to import the dump to a SQLite3 database.

It's designed to run on my RAMless netbook, so it doesn't make a single COMMIT for all INSERTs, but rather COMMITs every INSERT. I'll make a fast RAM sucker version tomorrow.

You must log in to answer this question.