I want to use the Stack Overflow database for a data classification project. How do I dump the Stack Overflow database?
3 Answers
You can download the data dump (produced monthly) at: https://blog.stackoverflow.com/category/cc-wiki-dump/
You will need a 7zip decompressor, XML parsing code, and probably a database to load the data into.
You may also be interested in https://data.stackexchange.com/ which has already done all this for you.
-
There is an RSS feed here: feed://www.clearbits.net/feeds/creator/146-stack-overflow-data-dump.rss Jun 14, 2011 at 5:12
-
Stackql.net is no longer available. Do you know another site that provides this functionality? Nov 15, 2012 at 16:26
-
2@Marcel: SE now runs a regularly-updated interactive query thingamajig here: data.stackexchange.com– Shog9Dec 30, 2012 at 3:38
-
@Shog9 thanks! the
thingamajig
is awesome! Is the (code.google.com/p/stack-exchange-data-explorer) also up-to-date? maybe we should link to that as well.– d-_-bDec 30, 2012 at 4:56
If you are using MySQL I have a schema and a script written in ruby that will load the data. Just grab the latest data dump, create a database, load this schema, modify this loader script to point to your database and then run the script. It only takes a few minutes to load all the data.
-
-
Cannot access the database files anymore. Anyone know what's wrong? There are backups? Wanna map full site to search offline :)– m3ndaAug 5, 2015 at 12:18
I just made a slow Python script to import the dump to a SQLite3 database.
It's designed to run on my RAMless netbook, so it doesn't make a single COMMIT
for all INSERT
s, but rather COMMIT
s every INSERT
. I'll make a fast RAM sucker version tomorrow.