What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

WARNING!!!

Users not particularly experienced in database-administration should NOT attempt the following.


Has anybody thought about hosting the data-dump on their own public-facing server, and allowing queries to be ran against it by visitors? Likely smart to put a cap on the query-execution-time to keep somebody from trying to smash your server. Would be smart to have it in its own database, with its own user-account too. Limiting the account to read-only queries is necessary as well.

Any thoughts on this?


Update: I'm currently working on getting all of this data into a MySQL database. Going to play with the idea locally, and maybe attempt to host the toy.

share|improve this question
    
Do it. Seems like a great idea. –  Assaf Lavie Jul 8 '09 at 4:56
add comment

6 Answers

up vote 1 down vote accepted

Recently I've been working on a fairly ambitious project to do something similar to this. I've been building a PHP/JS framework to calculate and graph statistics from the dump, which also includes a template system to analyze data that can't simply be calculated with a simple query.

A public interface to this seems like a good addition to this project, and I'll definitely invest some time in the idea.

I'm still a few weeks away from launching anything significant, but I'll update when I'm done. I've already registered what I feel is a rather fitting domain, statoverflow.com.

share|improve this answer
2  
SLAPS FOREHEAD STATOVERFLOW. DOH. Why didn't I think of that?! –  Paolo Bergantino Jul 8 '09 at 2:43
add comment

This exists now: data.stackexchange.com.

See the "question" Stack Exchange Data Explorer for some more information.

share|improve this answer
    
That exists now, but didn't back in 2009. :) –  Anna Lear Apr 19 at 16:32
    
@AnnaLear: sure, but it seemed a shame to leave this perfectly good question without a link to it :-) –  SamB Apr 20 at 2:18
add comment

I would give it 14 seconds before someone dropped all your tables.

You might want to prevent someone from doing that.

Edit Based upon the reassurance that you wouldn't leave every hole wide open, I would actually be interested in something like this because it would allow me to glean information from the data dumps without having to go through all the trouble myself in setting them up (laziness on my part).

It would also be interesting if you could hook in a reporting engine of some sort that would allow for more slick reports and charts on the fly.

share|improve this answer
    
SHHHHHHHHHHHHHHHHHHH. –  GEOCHET Jul 8 '09 at 0:21
    
Is it possible to open a database connection as read-only? –  Kyle Cronin Jul 8 '09 at 0:22
1  
@Kyle I could limit the account to select only. –  Jonathan Sampson Jul 8 '09 at 0:23
2  
It may be a good idea if you edit your post to include some of the safe guards. When you first read it it sort of sounds like something someone might want to do without first thinking about things like malicious users. –  TheTXI Jul 8 '09 at 0:34
1  
@TheTXI: Good suggestion. My question has been updated. –  Jonathan Sampson Jul 8 '09 at 0:46
2  
OOH! My first query is gonna be to see how little Richie B tables is doing. –  gnostradamus Jul 8 '09 at 1:10
add comment

I've got a publicly accessible SQL Server with the dumps from ServerFault, StackOverflow, SuperUser, and Meta. Connection info is here:

http://www.brentozar.com/archive/2010/02/querying-the-stackoverflow-data-dump/

share|improve this answer
add comment

You should definitely do that.

share|improve this answer
    
Rich B? Suspended? Again? –  deleted Dec 24 '09 at 2:20
add comment

Another 'sandbox' with Stackoverflow Dump data is:

http://www.rdbhost.com/rdbadmin/main.html?r0000000767

It uses the Postgresql engine, and thus the Postgresql version of SQL. It has indexes, and also offers a DB API module, for accessing the database from Python.

It was just updated with November's data.

share|improve this answer
    
I call it "November's" data because the archive has a November file timestamp. The data itself is for the month of October. Am I misrepresenting? –  user138716 Nov 10 '09 at 1:37
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .