14

WARNING!!!

Users not particularly experienced in database-administration should NOT attempt the following.

Has anybody thought about hosting the data-dump on their own public-facing server, and allowing queries to be ran against it by visitors? Likely smart to put a cap on the query-execution-time to keep somebody from trying to smash your server. Would be smart to have it in its own database, with its own user-account too. Limiting the account to read-only queries is necessary as well.

Any thoughts on this?

Update: I'm currently working on getting all of this data into a MySQL database. Going to play with the idea locally, and maybe attempt to host the toy.

Improve this question
1
  • Do it. Seems like a great idea.
    – Assaf Lavie
    Commented Jul 8, 2009 at 4:56

6 Answers 6

Reset to default
2

Recently I've been working on a fairly ambitious project to do something similar to this. I've been building a PHP/JS framework to calculate and graph statistics from the dump, which also includes a template system to analyze data that can't simply be calculated with a simple query.

A public interface to this seems like a good addition to this project, and I'll definitely invest some time in the idea.

I'm still a few weeks away from launching anything significant, but I'll update when I'm done. I've already registered what I feel is a rather fitting domain, statoverflow.com.

Improve this answer
1
  • 2
    SLAPS FOREHEAD STATOVERFLOW. DOH. Why didn't I think of that?!
    – Paolo Bergantino
    Commented Jul 8, 2009 at 2:43
7

This exists now: data.stackexchange.com.

See the "question" Introducing the Stack Exchange Data Explorer aka SEDE for some more information.

Improve this answer
2
  • That exists now, but didn't back in 2009. :)
    – Adam Lear StaffMod
    Commented Apr 19, 2014 at 16:32
  • @AnnaLear: sure, but it seemed a shame to leave this perfectly good question without a link to it :-)
    – SamB
    Commented Apr 20, 2014 at 2:18
2

I would give it 14 seconds before someone dropped all your tables.

You might want to prevent someone from doing that.

Edit Based upon the reassurance that you wouldn't leave every hole wide open, I would actually be interested in something like this because it would allow me to glean information from the data dumps without having to go through all the trouble myself in setting them up (laziness on my part).

It would also be interesting if you could hook in a reporting engine of some sort that would allow for more slick reports and charts on the fly.

Improve this answer
6
  • SHHHHHHHHHHHHHHHHHHH.
    – GEOCHET
    Commented Jul 8, 2009 at 0:21
  • Is it possible to open a database connection as read-only?
    – Kyle Cronin
    Commented Jul 8, 2009 at 0:22
  • 1
    @Kyle I could limit the account to select only.
    – Sampson
    Commented Jul 8, 2009 at 0:23
  • 2
    It may be a good idea if you edit your post to include some of the safe guards. When you first read it it sort of sounds like something someone might want to do without first thinking about things like malicious users.
    – TheTXI
    Commented Jul 8, 2009 at 0:34
  • 1
    @TheTXI: Good suggestion. My question has been updated.
    – Sampson
    Commented Jul 8, 2009 at 0:46
  • 2
    OOH! My first query is gonna be to see how little Richie B tables is doing.
    – gnostradamus
    Commented Jul 8, 2009 at 1:10
2

I've got a publicly accessible SQL Server with the dumps from ServerFault, StackOverflow, SuperUser, and Meta. Connection info is here:

http://www.brentozar.com/archive/2010/02/querying-the-stackoverflow-data-dump/

Improve this answer
1

You should definitely do that.

Improve this answer
1
  • Rich B? Suspended? Again?
    – deleted
    Commented Dec 24, 2009 at 2:20
0

Another 'sandbox' with Stackoverflow Dump data is:

http://www.rdbhost.com/rdbadmin/main.html?r0000000767

It uses the Postgresql engine, and thus the Postgresql version of SQL. It has indexes, and also offers a DB API module, for accessing the database from Python.

It was just updated with November's data.

Improve this answer
1
  • I call it "November's" data because the archive has a November file timestamp. The data itself is for the month of October. Am I misrepresenting?
    – Rdbhost
    Commented Nov 10, 2009 at 1:37

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .