I thought it would be interesting to plot the 'experience level,' as determined by medals or number of posts, vs calendar time for questions tagged with "R" (posted to SO ). Assuming I teach myself some of the R-packages for general data mining, is there an accessible dbase within SO that could be used for such a purpose? (Not that I would turn down a fully functional solution to this problem :-) )
What is meta?
- Any Stack Exchange user can ask a question
- The community provides support, votes on ideas, and reports bugs
- Your voice helps shape the way Stack Exchange operates
migrated from stackoverflow.com Nov 23 '12 at 14:47
This question came from our site for professional and enthusiast programmers.
The best place to find the data would be in Predict Closed Questions on Stack Overflow. This is a preprocessed dataset from Stack Overflow to predict if a question will be closed or not.
The next resource is the Stack Overflow Creative Commons dump.
And to get an insight into some fun stuff you could do with the data, check out the paper Discovering Value from Community Activity on Focused Question Answering Sites: A Case Study of Stack Overﬂow (published 2012-08-12) from a bunch of heavy hitters.
Stack Overflow has been publishing "data dumps" for quite some time, as well as offering an API for interactive exploration. Note that the full data dump is really large. The download is IIRC around 5 GB of 7-Zip compressed XML data; in order to fully decompress it you thus need a lot of disk space.
Note however the rather obvious privacy implications. The publicly available data of Stack Overflow is to some extend anonymized, but probably not to the extend that makes your efforts moot though.
Nevertheless, do not undererstimate the difficulties you will encounter.