What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

Stack Overflow is quickly reaching a point where any question that you might want to ask, may already have been asked. Wouldn't it be nice to be able to take this resource with you, even in an offline environment?

I've created a static mirror of SO which shows what I've done so far:

My vision for this is to (in collaboration with the community!) package this up into an app of some kind for iPhone/Android/whatever, with local search (Lucene or similar?) and obviously compression. Things that might be useful:

  • limit questions to only those over a certain vote threshold
  • slice and dice by tag, perhaps include only certain tags or exclude others

I'm not a mobile application developer, so I can't do this all by myself.

Currently the code that I've got to generate this is a hodgepodge of Python, Java, and XSLT. I started to do the whole thing with XSLT but the sheer volume of data brought my poor little server to its knees, so I had to break things up a bit. The order of processing goes something like this:

  1. Python script to preprocess posts.xml and group the answers by question into one xml file per question
  2. Java driver (I'm using xt) to run the XSLT question renderer for each question xml file (helps my server not run out of memory)
  3. Various other XSLT templates to render users, the index page, etc

Who is interested in helping to build this? I'll write up some basic instructions and put the code up on Github for everybody to look and poke at (but probably tomorrow). Code is now on Github.

share|improve this question
6  
Let's get Greg a beefier server! –  John the Seagull Sep 8 '09 at 10:06
    
I suppose I could have tried running it on an EC2 instance.. –  Greg Hewgill Sep 8 '09 at 10:15
1  
might I suggest the names "stackpack", "stacked lunch" or "stacker" as an homage to Stac Electronics. Something surely can be made from the OF and CF flags :) –  ShuggyCoUk Sep 8 '09 at 12:12
3  
StackOverflowOnTheGo –  TheTXI Sep 8 '09 at 13:55
    
I can't compete with tempo! –  ShuggyCoUk Sep 8 '09 at 23:02
    
I don't have permission to look at your server :-( –  Chris Huang-Leaver Sep 9 '09 at 10:30
    
@Chris: sorry, my mistake (I moved things around), fixed now. –  Greg Hewgill Sep 9 '09 at 19:04
    
site down?????? –  Midhat Mar 30 '10 at 8:53
    
@Greg is this project dead? –  systempuntoout Oct 5 '10 at 9:48
1  
@systempuntoout: No, it's just sleeping. But the SO API might be more fruitful at the moment. –  Greg Hewgill Oct 5 '10 at 19:13

4 Answers 4

Excellent!

What would be awesome is to create a portable edition generator, so each user could generate a view of the data he's interested in and download it as a prepackaged app for either mobile phones or for offline consumption on a laptop (I'm usually down at datacenters without internet connectivity, but with a laptop).

share|improve this answer
3  
Yes, exactly! That's what I alluded to with the comments about "include only certain tags or exclude others" without actually saying so. The associated application would be able to open any data set created by the generator toolchain. –  Greg Hewgill Sep 8 '09 at 10:07

If you are wanting to target this idea towards the mobile computing community...why would you not want something more up to date? Such as a live version that just happened to be optimized for mobile browsing (or a native app)? Note: After re-reading the question I can see you mention offline browsing.

With that said, here are some potential ideas:

  • Be able to "sync" up your application so that when you go out you have the most up to date information stored.

  • Only include questions and answers above a certain score (accepted answers are exception to rule)

  • Only include tags with X number of questions passing the previously asserted include criteria.

  • Comments On/Off

share|improve this answer

This sounds much like the eGranary project in minature. I wonder if there is any possibility of collaboration. I would imagine that SO information would be very useful to computer programming courses in undeveloped countries.

share|improve this answer

Looks like nothing has changed in the last four months on Github - are you aware of any further progress on this outside of Github?

share|improve this answer
    
Not on that codebase specifically, but I've heard mention of other similar projects. I started to convert my code to pure C# (to avoid the current polyglot mess) but naturally got distracted with other projects. :) –  Greg Hewgill Feb 4 '10 at 19:45
    
@Greg - Thanks for the update! –  Adam Davis Feb 4 '10 at 20:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .