What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

The data dump of some closed beta sites are available in xml format. Is it possible to see their content in a more readable form? Why are not these data available in HTML format too?

One of the problems, for example, is that if I want to see the comments of a post, I have to search for them in the comments.xml file looking up the post ID, which is not an easy thing to do:

<row Id="16" PostId="22" Text="comment..." 
    CreationDate="2012-01-11T21:33:50.133" 
    UserId="13" />
share|improve this question
6  
<comment><body>What, you think <abbr>XML</abbr> format is not readable?</body></comment> –  Cody Gray Apr 6 '12 at 17:55
add comment

1 Answer 1

up vote 6 down vote accepted

tl;dr:

I have started an app for this that is really just a proof of concept at this point.


The data is not available in HTML format because it's a data dump. That means it's data, not rendered pages. HTML would only really be useful for viewing, and even then it would be of limited usefulness as mush of the site would be missing.

What data are you looking for specifically? Have you tried opening the XML files in a capable browser like Chrome?

Perhaps if enough people were interested someone could build an app which allows one to browse a data dump.

EDIT: Maybe I am wrong, please stand by...

OK. I looked at the data dump. I didn't realize that post body was inside an XML attribute. This does make the posts far less readable.

If the body was actually the content of the <post> key then these would be more usable. I wonder what could be done about that...

UPDATE: Here is some slightly less terrible, awful ruby code which does work somewhat for converting the XML to HTML. This needs a lot of work to be useful!

You can view / download the code from my website. (View source, my web server isn't serving the proper mime type). To use, run it in a terminal and pass the path of a data dump folder. The script will output HTML to the console:

[jnet@Kyle Scratch]$ ./soDumpMe.rb ~/Downloads/012012\ Sexuality/012012\ Sexuality  > "012012 Sexuality.html"

The HTML that is produced groups Questions with their Answers and shows scores. Nothing more... yet.

share|improve this answer
    
The data dumps do provide rendered HTML in posts. They only included the original Markdown recently. –  NullUserException อ_อ Apr 6 '12 at 18:17
    
Oh I didn't realize that. What's the issue then..? >_> –  The Unhandled Exception Apr 6 '12 at 18:18
    
The posts' (questions and answers) contents (body) are in HTML, but the files per se are all in XML. I am not sure if the Markdown in comments is translated though. I am also not if I understand what the OP is asking for. –  NullUserException อ_อ Apr 6 '12 at 18:24
    
@TheUnhandledException Yup! Seems like we have to build an app to extract and link all data together from separate XML files and show them in a user-friendly manner. –  Meysam Apr 6 '12 at 18:30
    
@Meysam what OS do you use? –  The Unhandled Exception Apr 6 '12 at 18:41
    
@TheUnhandledException Windows –  Meysam Apr 6 '12 at 18:56
    
@Meysam Hm. I am not a windows developer. Do you have the ability to run Ruby scripts? I just hacked together a quick ruby script which converts the posts.xml to HTML. –  The Unhandled Exception Apr 6 '12 at 19:08
    
@TheUnhandledException That would be nice! Please provide a link to it in your answer. I will try to use it :) –  Meysam Apr 6 '12 at 19:19
    
Will do @Meysam. Right now it's a total mess, it has questions and answers mixed in with no apparent usefulness. But if I can rework it into something... useful... I will certainly post a link. –  The Unhandled Exception Apr 6 '12 at 19:41
    
@TheUnhandledException Thank you. I appreciate it! –  Meysam Apr 6 '12 at 19:46
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .