12

The data dump of some closed beta sites are available in xml format. Is it possible to see their content in a more readable form? Why are not these data available in HTML format too?

One of the problems, for example, is that if I want to see the comments of a post, I have to search for them in the comments.xml file looking up the post ID, which is not an easy thing to do:

<row Id="16" PostId="22" Text="comment..." 
    CreationDate="2012-01-11T21:33:50.133" 
    UserId="13" />
Improve this question
1
  • 8
    <comment><body>What, you think <abbr>XML</abbr> format is not readable?</body></comment>
    – Cody Gray
    Apr 6, 2012 at 17:55

1 Answer 1

Reset to default
7

tl;dr:

I have started an app for this that is really just a proof of concept at this point.

The data is not available in HTML format because it's a data dump. That means it's data, not rendered pages. HTML would only really be useful for viewing, and even then it would be of limited usefulness as mush of the site would be missing.

What data are you looking for specifically? Have you tried opening the XML files in a capable browser like Chrome?

Perhaps if enough people were interested someone could build an app which allows one to browse a data dump.

EDIT: Maybe I am wrong, please stand by...

OK. I looked at the data dump. I didn't realize that post body was inside an XML attribute. This does make the posts far less readable.

If the body was actually the content of the <post> key then these would be more usable. I wonder what could be done about that...

UPDATE: Here is some slightly less terrible, awful ruby code which does work somewhat for converting the XML to HTML. This needs a lot of work to be useful!

You can view / download the code from my website. (View source, my web server isn't serving the proper mime type). To use, run it in a terminal and pass the path of a data dump folder. The script will output HTML to the console:

[jnet@Kyle Scratch]$ ./soDumpMe.rb ~/Downloads/012012\ Sexuality/012012\ Sexuality  > "012012 Sexuality.html"

The HTML that is produced groups Questions with their Answers and shows scores. Nothing more... yet.

Improve this answer
10
  • The data dumps do provide rendered HTML in posts. They only included the original Markdown recently.
    – NullUserException อ_อ
    Apr 6, 2012 at 18:17
  • Oh I didn't realize that. What's the issue then..? >_>
    – The Unhandled Exception
    Apr 6, 2012 at 18:18
  • The posts' (questions and answers) contents (body) are in HTML, but the files per se are all in XML. I am not sure if the Markdown in comments is translated though. I am also not if I understand what the OP is asking for.
    – NullUserException อ_อ
    Apr 6, 2012 at 18:24
  • @TheUnhandledException Yup! Seems like we have to build an app to extract and link all data together from separate XML files and show them in a user-friendly manner.
    – B Faley
    Apr 6, 2012 at 18:30
  • @Meysam what OS do you use?
    – The Unhandled Exception
    Apr 6, 2012 at 18:41
  • @TheUnhandledException Windows
    – B Faley
    Apr 6, 2012 at 18:56
  • @Meysam Hm. I am not a windows developer. Do you have the ability to run Ruby scripts? I just hacked together a quick ruby script which converts the posts.xml to HTML.
    – The Unhandled Exception
    Apr 6, 2012 at 19:08
  • @TheUnhandledException That would be nice! Please provide a link to it in your answer. I will try to use it :)
    – B Faley
    Apr 6, 2012 at 19:19
  • Will do @Meysam. Right now it's a total mess, it has questions and answers mixed in with no apparent usefulness. But if I can rework it into something... useful... I will certainly post a link.
    – The Unhandled Exception
    Apr 6, 2012 at 19:41
  • @TheUnhandledException Thank you. I appreciate it!
    – B Faley
    Apr 6, 2012 at 19:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .