9

I've just got my hands on a data dump, and I'm disappointed to see that the Body field of the posts is in HTML rather than Markdown. I know there's markdown in the original database, because that's what I see if I try to edit an answer.

I don't want to make life difficult for Jeff Atwood and his team by requesting this as a feature, so I'm asking: How do I recover Markdown from a Stack Overflow data dump?


(Thanks to Eric and to Jon Skeet, who pointed me at the data dump.)

3 Answers 3

8

This is now available.

The records in PostHistory contain the raw markdown, while the records in Posts contain the cooked HTML from the most recent revision.

4

Can Markdown be recovered from the SO data dump?

Yes, but as with most translations the markdown may not match the original. However, the HTML output would be the same, so there's no real impact.

From SO podcast #80 we learned:

Stack Overflow user William Shields has taken up Joel’s challenge to write a Markdown parser the right way — and produced an excellent series of articles about what he’s learned in the process: one, two, three, four. It’s a perfect example of the type of learning that Stack Overflow itself is all about; kudos to William for sharing it!

William shares in those articles that one of the uses of his parser will be converting HTML into markdown. You might contact him to see where he's at, and what other resources he is aware of.

However, conversion of markdown to HTML wouldn't be a bad topic to discuss on SO as well, so you might consider using that resource.

0

It should be trivial to convert the two back and fourth by a simple replacement, <b> => ** and <i> => *, etc.

Some PHP programs will do this for you, for instance Markdownify.

10
  • 12
    trivial's not exactly the right word, there's a lot more to it than that...
    – Kip
    Aug 19, 2009 at 19:45
  • 2
    Just use a regex...its easy :P
    – jjnguy
    Aug 19, 2009 at 19:46
  • 2
    Or try jquery...
    – jjnguy
    Aug 19, 2009 at 19:47
  • I would consider it trivial. Each element in markdown has a equivalent in HTML, which is what makes it such a great medium for interchange. The conversion may take awhile but the process itself isn't that difficult. Aug 19, 2009 at 19:48
  • 6
    it might be trivial to make it work most of the time. but there are edge cases where markdownToHtml(htmlToMarkdown(str)) != str. For one example, <b>un</b>broken would not be properly converted to **un**broken, because (IIRC) the double-star in the middle of an unbroken word doesn't end the boldness.
    – Kip
    Aug 19, 2009 at 19:53
  • 1
    There are some examples where I have to carefully craft the Markdown, and HTML, for it to come out like I want it to. Aug 19, 2009 at 19:56
  • 5
    Technically, the html in the posts is markdown already...so the job is easy!
    – jjnguy
    Aug 19, 2009 at 19:58
  • <code>&#96;&#96;</code> Aug 19, 2009 at 19:58
  • Since several of you seem to disagree I'd be willing to convert it, but someone else will have to distribute it. Bandwidth is a very valuable commodity here. Aug 19, 2009 at 20:03
  • @Ian - bitbucket.org, codeplex.com, http://sourceforge.net amongst a few. If you don't want to setup an account, I can host in one of my repos. Apr 5, 2010 at 22:56

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .