Part of my post was replaced with random gibberish

Question

An answer that I posted on Meta.Programmers was rather broken, with a block quote replaced by apparently random letters and numbers. Aarthi fixed it with a non-breaking space, but I don't understand what went wrong.

When I opened the post for editing, the displayed text was what I intended to go there. It only happens when the post is viewed in the reading mode. The preview also displayed the text correctly.

Answer 1

The main issue is that Markdown parser can sometimes be a bit fragile when it comes to lists and block quotes, because the boundaries aren't as well-terminated as, say, code blocks.

Barring a fix, the user-side resolution of the problem is to make the parser's job as easy as possible, which in your case would have involved the Markdown:

 - > does something exist like this for full-text indexing/searching?

 - > Has anyone ever had a similar gripe before... or am I nuts and should just keep using SQL for these situations?

which renders as

• does something exist like this for full-text indexing/searching?

• Has anyone ever had a similar gripe before... or am I nuts and should just keep using SQL for these situations?

Note that the > is on the same line as the -, and that the newline between the two list items is necessary for everything to turn out right.

So, what goes wrong in your case? As I alluded to in the comments, you're actually being bitten by a blockquote-in-list bug which occurs with the second blockquote, which in turn is responsible for the bug you're actually seeing here.

While it'll take a bit more troubleshooting to see if there's a fix for the underyling issue, the trouble starts when the HTML block replacement for the second block quote inadvertently eats the closing </li></ul> from the list, leaving the parser with this (where ? is standing in for a non-printable character):

...\n\n<ul>\n<li>?H2035416125H</li>\n<li>\n\n\n?H977440878H\n\n...

Since there's no longer a closing tag for the list, the second call to HashHTMLBlocks that handles Markdown-generated blocks doesn't swap out the list as intended. This causes the input to FormParagraphs to be malformed, since even though it recursively subsitutes back in hashed blocks, it expects the top level block to be proceeded by two newlines.

Had the entire <ul> block been swapped out, this would have been fine, but because the other issue caused it not to be, the first hash marker ?H2035416125H fails the regex match and no replacement is performed.

Luckily, addressing the linked issue should cause this problem to simply go away. I've suggested some quick-fixes for that problem, so we'll see what balpha thinks.

Answer 2

Example

Source

 - 

> This text is going to be replaced.
 - 
> This text is going to remain.

HTML Output

<p><ul>
<li>H708767747H</li>
<li></p>

<blockquote>
  <p>This text is going to remain.</li>
  </ul></p>
</blockquote>

Rendered Output

• This text is going to be replaced.

• This text is going to remain.

Answer 3

This was fixed by Unexpected text at the bottom of posts. If you look at the revision view of Jeremy Banks' test answer or at the previously broken first revision of your own answer on meta.programmers (revision views are always live, so you see what the Markdown conversion does today), you'll see that it looks fine.

To be clear, the broken nesting of the HTML is still there – it looks like this now:

<ul>
<li><blockquote>
  <p>This text is going to be replaced.</p>
</blockquote></li>
<li>

<blockquote>
  <p>This text is going to remain.</li>
  </ul></p>
</blockquote>

– but the browser graciously works around that. That's the bigger general problem of blockquotes in lists* (and reported in various places), but at least there's no "H708767747H" anymore.

_{*That is also fixed now.}