1

I have found a bug in the quality filter or Markdown engine on stackoverflow.com. The following examples are correctly rendered by the Markdown engine, but are rejected by the quality filter on SO. They are rejected with the message:

"Your post appears to contain code that is not properly formatted as code"

As identified by @hammar in an answer below (Thanks!), the first two examples contain undocumented Markdown link syntax. Nevertheless, those examples are rendered correctly, and so if the bug is not in the quality filter, than it's in the Markdown engine.

The second two examples are variations on the first two that use only documented Markdown syntax, but are still rejected.

All the examples are accepted by MSO, as discussed here, because MSO does not use the same quality filter as SO.

Example 1: Here is a simplified (and hence artificial) submission that triggers the quality filter:

Please don't down vote: I need 10 meta reputation to use the meta.stackoverflow sandbox.

First sentence. More words;
More words. [Haskell link with a few
words].
More words. [Haskell link with a few words].

[Haskell link with a few words]: http://example.com

Here it seems the semicolon at the end of the line, the [...] reference split over two lines, and the second [...] reference are all needed to trigger the ill-formatted code warning. The first line of the example is related to a second question.

Example 2: Here is a real submission that triggers the quality filter:

It seems most answers assume that contracts are checked *dynamically*.  Note that in some systems contracts are checked *statically*.  In these cases you can think of contracts as a restricted form of dependent types which can be checked automatically (versus interactively).

See the "Specification Checking" section on [Dana Xu's page] for papers on static and hybrid (dynamic for any checks that could not be discharged statically) checking of contracts for Haskell and OCaml.  The contract system of Xu includes refinement types and dependent arrows, both of which are dependent types.   Early languages with restricted dependent types that are automatically-checked include the [DML] and [ATS] of Pfenning and Xi.  In DML, unlike in Xu's work, the dependent types are restricted so that automatic checking is *complete* (decidable).

[Dana Xu's page]: http://gallium.inria.fr/~naxu/pub.html
[DML]: http://www.cs.bu.edu/~hwxi/DML/DML.html
[ATS]: http://www.ats-lang.org/

Example 3: Here is a version of Example 1 that uses only documented Markdown, but is still rejected:

Read this before down voting:
https://meta.stackexchange.com/questions/130740/how-should-i-develop-a-minimal-example-of-an-so-bug-for-an-mso-bug-report

I will delete this question soon.  It is a fake question being used to
reproduce a bug that is specific to the SO quality filter and is not
reproducible on MSO.  I have chosen an obscure fake title in hopes
that most people will not be interested in reading this question.

First sentence. More words;
More words. [Haskell link with a few
words][Haskell link with a few words].
More words. [Haskell link with a few words][Haskell link with a few words].

[Haskell link with a few words]: http://example.com

Example 4: Here is a version of Example 2 that uses only documented Markdown, but is still rejected:

Read this before down voting:
https://meta.stackexchange.com/questions/130740/how-should-i-develop-a-minimal-example-of-an-so-bug-for-an-mso-bug-report

I will delete this question soon.  It is a fake question being used to
reproduce a bug that is specific to the SO quality filter and is not
reproducible on MSO.  I have chosen an obscure fake title in hopes
that most people will not be interested in reading this question.

It seems most answers assume that contracts are checked *dynamically*.  Note that in some systems contracts are checked *statically*.  In these cases you can think of contracts as a restricted form of dependent types which can be checked automatically (versus interactively).

See the "Specification Checking" section on [Dana Xu's page][Dana Xu's page] for papers on static and hybrid (dynamic for any checks that could not be discharged statically) checking of contracts for Haskell and OCaml.  The contract system of Xu includes refinement types and dependent arrows, both of which are dependent types.   Early languages with restricted dependent types that are automatically-checked include the [DML][DML] and [ATS][ATS] of Pfenning and Xi.  In DML, unlike in Xu's work, the dependent types are restricted so that automatic checking is *complete* (decidable).

[Dana Xu's page]: http://gallium.inria.fr/~naxu/pub.html
[DML]: http://www.cs.bu.edu/~hwxi/DML/DML.html
[ATS]: http://www.ats-lang.org/

As further evidence that there is a problem, here are 6 MSO questions related to non-code being treated as code: 1 2 3 4 5 6

Suggestion: I understand the desire prevent submissions with ill-formatted code that will have to be cleaned up by someone else 7 8. And, naturally, there will be false positives. But it would be really helpful if the quality filter gave some hints about what it has identified as ill-formatted code. If this is difficult, say because the filter does a global analysis of the submission, assigning it a score like a spam filter might, then there should be some way to override the filter.

A solution might be to provide an option to submit the question to a quarantine, where it would be reviewed by a moderator before appearing on the site. Presumably there are already queues for flagged submissions and such. To be sure user was making an effort, the option could appear only after a few failed submission attempts. And hopefully, the fact that the quarantined submission would not appear right away on the site would discourage the user from using this feature if they had not already made a sincere effort to properly format any actual code in their submission.

On the other hand, maybe this is only affecting epsilon percent of users, and hence it's not worth implementing a fix. In this case, it would be nice to have a FAQ which lists the known ways to trigger false positives in the quality filter. The quality filter could provide a link to this FAQ when it rejected a submission. The minimal examples in the present question, and corresponding examples extracted from the 6 questions linked to above, could start such a FAQ.

|improve this question
  • 1
    artificially short line length is going to be a problem. Don't do that. If you want paragraphs, type paragraphs. – Jeff Atwood May 1 '12 at 8:47
  • 1
    SIX years later, I was greeted by the same bug, because the filter decided that the code inside a Javascript/HTML/CSS code snippet inserted via the snippet editor and then edited in the message editor "was not formatted as code". Just saying, I actually considering dropping the answer and send my contribution to the trash can. I was able to save it only out of sheer luck, but I was very close to say "know what? I don't have time to waste trying to contribute here" - and obviously that isn't good for the site. – SPArchaeologist Sep 20 '18 at 9:58
5

I think it might be the type of links you're using. Most posts either use the inline [link text](url) or [link text][1] formats. I've never seen the [link text] format used on SO before.

Anyway, I fixed your answer by changing it to the numeric reference style.

|improve this answer
  • Thanks! Looking at the SO Markdown documentation, I see that the link notation I'm using is not mentioned! I guess I didn't read the documentation carefully enough the first time. The system accepts and correctly renders my link notation, so I didn't realize it wasn't standard. – ntc2 Apr 30 '12 at 2:34
  • 4
    Another markdown "bug" solved by reading the documentation! – Cody Gray Apr 30 '12 at 10:57
  • 3
    @TheEstablishment: I maintain this is a bug: the examples are accepted and rendered correctly by the Markdown engine (in the preview on SO, and everywhere on MSO). The SO quality filter rejects them on suspicion of containing code. So, either there is a bug in the quality filter, or there is a bug in the Markdown implementation. As evidenced by the many MSO questions related to the quality filter rejecting submissions in this way, there is a problem. Thanks again to hammar for identifying my use of undocumented Markdown. – ntc2 May 1 '12 at 0:31
  • That seems like reasonable behavior for the quality filter. What part do you want fixed? Do you want markdown to break when you use unsupported syntax? – Cody Gray May 1 '12 at 2:33
  • @TheEstablishment: fixing either part would be sufficient. Yes, if the syntax is not supported, then I think it should be rejected as such. Supporting it in the implementation, but not in the documentation, and then giving an unrelated error message, does not seem like a good design. But I doubt it's a design as such; I think it's a bug. – ntc2 May 1 '12 at 2:49
  • Note that I've added versions of the examples that use only documented Markup, but are still rejected. Maybe I should remove the examples that use the undocumented link syntax? – ntc2 May 1 '12 at 2:50
  • I can't think of any examples of other invalid markdown that is rejected. Why should we introduce special behavior just for this? – Cody Gray May 1 '12 at 2:52
  • 1
    @TheEstablishment: the syntax is supported by the implementation: writing [blah] when you have [blah]: http://example.com somewhere results in a link to example.com in the Markdown output. So, I would call it undocumented, not "invalid". In any case though, what I meant in my previous response was: if the syntax is forbidden, then the error message should indicate this. But I don't think anyone intended to forbid this syntax, I think it was an accident, and my examples of documented syntax leading to the same erroneous error message corroborates this. – ntc2 May 1 '12 at 3:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .