I have found a bug in the quality filter or Markdown engine on stackoverflow.com. The following examples are correctly rendered by the Markdown engine, but are rejected by the quality filter on SO. They are rejected with the message:
"Your post appears to contain code that is not properly formatted as code"
As identified by @hammar in an answer below (Thanks!), the first two examples contain undocumented Markdown link syntax. Nevertheless, those examples are rendered correctly, and so if the bug is not in the quality filter, than it's in the Markdown engine.
The second two examples are variations on the first two that use only documented Markdown syntax, but are still rejected.
All the examples are accepted by MSO, as discussed here, because MSO does not use the same quality filter as SO.
Example 1: Here is a simplified (and hence artificial) submission that triggers the quality filter:
Please don't down vote: I need 10 meta reputation to use the meta.stackoverflow sandbox.
First sentence. More words;
More words. [Haskell link with a few
words].
More words. [Haskell link with a few words].
[Haskell link with a few words]: http://example.com
Here it seems the semicolon at the end of the line, the [...]
reference split over two lines, and the second [...] reference are
all needed to trigger the ill-formatted code warning. The first line
of the example is related to a second question.
Example 2: Here is a real submission that triggers the quality filter:
It seems most answers assume that contracts are checked *dynamically*. Note that in some systems contracts are checked *statically*. In these cases you can think of contracts as a restricted form of dependent types which can be checked automatically (versus interactively).
See the "Specification Checking" section on [Dana Xu's page] for papers on static and hybrid (dynamic for any checks that could not be discharged statically) checking of contracts for Haskell and OCaml. The contract system of Xu includes refinement types and dependent arrows, both of which are dependent types. Early languages with restricted dependent types that are automatically-checked include the [DML] and [ATS] of Pfenning and Xi. In DML, unlike in Xu's work, the dependent types are restricted so that automatic checking is *complete* (decidable).
[Dana Xu's page]: http://gallium.inria.fr/~naxu/pub.html
[DML]: http://www.cs.bu.edu/~hwxi/DML/DML.html
[ATS]: http://www.ats-lang.org/
Example 3: Here is a version of Example 1 that uses only documented Markdown, but is still rejected:
Read this before down voting:
https://meta.stackexchange.com/questions/130740/how-should-i-develop-a-minimal-example-of-an-so-bug-for-an-mso-bug-report
I will delete this question soon. It is a fake question being used to
reproduce a bug that is specific to the SO quality filter and is not
reproducible on MSO. I have chosen an obscure fake title in hopes
that most people will not be interested in reading this question.
First sentence. More words;
More words. [Haskell link with a few
words][Haskell link with a few words].
More words. [Haskell link with a few words][Haskell link with a few words].
[Haskell link with a few words]: http://example.com
Example 4: Here is a version of Example 2 that uses only documented Markdown, but is still rejected:
Read this before down voting:
https://meta.stackexchange.com/questions/130740/how-should-i-develop-a-minimal-example-of-an-so-bug-for-an-mso-bug-report
I will delete this question soon. It is a fake question being used to
reproduce a bug that is specific to the SO quality filter and is not
reproducible on MSO. I have chosen an obscure fake title in hopes
that most people will not be interested in reading this question.
It seems most answers assume that contracts are checked *dynamically*. Note that in some systems contracts are checked *statically*. In these cases you can think of contracts as a restricted form of dependent types which can be checked automatically (versus interactively).
See the "Specification Checking" section on [Dana Xu's page][Dana Xu's page] for papers on static and hybrid (dynamic for any checks that could not be discharged statically) checking of contracts for Haskell and OCaml. The contract system of Xu includes refinement types and dependent arrows, both of which are dependent types. Early languages with restricted dependent types that are automatically-checked include the [DML][DML] and [ATS][ATS] of Pfenning and Xi. In DML, unlike in Xu's work, the dependent types are restricted so that automatic checking is *complete* (decidable).
[Dana Xu's page]: http://gallium.inria.fr/~naxu/pub.html
[DML]: http://www.cs.bu.edu/~hwxi/DML/DML.html
[ATS]: http://www.ats-lang.org/
As further evidence that there is a problem, here are 6 MSO questions related to non-code being treated as code: 1 2 3 4 5 6
Suggestion: I understand the desire prevent submissions with ill-formatted code that will have to be cleaned up by someone else 7 8. And, naturally, there will be false positives. But it would be really helpful if the quality filter gave some hints about what it has identified as ill-formatted code. If this is difficult, say because the filter does a global analysis of the submission, assigning it a score like a spam filter might, then there should be some way to override the filter.
A solution might be to provide an option to submit the question to a quarantine, where it would be reviewed by a moderator before appearing on the site. Presumably there are already queues for flagged submissions and such. To be sure user was making an effort, the option could appear only after a few failed submission attempts. And hopefully, the fact that the quarantined submission would not appear right away on the site would discourage the user from using this feature if they had not already made a sincere effort to properly format any actual code in their submission.
On the other hand, maybe this is only affecting epsilon percent of users, and hence it's not worth implementing a fix. In this case, it would be nice to have a FAQ which lists the known ways to trigger false positives in the quality filter. The quality filter could provide a link to this FAQ when it rejected a submission. The minimal examples in the present question, and corresponding examples extracted from the 6 questions linked to above, could start such a FAQ.
