A recent question on Meta.StackOverflow makes me question the practice of scrubbing comments from audits.*
The user in question was presented with a poorly formatted but decent answer. There was only a hint something was wrong ("Solved the problem for me." at the bottom), but that hint would only be obvious to people with lots of experience handling the many variations of "Thanks!" answers.
If the comments had not been scrubbed from the answer, the situation would have been clearer. The comments were of the standard automated variety, pointing out that "mee too" and "thanks" answers aren't allowed.
Clearly, the comments add context to the post under review. Stripping off this context removes information often necessary to make the correct judgement when reviewing.
Stripping context to make the post fit into the review space is necessary, but I think it goes a little too far to remove the comments. Why strip the comments as well as removing the post from the original question?
I believe the purpose of this is to try and make the audit a better, more valid test of the audited's decisions. In practice, isn't it just turning the audit into a trap, one that potentially snares everyone indiscriminately?
Yeah, leaving the comments may take more room or make the correct choice obvious, which can result in less people being tricked by the audit... but audits shouldn't be trying to "trick" anybody. Review audits should be a speedbump to roboreviewers or for people who honestly don't know how SE works.
Speedbumps don't trick you, popping up and ripping out your transmission when you launch over them--they're painted in DOT yellow (or whatever colors your backwater prefers), right out in the open, waiting to catch people who honestly aren't paying attention, are speeding while taking selfies, or posting funnay imaeg to questions on meta.
Should we not be removing comments from review audit posts?
* Am not sure if audits scrub comments from all post and queue types, so there might be an argument for a given combination of the two where they should still be scrubbed... Also, a similar question can be found here, but is limited to answer audits and with less feature-request
