Use review test questions to better train reviewers [duplicate]

Question

After reading What could be done to stop code formatting misuse? and BoltClock's hilarious Reject an already-approved suggested edit when rolling it back, there seems to be a consensus that bad reviewers can be at least as much problem as bad editors.

The new (I think they're new) test questions inserted into the review queue to make sure that reviewers are doing their job appropriately seem like a great idea, but currently the test questions are easy to spot because the edits they show are nonsensical defacements that real editors are unlikely to create. Perhaps that's the point — they're just there to see if you're paying any attention at all. But I think test questions have the potential to be even more useful.

Subtle, real-world test questions could help inexperienced but well-intentioned reviewers learn to distinguish between good and bad edits.

For example, there are lots of proposed edits where the only change is to apply code formatting to a few inline class names. Such edits might seem reasonable to some reviewers who don't know any better, but they increase the post's edit count without improving the post, and they should be rejected. On the other hand, it's often helpful to apply code formatting to language keywords like for and while so that it's clear that those words are being used as keywords rather than as their English namesakes. Real-world examples of each case, perhaps taken from real edits reviewed by moderators, could be inserted into the queue. If a reviewer's decision differs from the "correct" answer, an explanation of why the question should be rejected or accepted would be shown, like:

This edit should be rejected as too minor because the only change is to apply code formatting in a way that doesn't clarify the question or make it easier to read.

Or:

This edit should be accepted because the use of code formatting helps to distinguish programming keywords from their English counterparts, thus reducing possible ambiguity.

Code formatting is far from the only example of situations where reviewers need to make judgement calls. Using test questions as a training tool to help reviewers learn how to choose appropriately could lead to a real improvement in the quality of reviews. Improving reviews will ultimately improve edits as users will get better feedback about their edits from reviewers.

Answer 1

I was thinking about this the other day, as I'm using the tools more the edits are increasingly more easier to spot.

Although I agree, we need to be careful that these audits don't fall into the grey area. People's opinions differ, the only things that are agreed upon are when things are completely defaced (which is what the current audits are like).

Maybe the mods should take a pool of question edits, see which edits form a consensus vote in which all agree on, then propose those questions for audits.

Another type that needs further reviewing audits is Close Votes. At the moment, you can blindly close anything for any reason without ever failing an audit. There should be an audit there to prevent this, with it being a question that definitely shouldn't be closed.