After reading What could be done to stop code formatting misuse? and BoltClock's hilarious Reject an already-approved suggested edit when rolling it back, there seems to be a consensus that bad reviewers can be at least as much problem as bad editors.
The new (I think they're new) test questions inserted into the review queue to make sure that reviewers are doing their job appropriately seem like a great idea, but currently the test questions are easy to spot because the edits they show are nonsensical defacements that real editors are unlikely to create. Perhaps that's the point — they're just there to see if you're paying any attention at all. But I think test questions have the potential to be even more useful.
Subtle, real-world test questions could help inexperienced but well-intentioned reviewers learn to distinguish between good and bad edits.
For example, there are lots of proposed edits where the only change is to apply code formatting to a few inline class names. Such edits might seem reasonable to some reviewers who don't know any better, but they increase the post's edit count without improving the post, and they should be rejected. On the other hand, it's often helpful to apply code formatting to language keywords like
while so that it's clear that those words are being used as keywords rather than as their English namesakes. Real-world examples of each case, perhaps taken from real edits reviewed by moderators, could be inserted into the queue. If a reviewer's decision differs from the "correct" answer, an explanation of why the question should be rejected or accepted would be shown, like:
This edit should be rejected as too minor because the only change is to apply code formatting in a way that doesn't clarify the question or make it easier to read.
This edit should be accepted because the use of code formatting helps to distinguish programming keywords from their English counterparts, thus reducing possible ambiguity.
Code formatting is far from the only example of situations where reviewers need to make judgement calls. Using test questions as a training tool to help reviewers learn how to choose appropriately could lead to a real improvement in the quality of reviews. Improving reviews will ultimately improve edits as users will get better feedback about their edits from reviewers.
code blockswhere they don't belong.