I review edits for some time now ( I got the steward badge ). I often wondered if I was too severely judging the edits and rejecting too many, or if on the other hand I let pass mediocre edits.
The test questions helped me alot to get right feeling. But I also find that there's a big potential for improvement.
Most of the test edits that needed to be rejected were examples of vandalism with some nonsense words randomly inserted, while in the real edits I don't remember one such edit.
On the other hand there are missing examples of common invalid edits like introducing wrong formatting or errors in the code, arbitrary tag changes, etc, that should also be rejected.
I don't know how these fake edits are selected or generated, but in my opinion there is a vast field for improvement.
Follow up and sanctions
I saw discussions about reviewers that mechanically approve all questions. I think that the test edits are a good means to educate those reviewers, especially if they were better sampled. Also it would be good for this purpose if a repeated incorrect review of the test questions would result in a sanction, like : only be allowed to review 10 edits per day instead of 20, until some test questions are answered correctly.
Such a sanction would have more effect for educating superficial reviewers than the "Stop and listen" message only.
What do you think about these improvements ?