We've rolled out audit tasks for suggested edits in our more recent builds. What we're doing here is actually kind of fun. They're also sometimes hilarious.
Since we know suggested edits have really noisy history, the approach we use for other queues (selecting "known good" or "known bad" content to then fake numbers on) won't really work. Instead, we're actually creating new, bad, suggested edits*.
We're building a super simple model (basically a Porter Stemmer + Markov Chains**) of a few thousand posts per-site, and using that to create "looks OK at a glance, but deeply flawed" audit edits. Thus "involve boy code machines".
We'll probably keep tweaking the algorithm, but based on a day or so of data it looks like it's convincing enough to catch really egregious reviewers.
*Not in a technical sense, these audits never get into the DB as suggested edits; but in a display sense.
**We're not doing anything fun with Markov Chains, just a random walk through the model to generate text.