What is the purpose of this test?
The test (known as a review audit) is designed to help new reviewers hone their moderation skills, while nudging more experienced users who don't seem to be paying close attention to what they're reviewing. Some people were in the habit of positively reviewing everything without regard to quality; extremely low-quality posts with a high score are very problematic in a system where the best information naturally rises to the top, so "audit" tasks are mixed in to make sure folks are paying attention.
This is currently active only on Stack Overflow, Server Fault, Super User, Software Engineering, Ask Ubuntu, Mathematics, Stack Overflow em Português, Stack Overflow en Español, and Stack Overflow на русском.
How does the audit work?
First questions, First answers, Late answers, Triage, and Low quality posts/answers: The test presents a reviewer with a post that was previously deleted and known to be of extremely poor quality (flagged as spam, "not an answer", or "very low quality"), or known to be of high quality (many up-votes, no down-votes, etc).
Close votes and Reopen votes: The test presents a reviewer with a question which the system has determined should be open or closed based on various criteria (highly-voted with no close votes, quickly closed with no answers, etc).
Suggested edits: The test presents a reviewer with a randomly generated edit utilizing the Markov chain algorithm to randomly insert various words that may appear to be useful at first glance but actually make no sense and are meant to vandalize the post.
If a user passes a test, they are informed that they passed and thanked for their time and feedback. If a user fails a review audit, they are warned that they didn't pass and encouraged to pay more attention in the future.
The result is then recorded for analysis. Only diamond (♦) moderators or Stack Exchange employees can see the full data; it is not accessible publicly even to high-reputation users. Normal users can see if a given post was used to generate audits in the post's timeline; the entry will appear as "audit", and clicking on it will access the review task which will indicate the pass/fail outcome and the user to whom it was presented.
What happens if I fail?
For most, nothing. You'll be given some guidance on why the post was inappropriate and what you could have done instead, and then allowed to continue reviewing.
Repeated failures, especially in short succession, can lead to your review privileges being suspended for a few days or longer, depending on the frequency of the failures. Stack Exchange is continuing to analyze the data to determine the best action to take in these cases.
Why was the system expecting me to perform the wrong action on a post?
Review audits are chosen automatically. The system isn't perfect, meaning that every so often a post slips through normal community detection and moderation, causing the system to generate a bad, invalid, or questionable audit.
These can be split into two cases:
The system expected me to positively review a bad post
- The most common case is when a low-quality, off-topic, or otherwise close-worthy post gets upvoted. Sometimes visitors may upvote posts without paying attention to the site rules.
- You identify a post as being an exact duplicate of an older, highly-voted post. To prevent audits from being obvious, the vote count is displayed as zero and the user card is either anonymized or shown as having been posted by a new user. This may make you think you are looking at an exact duplicate.
The system expected me to negatively review a good post
- Other users may have cast subjective downvotes, which means they disagree with the post rather than think it is a bad post. You may have had a different opinion, or attempted to review it objectively.
- The post may have been marked as spam as part of a spamming attack where only a few users knew the details. In this case, moderators can clear the spam flags, to prevent them from being used as audits.1
Don't worry! The fact that you identified a bad audit means that you are paying attention, and are not the type of reviewer that the audits are intended to catch.
If you encounter a bad audit, post it on the per-site meta tagged supportdisputed-review-audits to draw attention to it, containing a link to either the audit review task or the post, and why you disagree with the audit. Other users can cast votes to counteract the automatic decisions that led to it being chosen as the wrong type of audit (e.g. downvote and close an off-topic question that got picked as a known-good audit). If you get suspended from reviewing as a result of failing one or more bad audits, the review suspension can be lifted or shortened by ♦ moderators at their discretion.
A review is obviously an audit. Should I report it?
There are many ways to identify audits (e.g. click to see the post itself, the user, the vote count, etc.). Almost all of these ways involve having to further investigate the post. By this time, you've already carefully reviewed the post, which is the intent of the queues.
As stated earlier, the main purpose of audits is to catch robo-reviewers, who mindlessly review without even looking at the posts, and to help educate new reviewers. Easy-to-identify audits help legitimate reviewers pass and help better hone new reviewers' skills.
On the other hand, if you spot a user who appears to be gaming the system (e.g. they are making bad reviews but somehow passing every audit) flag one of their posts for moderator attention and explain the situation; moderators will take a further look and maybe issue a manual review suspension.
Also, if you find something in the review audit system that makes audits extremely obvious (as in one can immediately tell that a review is an audit, with almost no extra investigation), feel free to report that flaw.
Do review audits count towards review badges?
Passed audits count as a successful review towards the review badges. A failed audit does not count toward the badges, but does not decrement your progress either; it simply adds none.
Do robo-reviewers dream of electric sheep?
Robo-reviewers don't sleep. They wait.
Important Note:
The information contained in this answer is subject to change as the review system is further refined. Changes that impact the mechanics of the test, or consequences of failing it repeatedly will be reflected in this answer as they're made, but not necessarily immediately after they're made.
1. In December 2018, a change was made where only posts that are deleted by the system in response to the threshold number of spam flags (currently four from normal users or a single one from a moderator) will be used as known-spam audits in the Triage and Low quality posts (answers) queues (as well as entirely excluding posts with one or more valid "rude or abusive" flags in those queues). However, you may still see this happen, e.g. if the flags come from Charcoal users, as well as in other review queues.
