Meta Stack Exchange is a question and answer site for meta-discussion of the Stack Exchange family of Q&A websites. It only takes a minute to sign up.
Sign up to join this community
I propose to make review audits not counting to the count of daily reviews.
Why?
Because reviewing review audits doesn't give anything for SE community (review audits are only checking is reviewer paying attention), but they're counted to the count of daily reviews - so if somebody reviewed two review audits, will be able to review only 18 real posts!
Conclusion
Review audits - yes!, but not counting to the count of daily reviews.
Review audits are useful and should count (credit) towards the 'total number done', but shouldn't count (decrement) against the 'total number available'.
As soon as the audit is revealed you should be +1 for the effort, and -0 for the remaining count.
That way the benefit is correctly credited, without an unwanted penalty.
They do give something to the SE community; they help ban "bad" reviewers from reviewing. Good reviews are of concern to the community and proving that you can perform them "well" is of high value. If you decide that the review audits are not "good" then that is a separate matter.
I'm ambivalent whether review audits should be counted but I believe your argument to not count them is flawed. If your argument was a good one my personal opinion is that it doesn't matter enough to make the effort to change the code.
I'm almost on the verge of advocating to keep the status quo as is, but I do see your point.
A few facts for context:
The Reviewer and Steward badges (for reviewing 250 and 1,000 tasks in a single queue, respectively) count towards moderator candidate scores in elections, meaning that the SE team considers a user who has one or both of these badges as more likely to be a better moderator than one who doesn't. In my opinion, steps should be taken to make sure that users can't cheat their way into getting these badges.
It's possible for really savvy users to figure out which tasks are audits and which aren't. This does require paying some attention, and the whole point of audits is to ensure that reviewers are paying attention, but it's possible for users to game the system. This gets even easier if the user uses one of many user scripts that others have written to detect review audits.
I disagree with the solution proposed in the question, because reviewing an audit usually takes as much time as reviewing a real task, and I feel that legitimate users should be rewarded for their effort.
The solution proposed in Rob's answer seems like a good idea, but it's easy to game, especially on sites with a large amount of review tasks in queues (such as many of the queues on Stack Overflow). If Rob's answer were to be implemented, one could do the following: review 19/39 real tasks, change their strategy to review only audits for a while (while skipping all the real reviews), then when no more audits show up after some time, just make another real review to finish off 20/40.
This means that that solution would make it easy for users to game badges that help differentiate moderator candidates in elections, and I believe that that's a very bad thing.
To that end, I'm putting forth an idea that would help solve all problems: audits should allow users to continue reviewing, legitimate users should be rewarded for passed audits, and users should not be able to game the system to earn those badges. This is a mixture of the solutions proposed in the question and in Rob's answer:
The "maximum number" above should be high enough so that legitimate reviewers keep getting rewarded for their effort, but low enough so that users who try to game the system get stopped. I'd say, around 3-5.
An audit still chews up time and attention, shaving irreplaceable seconds from your life, leaving you that much closer to death.
The least we can do is increment a number in exchange for that precious gift.
