12

I propose to make review audits not counting to the count of daily reviews.

Why?

Because reviewing review audits doesn't give anything for SE community (review audits are only checking is reviewer paying attention), but they're counted to the count of daily reviews - so if somebody reviewed two review audits, will be able to review only 18 real posts!

Conclusion

Review audits - yes!, but not counting to the count of daily reviews.

  • Do you also propose to see the count not added to your total review count too then? So audits count for nothing other than a trap and if failed adds a score towards a potential ban? – James Aug 15 '18 at 21:32
8

They do give something to the SE community; they help ban "bad" reviewers from reviewing. Good reviews are of concern to the community and proving that you can perform them "well" is of high value. If you decide that the review audits are not "good" then that is a separate matter.

I'm ambivalent whether review audits should be counted but I believe your argument to not count them is flawed. If your argument was a good one my personal opinion is that it doesn't matter enough to make the effort to change the code.

  • 2
    But presumably the limits are in place to avoid abuse, or at least, reduce the damage that can be done by any one reviewer. I imagine Ty221 was actually trying to convey that your action in an audit can't possibly harm the community and so it makes sense that they needn't count towards that limit. I agree with your last point though. – OGHaza Mar 5 '14 at 10:18
  • 2
    If we imagine that the limits represent some kind of an average human review capacity, then audits should count. – BartoszKP Dec 1 '14 at 14:46
8
+25

Review audits are useful and should count (credit) towards the 'total number done', but shouldn't count (decrement) against the 'total number available'.

As soon as the audit is revealed you should be +1 for the effort, and -0 for the remaining count.

That way the benefit is correctly credited, without an unwanted penalty.

  • I just want to be clear: This is not currently how this works, and is a further clarification of the feature request, right? – FrankerZ Aug 5 '18 at 2:03
  • It's not a clairification (question), it's an answer (solution) unique to what has already been offered. – Rob Aug 5 '18 at 2:19
  • 1
    I just wanted to make sure that this isn't something that isn't implemented already. – FrankerZ Aug 5 '18 at 2:20
  • 3
    @FrankerZ No, this isn't implemented. Otherwise, you'll see a lot of people reviewing slightly more than 20 reviews per day. – Sonic the Reinstate Monica-hog Aug 5 '18 at 3:00
3

I'm almost on the verge of advocating to keep the status quo as is, but I do see your point.

A few facts for context:

  • The Reviewer and Steward badges (for reviewing 250 and 1,000 tasks in a single queue, respectively) count towards moderator candidate scores in elections, meaning that the SE team considers a user who has one or both of these badges as more likely to be a better moderator than one who doesn't. In my opinion, steps should be taken to make sure that users can't cheat their way into getting these badges.

  • It's possible for really savvy users to figure out which tasks are audits and which aren't. This does require paying some attention, and the whole point of audits is to ensure that reviewers are paying attention, but it's possible for users to game the system. This gets even easier if the user uses one of many user scripts that others have written to detect review audits.

I disagree with the solution proposed in the question, because reviewing an audit usually takes as much time as reviewing a real task, and I feel that legitimate users should be rewarded for their effort.

The solution proposed in Rob's answer seems like a good idea, but it's easy to game, especially on sites with a large amount of review tasks in queues (such as many of the queues on Stack Overflow). If Rob's answer were to be implemented, one could do the following: review 19/39 real tasks, change their strategy to review only audits for a while (while skipping all the real reviews), then when no more audits show up after some time, just make another real review to finish off 20/40.

This means that that solution would make it easy for users to game badges that help differentiate moderator candidates in elections, and I believe that that's a very bad thing.

To that end, I'm putting forth an idea that would help solve all problems: audits should allow users to continue reviewing, legitimate users should be rewarded for passed audits, and users should not be able to game the system to earn those badges. This is a mixture of the solutions proposed in the question and in Rob's answer:

  • Audits should not count towards the daily review limit.
  • Passed review audits should count as a review done, up to a daily maximum number.
  • After a certain number of passed audits per day, users should continue to be given audits (to check for attention), but passed audits shouldn't count towards the total number of reviews done.

The "maximum number" above should be high enough so that legitimate reviewers keep getting rewarded for their effort, but low enough so that users who try to game the system get stopped. I'd say, around 3-5.

-1
+100

An audit still chews up time and attention, shaving irreplaceable seconds from your life, leaving you that much closer to death.

The least we can do is increment a number in exchange for that precious gift.

  • 2
    That's not at all the gist of what the user's asking for. They want audits to not count towards the daily limit of 20/40, so that they can review a total of 20/40 real reviews a day. – Sonic the Reinstate Monica-hog Aug 15 '18 at 22:55
  • Nothing here needs that level of complexity. – Shog9 Aug 15 '18 at 23:33
  • 2
    I would add that excluding audits would probably make some reviewers complain about this, especially new ones (those who we are trying to involve, because we need them). Not all new reviewers hunt for badges but still, making badges harder to obtain because of audits (especially passed ones) would probably feel unfair to many of them. Oh and of course filed audit means one would benefit from making less reviews - in this case decrease of remaining number looks totally justified to me, no matter if reviewer is new or experienced – gnat Aug 16 '18 at 5:34
  • I'm a little sad, because I don't believe it involves all that much complexity. Tired of having to identify/skip audits because they just waste my time reviewing real issues on the site. – FrankerZ Aug 19 '18 at 5:35
  • Audits still waste time whether or not we count them, @frankerZ. That's why we reduce the number of audits shown to people who routinely pass them - the only way to not waste time on audits is to not show them at all. – Shog9 Aug 19 '18 at 15:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .