What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

There is a serious problem with the review system, which is that all the review queues offer a simple gaming option: keep clicking on the same no-effort decision, and you get an easy set of badges. This topic has come up several times on MSO: The current review system encourages fake reviews; some people upvote everything rather than actually fixing problems Can we do something against serial "Do Not Close" users? What can we do to stop bad edits getting accepted? and more.

I have seen suggested edit reviewers who never reject an edit, so I know the problem exists. I don't know how widespread that behavior is. Stack Exchange staff have the data to anaylze, I don't, so I won't speculate.

I conjecture that part of the cause for the serial reviewers is the existence of the badges. I fear that removing the badges would remove part of the incentive for the good reviewers as well, so that wouldn't be good. I think a small change to the badges can make them a better incentive. Currently, if you upvote 1000 first questions, approve 1000 suggested edits, etc., you get a badge.

I propose to make the Reviewer and Stewart badges require a minimum proportion of each review decision, in addition to the number of reviews. For example:

  • Close, reopen: at least 20% close/DNR, 20% edit or DNC/reopen, 10% edit
  • Late answers, first posts: at least 10% upvote, 10% downvote or flag, 10% edit, 10% comment or upvote on a comment
  • Low quality: at least 20% edit, 10% comment — and I wish I could put voting and flagging on that list
  • Suggested edits: at least 20% approve, 20% reject, 10% edit

The numbers are just a first suggestion. They may not be the right ones at all — please make better-informed proposals if you can. I don't know if they'd need to be adjusted for each site (I hope not, that would make things really complex).

This won't stop someone from choosing decisions at random, but hopefully this way there'll be more people who are pushed towards making a decision instead of clicking blindly.

share|improve this question
6  
While essentially good suggestion, I'm against it. In my opinion, it will just cause the badge hunters to simply start rejecting randomly. Better have false approvals (that can later be easily reverted) then false rejections which means good edits goes to oblivion. –  Shadow Wizard Oct 21 '12 at 14:30

You must log in to answer this question.

Browse other questions tagged .