We've been discussing the quality of some reviews on Ask Ubuntu. Some of us were getting frustrated that some users were smashing through 20 reviews in a minute so I decided to lay down the law.
Aaaanyway, as part of this one of our newer reviewers brought up the problem that if they take too long being diligent (looking into duplicates properly, checking rules, etc) there's a very good chance that somebody else will have reviewed that post and their decision (and the time that went into it) is wasted. That doesn't seem like a fair reward for what is an otherwise thankless task.
So I'm suggesting the a review "reservation" system so only one person can review something at once:
- Once something is opened in review, it becomes unavailable to other users to review. That includes other queues. A reservation or lock should be against a post, not a post in a queue.
- The reviewer should get, for example, two minutes of alone-time with a post.
- If they review before their time is up, the lock goes away.
- If they're still reviewing when their time runs out, a notification should pop up (audio too?) and allow them to request an extension... Perhaps with 10 seconds grace to stop somebody else being allocated something.
- Questions are simply allocated on a first-unlocked-post basis.
- There should be a safety-limit on the number of times something can be review-locked or extended so that somebody can't wrongfully extend the life of a bad post (offensive/spam/etc) by gumming up review.
It sounds messy (and there's certainly some concurrency issues if you've split the servers out horizontally) but data-wise, it's actually not too hard to implement (speaking as somebody who has implemented an edit-lock system before).
Above all else, it shows reviewers the respect they deserve for a job that few of us really want to do.
