14

Anticipating a thought I had recently, concerning reviewers with faultless records of pressing "No Action Required" on first-post reviews, Servy comments:

I'd rather they just ditch both the daily and all time charts altogether because it's a big incentive that's generating so many of the poor quality reviewers. Making the all time board more robust would be detrimental to the system in my eyes. – Servy Nov 28 '12 at 17:24

Is there actual value to having this leaderboard? Unlike the Review History pages, which allow us to see the actual review activity, this screen only seems to amplify the skewed incentives in the current review mechanism.

Improve this question
6
  • 4
    But how else can I be incentivised to be the best and do the most reviews? More reviews == better after all.
    – ben is uǝq backwards
    May 15, 2013 at 13:04
  • 1
    @benisuǝqbackwards - badges, man. Shiny, collectible badges! They're like Pokemon...
    – LittleBobbyTables - Au Revoir
    May 15, 2013 at 13:05
  • @benisuǝqbackwards: You think the badges are not incentive enough? There's no leaderboard for voting.
    – Charles Stewart
    May 15, 2013 at 13:06
  • 2
    It was a joke @Charles :-), and there is a leaderboard for voting... stackoverflow.com/users?tab=voters
    – ben is uǝq backwards
    May 15, 2013 at 13:08
  • @benisuǝqbackwards: Sense of humour failure, sorry. Though I guess it might be the reason it is there.
    – Charles Stewart
    May 15, 2013 at 13:09
  • Pondering what I could write in "Structured participation with Review Stats".
    – Daniel Fischer
    May 15, 2013 at 13:37

1 Answer 1

Reset to default
11

I don't think the review leader boards are the cause of the problem. Looking at the stats for the First Posts review queue for people who have a minimum of 300 reviews in the last 30 days, I get a list of 45 names. I can see a decent number of names from both the "All Time" and "Today" leader boards. On average this group hits "No Action Needed" 63% of the time. The median is 67%, and half of them fall in the 54% - 77% range. This group is also overwhelmingly passing review audits. The lowest audit pass rate is 79%, with many people passing audits at a 100% rate. (The lowest number of audits done by anyone in that group in the last 30 days was 19. The highest number of audits was 163.) Based on these stats, I don't think these are the robo-reviewers.

Note: I have to point out that Servy's original point about the review leader boards was posted on November 28, 2012, which looks like it was one day before review audits were rolled out. That puts it in a different review "era" than we're in now. He might have been right at the time, and audits have just helped to correct a problem that really did exist.

Improve this answer
1
  • 2
    I think @CharlesStewart had sites in mind that currently don't use review audits. For those sites, his idea sounds good to me.
    – mafp
    May 17, 2013 at 15:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .