What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 129 Stack Exchange communities.

During a recent Review Suggested Edits session, I came across a reviewer who is clearly rubber stamping edits (Approved 550 edit suggestions and rejected 7), and who had missed an obvious case of vandalism by an anonymous miscreant.

How about considering mechanisms, which:

  • Give feedback to reviewers that their reject rate lies outside a normal range of the larger community (e.g. outside of 2 sigma)
  • Introduce some kind of penalty for repeat rubberstamping (e.g. revoking review rights and / or revocation of review badges?)
share|improve this question
10  
Welcome to the club –  Daniel Fischer Dec 3 '12 at 7:17
6  
+1. Generally, if I find such robo-reviewer, I comment on their older post and request them to not review like a robot. I planned to add a feature-request about giving feedback to other reviewer. Thanks. –  hims056 Dec 3 '12 at 7:20
4  
meh, i disagree... skipping should not be a punishable offense. aren't the audits supposed to take care of this problem? –  Jeff Dec 3 '12 at 7:43
3  
@Pekka He means if they see 20 good edits and 10 bad ones, they'll accept the 20 and skip the 10 (I assume because they feel bad rejecting? I'm not sure why) and have a 100% accept rate –  Michael Mrozek Dec 3 '12 at 8:09
6  
honeypot items for robo-approvers are easy to do: take a reasonably good post, make a suggestion to wrap it into senseless "Hi" and "Thanks", add an absurd edit comment like "improved formatting" and voila. Similar items with unambiguously good suggestions should be there, too, in order to avoid converting robo-approvers into robo-rejectors. Handful reviews proven fake => two weeks review suspension –  gnat Dec 3 '12 at 8:56
1  
...making unambiguously good honeypot item isn't a rocket science either: take a reasonably recent post with hi-thanks and missing code formatting; suggest to remove hi-thanks and format the code with comment like "1) code formatting 2) personal stuff removed" –  gnat Dec 3 '12 at 10:07
1  
i dont know why this fahim-parkar user approve .... –  NullPoiиteя Dec 3 '12 at 10:47
1  
@hims056 you mean, "please don't approve..." comments like this, this and this? I really like it! While SE team is hanging there in analysis paralysis, this seems to be the best community can do –  gnat Dec 3 '12 at 12:20
1  
@gnat - Yes. And now I also include another sentence like [I will remove this comment after your reply] So that they don't repine and reply us properly. :) –  hims056 Dec 3 '12 at 12:33
2  
@hims056 well I am not sure that replying helps. :( User who replied to second comment listed above, still has stats that raised my eyebrows "approved 438 edit suggestions and rejected 1" –  gnat Dec 3 '12 at 12:38
1  
@gnat - Yes I know that particular user but we can't handle every people but at least we should try to improve review quality. –  hims056 Dec 3 '12 at 12:41
2  
@gnat you would like to see status of stackoverflow.com/review/suggested-edits/1044307 here stackoverflow.com/review/suggested-edits/1044307 even not rejected one edit –  NullPoiиteя Dec 3 '12 at 12:48
4  
@NullPointer - OMG! Looks like Reject and Skip buttons are not visible on that user's browser. approved 999 edit suggestions and rejected 0 –  hims056 Dec 3 '12 at 12:53
5  
@gnat - and everyone else who is part of this conversation. Please refrain from vandalizing other posts and their comment-walls with references to people's approve/reject posts. That doesn't "improve the quality of the site's content" in any way - it merely muddies up what appears on other pages in an effort to penalize or shame someone. I agree that there needs to be a mechanism for noticing rubber-stamping - but that mechanism shouldn't be to turn our comment boards into trolling chat-windows. –  Troy Alford Dec 4 '12 at 18:28
3  
@GamecatisToonKrijthe's prediction has come to pass - the reviewer identified by NullPointer got his gold, and stopped. He has a perfect 1000:0 record as at the time of writing –  StuartLC Dec 7 '12 at 9:15

1 Answer 1

up vote 10 down vote accepted

As of now, there is no systemic way to provide feedback like one you ask about. As far as I understand, SE team is working on this.

Meanwhile, technically there is an option to politely contact reviewers making questionable decisions via comments as discussed above.

As an example, here is a comment to reviewer mentioned in your question:

@FahimParkar Please don't approve suggested edits like this. If needed, refer to this meta post for details.


Some more example comments submitted to reviewers who were complained about in meta posts...

share|improve this answer
1  
Your ideas about the honeypot - both false positive and false negative - are great way to detect and 'sentence' offenders. –  StuartLC Dec 4 '12 at 8:22
1  
As in my above comment - I don't think this is a particularly appropriate way to keep our community clean. It changes the topic of other postings and muddies what appears when users search for other topics. –  Troy Alford Dec 4 '12 at 18:29
1  
I'd just like to point to 2 legitimate reasons for your comment @gnat, so you can be aware that not everyone is simply rubber-stamping. 1. Reviewing on a mobile device means that sometimes the wrong button is clicked accidentally - there is no 'undo' button, but thats why there are 3 reviews before anything is accepted. 2. Also due to mobile devices, it is easier to Approve legitimate posts and Skip incorrect ones, then it is to perform Edits, therefore I have a generally higher accept count compared to my rejects/edits. Please dont be so quick to lump everyone in the same basket! –  WATTO Studios Dec 5 '12 at 1:35
    
I do not think systemic means what you think it means :] –  Emrakul Dec 25 '13 at 18:19
    
@Emracool to me, systemic means a way that wouldn't involve personal comments to randomly discovered problematic reviewers –  gnat Dec 25 '13 at 19:11
    
Ah, I meant by the actual definition. But I think you're right, technically it is an appropriate use of systemic. It just sounded weird to me xD –  Emrakul Dec 25 '13 at 19:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .