What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 133 Stack Exchange communities.

From what I've seen, many SO moderators trawl completed reviews and watch out for robo reviewers; and manual banning them when they see enough (>= 3?) bad reviews. This is inefficient because:

  • It doesn't integrate into the audit system.
  • There's no "memory" to it. If mod A finds that user X has made two (any number less than the threshhold) bad reviews, he may make a mental note but there probably won't be any permanent/shared memory of this. If moderator B finds this user at a later date, he will have to look through the entire history, including stuff that mod A has already checked.

What I propose is that moderators get the ability to mark individual reviews as "invalid" from the completed review page. This is only to be used for reviews where any reviewer who is paying attention (and knows the rules) would choose the opposite reason (spam and other low quality stuff).

These bad review flags are treated like audits for a user -- if the number of failed audits and bad reviews is enough, the user gets autobanned. The nice thing is that this system should (probably) easily be able to link the banned user to the failed audits and/or bad reviews, fixing the "Why was I manual banned?" problem.

This could also be integrated well with this feature request.

share|improve this question
11  
"...and minitech, as he trudged through every single review item, was never seen in the flags queue again..." –  animuson Sep 7 '13 at 22:49
3  
I just love how we all handle flags, but each of us has something we like to specialize in. –  Andrew Barber Sep 7 '13 at 22:52
6  
@AndrewBarber because each one of you has a pet peeve that drives you up the wall. I think we've figured out that minitech hates 2 things: having rep on meta (since he gives it away as soon as he gets it) and poor reviewers. –  psubsee2003 Sep 7 '13 at 23:09
    
One little point: make sure the reviewed post can still be seen as it was when the review was made. –  Mark Hurd Sep 13 '13 at 5:23
3  
Please please please yes this. We can call it the "Minitech Signal". –  LittleBobbyTables Nov 4 '13 at 18:01

3 Answers 3

I like the idea of marking bad reviews, because right now I either have to keep a "naughty" file with a list of these, or permanently annotate someone's account with them.

There's one specific case where this could even be automated. For answers in the "Late Answers" and "First Posts" queue, I would like some sort of marking or identification when a user votes "No Action Needed" on a post that is later flagged as spam / offensive and then deleted. People who approve these are causing active harm to the site, and I want to know about these instances. There would be very few false positives from this, in my experience.

For example, I've made a habit lately of checking the review history for every piece of spam I find. In one recent case, 5 people voted "No Action Needed" on multiple instances of our recent "Watch Streaming Football Now" spam over a 24 hour period. I had to annotate their accounts with this, so that myself or any moderator in the future would see why these people were taking a very long vacation from review. Every one of these users would have been marked by the above test.

Even better would be some kind of addition to the review stats moderators see, where we could sort by tagged bad reviews to immediately identify problematic reviewers for further scrutiny.

share|improve this answer
    
I have noticed a lot of bad suggested edit reviews, can a review be flagged? Would that even be desirable, I can see how that would add a lot of noise. –  Shafik Yaghmour Feb 26 at 17:01
    
@ShafikYaghmour - If you see something particularly bad in a review (vandalism or spam being approved, etc.), feel free to flag the post with what you saw and we'll take a look at it. Disagreements about reviews of formatting or tagging we might not step in on unless it got really bad, but very damaging reviews that get approved are helpful to let us know about. –  Brad Larson Feb 26 at 17:15

I noticed some real egregious reviewing going on also.

I was thinking that maybe there should be a mechanism to "flag"/"vote on" the reviewer and the specific review, by clicking on his name on that review and a dialog comes up, and ultimately building a rating for that reviewer based on commmunity input, that only the moderators could see. Eventually, this might give them some data to correlate to bad reviews and they could implement a heuristic to use that. For example, they could use it to influence the audits given a particular reviewer. I don't think I'd want to go to direct autobanning though.

I'm sure with my contrarian views, I'd get tagged a lot, but I'm comfortable with the mods (as a group) reviewing the extreme cases and making decisions based on that.

share|improve this answer
3  
This would be awesome. I have some backburner'd scripts that track down bad reviews (finding a single bad review can lead to more, and more bad reviews means more bad reviewers), and that would be really useful here. –  Manishearth Nov 4 '13 at 18:03

The review history itself is available to even low-rep commoners like myself.

A simple flag option as I have for dubious questions, answers or comments would allow some "reviewing of the reviewers" by the large number of people who could and would feel compelled to flag questionable edits, like the one I just saw approved and wish I didn't:

enter image description here

UPDATE:

Seems like that particular review did get discovered by a greater power and thankfully overruled.

My case in point still stands though as this is not an isolated case (and I don't want to take down my unicorn pic or people's lives may be that little less colorful).

share|improve this answer
    
Is the pony a screen-overlay or a CSS-background? –  Synetech Dec 23 '13 at 7:43
    
it is but a memory fragment of our childhood years, when we played with semi-transparent unicorns (or rather, a background layer in GIMP, with opacity set very low) –  Leon Stafford Dec 23 '13 at 11:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .