Give moderators the ability to mark reviews as "bad"

Question

From what I've seen, many SO moderators trawl completed reviews and watch out for robo reviewers; and manual banning them when they see enough (>= 3?) bad reviews. This is inefficient because:

• It doesn't integrate into the audit system.
• There's no "memory" to it. If mod A finds that user X has made two (any number less than the threshhold) bad reviews, he may make a mental note but there probably won't be any permanent/shared memory of this. If moderator B finds this user at a later date, he will have to look through the entire history, including stuff that mod A has already checked.

What I propose is that moderators get the ability to mark individual reviews as "invalid" from the completed review page. This is only to be used for reviews where any reviewer who is paying attention (and knows the rules) would choose the opposite reason (spam and other low quality stuff).

These bad review flags are treated like audits for a user -- if the number of failed audits and bad reviews is enough, the user gets autobanned. The nice thing is that this system should (probably) easily be able to link the banned user to the failed audits and/or bad reviews, fixing the "Why was I manual banned?" problem.

This could also be integrated well with this feature request.

Answer 1

I like the idea of marking bad reviews, because right now I either have to keep a "naughty" file with a list of these, or permanently annotate someone's account with them.

There's one specific case where this could even be automated. For answers in the "Late Answers" and "First Posts" queue, I would like some sort of marking or identification when a user votes "No Action Needed" on a post that is later flagged as spam / offensive and then deleted. People who approve these are causing active harm to the site, and I want to know about these instances. There would be very few false positives from this, in my experience.

For example, I've made a habit lately of checking the review history for every piece of spam I find. In one recent case, 5 people voted "No Action Needed" on multiple instances of our recent "Watch Streaming Football Now" spam over a 24 hour period. I had to annotate their accounts with this, so that myself or any moderator in the future would see why these people were taking a very long vacation from review. Every one of these users would have been marked by the above test.

Even better would be some kind of addition to the review stats moderators see, where we could sort by tagged bad reviews to immediately identify problematic reviewers for further scrutiny.

Answer 2

I noticed some real egregious reviewing going on also.

I was thinking that maybe there should be a mechanism to "flag"/"vote on" the reviewer and the specific review, by clicking on his name on that review and a dialog comes up, and ultimately building a rating for that reviewer based on commmunity input, that only the moderators could see. Eventually, this might give them some data to correlate to bad reviews and they could implement a heuristic to use that. For example, they could use it to influence the audits given a particular reviewer. I don't think I'd want to go to direct autobanning though.

I'm sure with my contrarian views, I'd get tagged a lot, but I'm comfortable with the mods (as a group) reviewing the extreme cases and making decisions based on that.

Answer 3

The review history itself is available to even low-rep commoners like myself.

A simple flag option as I have for dubious questions, answers or comments would allow some "reviewing of the reviewers" by the large number of people who could and would feel compelled to flag questionable edits, like the one I just saw approved and wish I didn't:

UPDATE:

Seems like that particular review did get discovered by a greater power and thankfully overruled.

My case in point still stands though as this is not an isolated case (and I don't want to take down my unicorn pic or people's lives may be that little less colorful).