77

As a moderator on Stack Overflow, I regularly look through the review history of obvious spam that has lasted more than an hour or so on the site. I often find one or two reviewers who voted "No Action Needed" in the First Posts or Late Answers review queue, allowing that spam to pass through review. I then have to look into the review history for each user and apply manual review bans as needed.

Almost always, the reviewers that have done this have failed multiple audits and have been banned from review on several occasions. It's a time-consuming and frustrating process to go and manually evaluate and ban each of these reviewers for approving obvious spam that I or the community later found.

Therefore, I'm proposing that when a post is destroyed by the community as spam (6 spam flags or one hard moderator spam flag), any reviewer who previously voted "No Action Needed" on that spam is immediately hit with a review audit failure as if they had reviewed that post incorrectly at the time it was destroyed. For those who had failed other audits, this would automatically trigger a review ban of the appropriate duration.

This would automate the process we already follow when dealing with those who approve spam. It would only apply to cases of spam so clear that the community or a moderator had to step in and destroy (not just delete) the post, yet someone voted "No Action Needed" on. Such posts automatically become audit candidates anyway, and currently register as an audit failure if someone votes this way after they are destroyed.

I can see placing a statute of limitations on this, such as only applying to reviews in the last 30 days or so. I don't ban reviewers I find who approved spam months ago (usually because they've been banned at least once in the interim).

I believe a system like this would help to identify and limit the damage of those who are letting spam slip through review without hitting attentive reviewers.

  • 2
    I find this very useful. Support! – ʇolɐǝz ǝɥʇ qoq Oct 15 '15 at 17:24
  • 3
    This almost sounds to generous. Someone who has already had multiple review bans and robo approves something which is deleted as spam? Don't just slap their wrists, give them a permanent review ban. – enderland Oct 15 '15 at 17:31
  • 14
    @enderland - Oh, how I wish we could for some people. Max is 30 days, and we've got some people on SO who are now on their 40th+ review ban. We've had to start outright suspending some accounts to get certain high-rep users to stop this. This proposal is more about automating the identification and handling process for people the system may not have caught yet. – Brad Larson Oct 15 '15 at 17:42
  • Good idea. I definitely agree with the statute of limitations, though; sometimes we find and destroy old spam, and I wouldn't want to ding a user who has long since learned about that. @enderland I wouldn't agree to an automatic perma-ban because sometimes spam seeds look credible, but letting mods separately perma-ban people would be good. (Currently there are no perma-bans AFAIK.) – Monica Cellio Oct 15 '15 at 17:44
  • Community flags may be added to the package too. – Braiam Oct 15 '15 at 19:59
  • I really dislike your proposed method; you're using a concept for something it wasn't designed for and it just makes the data overly funky. Why not just review ban for a period. There's no need to create non-existent reviews in order to get there. A ban is a different concept than an audit and doesn't have to be created by one. – ben is uǝq backwards Oct 15 '15 at 20:47
  • 5
    @benisuǝqbackwards - The reason I was using a generated and immediately failed audit as a mechanism is that bans have to have some reason to them, and failed audits automatically provide that. Additionally, the severity of the ban would be calculated based on the number of recent failed audits / bans before this one, which is already done for failed audits. Seemed like an easy way for the developers to inject this into the existing system without the addition of a completely new mechanism. – Brad Larson Oct 15 '15 at 20:51
  • 5
    Make sure there hasn't been an edit (e.g., revision one looks legit, goes through review, spammer edits spam in after review is done). Not sure if any spammers actually do this... – derobert Oct 23 '15 at 17:06

You must log in to answer this question.

Browse other questions tagged .