86

Currently, as reviewers fail audits, they are banned from review for increasing durations (2 days, then 7, then 30). At present, this caps out at 30 days. The next time a reviewer is banned, the ban duration drops down to 2 days. Even moderators cannot hand out review bans of longer than 30 days.

Over at Stack Overflow, we have quite a few users who have been banned from review 20+ times. All of them kept coming back after being banned, since they knew they could just wait out each 30 day ban and proceed to game reviews after that. For example, I saw this in the history for one user today:

Jan 21  user has been banned from review    duration = 30 days
Jan 11  user has been banned from review    duration = 7 days
Jan 4   user has been banned from review    duration = 7 days
Dec 5 '15   user has been banned from review    duration = 30 days
Nov 27 '15  user has been banned from review    duration = 7 days
Nov 25 '15  user has been banned from review    duration = 2 days
Oct 21 '15  user has been banned from review    duration = 30 days
Oct 14 '15  user has been banned from review    duration = 7 days
Oct 12 '15  user has been banned from review    duration = 2 days
Sep 11 '15  user has been banned from review    duration = 30 days
Sep 3 '15   user has been banned from review    duration = 7 days
Sep 1 '15   user has been banned from review    duration = 2 days

Moderators receive no notification of these serial review abusers, so we have no idea what they're doing until we happen across their ban history. We've been instructed to start suspending their main account on repeated review bans, but without a way of knowing this is happening, that's a limited way of preventing them from causing damage to the site.

Therefore, I'm requesting that we allow for further escalation of the review ban duration past 30 days. This would require extending the sliding window on which bans are calculated. At present, this is 30 days, but that window could scale with the duration of the last review ban.

I'd like to see an escalation on the order of 2 days -> 7 days -> 30 days -> 60 days -> 365 days and the ability of moderators to impose manual review bans of up to 365 days, in line with our ability to suspend accounts for up to 365 days.

Moderators can always manually remove review bans, so if someone is trapped by a series of unfortunate circumstances (bad audits, etc.), we can still override the system.

I really do think we need a better way to stop the harm that certain abusive reviewers are causing to sites when they aren't deterred by the current review ban cycle.

  • 5
    I just want to know why they keep coming back after being repeatedly banned. WTH? – Josh Caswell Feb 16 '16 at 19:28
  • 9
    Funny user. The ability to get banned as soon as the previous ban ends is hillarious. User never missed a day in between the bans. – Infinite Recursion Feb 16 '16 at 19:28
  • @JoshCaswell For badges, or to get on the review stats leaderboard. I bet all their reviews are No Action Needed (or the one-click equivalent in the other queues). – Ben N Feb 16 '16 at 19:29
  • 11
    Perhaps if a user fails too many audits, the last X reviews they did should be invalidated - deducted from their count and put back in the queue. – Ben N Feb 16 '16 at 19:32
  • 2
    @JoshCaswell - In this case it was very clearly just to get the gold review badge for a queue. At which point they immediately stopped reviewing in that queue and moved on to the next (and got banned there). You see this pattern quite often. – Brad Larson Feb 16 '16 at 19:43
  • 4
    @BenN Or better yet, just automatically award them the gold badge so they no longer have an incentive to continue reviewing, and don't even try to subvert whatever systems we put in place to keep them out. – Servy Feb 16 '16 at 22:06
  • 7
    @Servy: That cheapens the gold badge for everyone who got it legitimately, though (even more than it already is). Giving them a gold badge that's mutually exclusive with Steward might be more interesting, though. – Nathan Tuggy Feb 17 '16 at 1:56
  • 2
    @NathanTuggy To cheapen it would be to assume it has any value to begin with. I consider it a false assumption. The only people who I've ever seen attach any value to the badge are people who have no interest in performing quality reviews. I've never met a single person to have considered someone else a good reviewer simply because of a badge, or attached any value to any other person's review badge ever. If giving them the Shiny Thing means that they'll stay out of the review system and let the actual reviewers review, I'd consider it no loss to anyone, and a big gain for the site. – Servy Feb 17 '16 at 4:37
  • 1
    @Servy: hyperboleCounterExamples++; – Nathan Tuggy Feb 17 '16 at 4:39
  • 1
    @Servy: Well, you can (as implied) either consider this your first sighting of such a person in the wild, or write me off as having no interest in performing quality reviews. I don't consider someone a great reviewer just because they have a gold badge, but I do consider total gold badges somewhat of an indicator of how much focused effort someone has put into the site. (And where there's one person you hadn't previously spotted, there are likely to be more.) – Nathan Tuggy Feb 17 '16 at 4:46
  • 1
    @NathanTuggy I guess I'm just rather shocked that you consider mashing a button 20 times a day, knowingly harming the quality of the site in the process, for just under 2 months to be a meaningful amount of effort worth praising. But hey, I guess the bar for some people just isn't very high. – Servy Feb 17 '16 at 4:51
  • 1
    @Servy: You must have gravely misread me somewhere. I said, very specifically, that it cheapens the badge for those who got it legitimately. How you got from there to assuming I highly value a badge I know specifically to have been earned by terrible review actions I cannot imagine. – Nathan Tuggy Feb 17 '16 at 5:29
  • 2
    Maybe it would be a good solution to decrease the badge progress when a user is banned. – Laurel May 8 '16 at 1:31
  • 1
    Not sure why we don't just give lifetime bans after a certain number of failures. Why is reviewing a 'right' for those people who have consistently displayed behavior of doing absolutely the minimum possible to accrue large review counts and badges? I don't think there is any overlap between these people and the people who fanatically review because they love to correct things and keep the site clean. – user219993 Nov 28 '16 at 9:55
  • 1
    @GhostCat 1. It is not true that everyone would be banned. 2. Doing 5% wrong reviews has a cost which more than cancels out the value of the right ones. 3. There is a fundamental difference in helpfulness between someone who wants to improve the site, and someone who will do anything, positive or negative, to get badges and points. – user219993 May 11 '17 at 20:41
64

Starting today, thanks to Oded, moderators now have a way to give extended bans from review. When manually banning a user from the mod tools, moderators will see a list of options similar to what is shown when suspending a user:

review ban

Not only can you hand out longer bans, we've also adjusted the way automatic bans are handed out to users who fail audits. When calculating the automatic length of the ban, we check to see when the last ban ended and determine the new length based on the following requirements:

  • If the time since the last ban ended is less than or equal to 30 days, then the new ban is double the length of the last one. This is not capped, meaning if the last ban was for 365 days, then the new one is 730.
  • If the time since the last ban ended is greater than 30 days, then the new ban is half the length of the last ban or 2 days - whichever is greater.

This should help with reviewers who continually fail audits and need extended time away from reviewing. It also allows moderators to 'reset' the ban length by handing out a ban of any length, which would then be used by the automatic calculation of time-off.

  • 11
    I wonder, will review ban affect a moderator? (feel free to give me a diamond and test it on me... ;)) – Shadow The Dragon Wizard Nov 23 '16 at 22:09
  • 7
    @ShadowWizard Yup, mods can get banned. – Taryn Nov 23 '16 at 22:12
  • Huh! Has it ever happened for real? – Shadow The Dragon Wizard Nov 23 '16 at 22:12
  • 21
    Yup Shog has been banned from reviewing – Taryn Nov 23 '16 at 22:13
  • 5
    @ShadowWizard I wondered that too! bans self from review queues – Rand al'Thor Nov 23 '16 at 23:40
  • 6
    It was easier for me to think about the automatic ban like this: people have a 30 day probation from when their last ban ended. If they get banned within the probation period, they get double the last ban, if not, the ban is half the last ban. – Oded Nov 24 '16 at 9:34
  • 11
    Could you ban Community diamond? It (?) seems to approve a lot of crappy suggested edits and consistently displays passive-aggressive behavior when you try to confront it. – Robert Columbia Nov 26 '16 at 13:08
  • 1
    @RobertColumbia That is not how it works. Community♦ is a bot that does actions as a consequence of actions of community members. See meta.stackexchange.com/questions/184992/… for a detailed explaination, – wythagoras Nov 26 '16 at 14:10
  • 1
    Does this mean that after 30 days, ban goes to half and after another 30 it goes to half? Or just once? – Vojtech Ruzicka Nov 27 '16 at 12:40
  • 1
    @Oded does this completely replace the old 30 day progression (meta.stackexchange.com/a/190562/182513), or does this kick in after the first 30 days? – psubsee2003 Nov 27 '16 at 13:03
  • 5
    @psubsee2003 - it replaces it. – Oded Nov 27 '16 at 16:50
  • @VojtechRuzicka Now it depends on when your last ban ended. If you were last banned within 30 days, your new ban is double the last one. If it was over 30 days, then your ban will be half your previous ban. – Taryn Nov 28 '16 at 14:58
  • @bluefeer So it does not eventually degrade back to 2 days, right? I will be stuck forever with possibility of having ban, which is minimum half of my latest ban? And the threshold will be forever one failed attempt? Eg. If I have ban for two months - after 30 days I will have ban for one month after one failed attempt. After a year the same? Just making sure I understand it correctly :) – Vojtech Ruzicka Nov 28 '16 at 15:03
  • 1
    @VojtechRuzicka If it's been more than 30 days since your last ban, and your last ban was 30 days, then the next ban is 15 days. And it continues the same way. If you don't get banned again for 30+ days, then your next ban would be cut in half. So eventually you could work back down to 2 days. – Taryn Nov 28 '16 at 15:10
  • 4
    @VojtechRuzicka Or just make sure you don't ever get banned again. Then the theoretical length of your next ban won't matter at all. – user219993 May 11 '17 at 20:43
17
+200

I think this is a very good idea. However, I think that the ban level should go down after a few months. Everyone can make mistakes on occasion, I think I've been a few times very near a review ban (at least I failed a few consecutive audits) on the close queue because the site changed it viewpoint on homework and the reviews were old posts.

So I propose this:

  • Users start at level 1.
  • After every ban, it goes up by one level.
  • After 90 days after the last ban ended, it decreases by one level.

Maybe this should only happen at level 2 (when the level is 3 or higher, it can't be a mistake, then it has to be robo-reviewing, or just very careless reviewing).

  • 5
    I think this would be handled by the sliding window on bans. Currently, bans are de-escalated if the audit failures come a long enough time after the last failure or ban. We'd have to scale the sliding window to account for the ban duration, but it still would be present to allow for these times to come down with good review behavior. – Brad Larson Feb 16 '16 at 17:52
  • @BradLarson I agree. But the current sliding window is 30 days, it should be longer. – wythagoras Feb 16 '16 at 19:51
  • @wythagoras: I don't think it should be longer, necessarily; it's already at one failure in 30 days, that's fairly harsh. (I'm speaking as someone who has not yet suffered any review bans at all, by the way.) – Nathan Tuggy Feb 16 '16 at 19:56
  • @NathanTuggy No, one failure doesn't get you another ban immediately. – wythagoras Feb 16 '16 at 20:23
  • 2
    @wythagoras: Per e.g. meta.stackexchange.com/questions/226815/… and some dupes, yes, a single failure within the window after a ban does instantly ban you again. – Nathan Tuggy Feb 16 '16 at 20:39
  • 1
    I have awarded my bounty to this answer because it is closest to what I'd like to see. I don't think the ban level should go down automatically; it should only go down by passing audits, and (obviously) not getting manual suspensions for robo-reviewing. – S.L. Barth May 9 '16 at 11:02
  • @S.L.Barth Thank you very much! I was like, where did that +200 come from? – wythagoras May 9 '16 at 11:21
  • You're welcome! I started the bounty to give this feature request more attention. With the bounty period expired, it was time to award it. As I said, your answer comes closest to what I believe should be done. – S.L. Barth May 9 '16 at 11:31
  • Based on this discussion, if I come out of a ban and sit on my laurels without reviewing for 30 days, I'm going to have a higher review rep then someone who comes out of a ban and starts reviewing immediately - crazy – danday74 Oct 30 '18 at 8:50
13

This sounds like a suitably rare occurrence that it should be enough to just raise an automatic mod flag on the ban immediately following a 30 day ban. There should be few enough cases that it wouldn't be burdensome for an actual person to evaluate these case. If it's clear that the user is a serial abuser demonstrating no intention of rectifying their behavior, then extremely severe actions can be taken at that time, and if it's a user that's clearly doing their best, but just making more mistakes than they really should be, the mod can respond appropriately to aid the user in understanding what they're doing wrong and taking the appropriate steps to learn how to improve.

  • 3
    As far as frequency, looking at just the users currently banned from review on Stack Overflow, this would have applied to 29 of them. Looks like we'd get roughly a flag a day about people coming off a 30-day ban and getting banned again within the rolling window. – Brad Larson Feb 16 '16 at 21:55
  • 5
    @BradLarson Well, if some of these people end up with more serious consequences as a result of the added attention, hopefully that number would go down a bit after the first month ;) – Servy Feb 16 '16 at 22:04

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .