54

As users are now inbox-notified for each review suspension, there is now one edge case that may cause confusion to users --

Background

On Stack Overflow, we have users1 who have over 50 previous review suspensions, and moderators have previously sent such reviewers with poor review histories away with a manually-applied "permanent"2 review suspension with this message:

Due to your poor review history as well as no signs of improvement after many review suspensions, you won't be able to use any of the review queues on the site any longer.

Issues

  1. As durations for manual review suspensions are capped at 365 days, moderators have to occasionally revisit the current suspensions and reapply a new 365-day review suspension with the same message, to ensure that the "permanent" review suspension remains permanent.

    Image below: Shows the final step of the mod form to manually review suspend a user, maximum allowed is 365 days (while there is no limit when user is suspended for failing audits)

    enter image description here

  2. This now may cause confusion to the "permanently"-review suspended users with the recent change to notify users with each review suspension, as there was no new review action taken by them to get a new review suspension, while still getting a notification for each renewal in the inbox, reminding them that they are still review suspended.

    Image below: Shows the review suspension history of a user, and that they see the permanent review suspension renewal notice. This UI has been supplemented with Sam's UserReviewBanHelper userscript to show time taken for viewing of review suspension notices.

    enter image description here

  3. Also, the steps taken to manually-unsuspend and re-suspend with the same message is quite tedious (no joke), and due to this, moderators do not actually care about monitoring and renewing permanent review suspensions. I have written a userscript that currently does the renewal in one click, but this only works for as long as I'm a moderator on Stack Overflow.

    Image below: Shows one user in the list of currently review suspended users, who has a manual permanent review suspension, and the button added by Sam's UserReviewBanHelper userscript which allows mods to reapply the manual permanent review suspension in one click.

    enter image description here

Feature Request

Please allow moderators to apply permanent review suspensions (as opposed to manually renewing 365-day suspensions).



Footnotes:

1 At time of post, Stack Overflow has 33 such users, as tracked in my Stack Overflow Review Ban Stats spreadsheet.

2 "permanent" is not truly permanent, as it's just a manually-applied and manually-renewed review suspension and can accidentally expire like all review suspensions if no moderators remember to perform the renewing ritual.

14
  • 2
    Why would you have to extend ban every year in the first place is beyond me.
    – Alvi15
    Nov 27 '20 at 8:01
  • 6
    We have to unsuspend and resuspend before the 365 days is up. So to avoid "forgetting" to do it, I try to renew "permanent" review suspensions every few months or so. This notifies all the "permanently"-review suspended users that they are still review suspended. Nov 27 '20 at 8:04
  • 8
    Just out of curiosity, when is the moderator team willing to entertain appeals of indefinite review suspensions? Nov 27 '20 at 8:10
  • 11
    @Sonic Usually when it gets to the stage where the mod team decides that a user should not be doing any more reviews as the user has shown no sincere attempts at correcting the way they are reviewing after extreme leniency of 30+ prior suspensions, it is unlikely the user has any basis for an appeal (and unlikely to appeal anyway). Nov 27 '20 at 8:14
  • 1
    Current feature is a pain for moderator and for the permanently banned user. Time to wait for approval ?
    – Alvi15
    Nov 27 '20 at 8:15
  • Thanks. Also, as far as I'm aware, review suspensions for failing audits are uncapped by time, so if a user fails a single audit within 30 days of coming off a 365-day review suspension, they'll be suspended for 730 days, so this is technically possible. Nov 27 '20 at 8:17
  • 4
    If a user is suspended from review, can't they still continue their behaviour outside of reviews? At that point I think a full suspension would be more appropriate. Nov 27 '20 at 8:36
  • 5
    @Wrigglenite in-review behavior tends to make more impact. The important difference is that inappropriate reviewing may do more harm, like approving and encouraging low quality / spam posts at the very stage that is intended to catch and stop these. Out-of-review behavior issues, like inappropriate voting, are supposed to be more tolerable just because these are expected to be corrected later in the course of review - and if this doesn't happen (eg because of reviewer negligience) site tends to be polluted by an unhandled garbage
    – gnat
    Nov 27 '20 at 9:11
  • 1
    "This now may cause confusion to the "permanently"-review suspended users". Is reducing the slight confusion of such users (who, I hope, are few in number) worth the programmer time?
    – Raedwald
    Nov 27 '20 at 17:47
  • 2
    @gnat In the Low Quality review on Stack Overflow, two users reviewing Looks OK are enough to outweigh three users reviewing to delete an answer. (On other sites, it's even worse: a single Looks OK reviewer can outweigh five users voting to delete.) There is an automatic moderator flag raised in this case, but if too many people do it... Nov 27 '20 at 18:30
  • A solution is to double or triple the 365 day limit (for any kind of suspension, as long as a subsequent one is only double a prior one); if someone's suspension gets pinned at the limit it should be flagged for attention by a CM to determine if a general, or lengthier, suspension is called for. --- What kind of behavior is it to receive over 50 prior suspensions (of any category); yet we would expect them to be useful in their other endeavors - if there's one thing that doesn't sink in I think we need to have staff look at that. SA: meta.stackexchange.com/a/211220/282094
    – Rob
    Nov 27 '20 at 18:51
  • 12
    My lowly-user opinion: If you've been banned that many times, then permanent should mean permanent and no further action should need to be taken by the mod team. That's an absurd number of bans and has given them way more opportunities to reform than they probably deserved. Permanent should mean exactly that.
    – KGIII
    Nov 27 '20 at 20:51
  • 4
    @Raedwald perhaps not but I'd say that reducing the workload of moderators for something that is a routine task is worth it. It eliminates the need for them to roll their own tools. These tools might get broken by a site update of some sort and would further impact moderators. Moreover, any such tool carries the risk of doing the wrong operation for whatever reason. So, the real question is whether or not better and more accurate workflow for diamond mods is warranted. The benefit to the "permabanned" users might negligable but comes for free with this. It's not really the focus of the request
    – VLAZ
    Nov 30 '20 at 8:52
  • 2
    Just ban them from the site writ large if they are so intent on abusing one of the site features. I don't think we need a permanent ban on one feature before we try banning them entirely.
    – TylerH
    Nov 30 '20 at 14:30
8

One of the central tenets of our moderation policy is that people should be given a chance to show if they've changed their behaviors and we do that in various ways.

  • We allow one question every six months from a question banned user.
  • Suspensions generally max out at a year without staff issuing a network-wide suspension.
  • Chat suspensions generally max out at 9999 hours (~417 days).
  • Review suspensions are a year max.

I understand that there can be users who are frustratingly consistent in their bad reviewing practices but, particularly now with the review UI being improved to help users be more successful in review and the new help center pages about how to review correctly, we want to ensure that users get that chance to try again.

Review is generally not a place that a lot of damage can be done. In most cases, reviews require more than one participant to clear a review item, so I don't see permanent review bans as preventing major harm to a site.


As to the people you're mentioning who your script automates suspensions for - I've reviewed several of them and, while I do see that some have >10 suspensions, most have had suspensions for actions in different queues, so they might have been suspended once for Triage and another time for First Posts and another time for Low Quality Posts. While it can be difficult to see people hitting multiple walls, it's not as if they're always getting review suspended for the same incorrect choice every time. Review is a complex learning process and each review queue is different and can take some time to learn about.

Additionally, I'd point out that many of these review suspensions could have gone unseen by the user who was suspended because the messages weren't persistent in someone's inbox until July of 2020. In fact, most of the people I reviewed had gotten their first very long (365 day) suspension prior to that date.

I very much appreciate your goals in trying to keep poor reviewers out of review but I think, in this case, we've made a ton of changes to review and review suspensions in the last 9 months so we need to see how that impacts review behavior overall. My sincere hope is that most of these users find that they're better able to review effectively and are actually getting the assistance they need to understand when they reviewed incorrectly and how they should have reviewed instead.

As such, I'd like you to disable the script you're running to automate review suspensions for these users and it'd be super cool if y'all could review the list of people who have been review suspended and take into account when the prior suspensions happened, whether those suspensions were in the same queue or different queues, and consider whether they've served enough time off to give them another chance to use the new UI and help information for reviewers.

If so, feel free to unsuspend them from review and feel free to keep an eye on their reviews (if they perform any). If it seems like they should finish their existing suspension period, that's fine too, though I think it's worth pointing out that they were all suspended for a year in September 2020 and then resuspended in January 2021 for another year, which I could imagine would be somewhat confusing to them - to get an alert that they were suspended from review (again) when they were already suspended from review.

For all of these reasons, we're declining this request.


Regarding this:

On Stack Overflow, we have users who have over 50 previous review suspensions

To add a bit of data to this - there are three users with >50 review suspensions. Two of them are not currently suspended and haven't been since 2018 or 2016 (the latter of which is still reviewing as of December 2020 and seems to be doing OK).

The one who is still review suspended (54 total review suspensions) has been continuously suspended since March 6, 2017 other than a 4-day lapse in 2018, and only those repeat suspensions carry them over the 50 mark, including four 365 day suspensions within a one-year period (for some reason?).

There are 27 users with 30-50 review suspensions each. Of them, 20 (74%) aren't currently suspended from review and at least half of those haven't been suspended from review in several years. Similarly, there are 68 users with 20-29 review suspensions and 59 (87%) aren't currently review suspended and about half haven't been review suspended in the last two years.

So - what am I getting at? Sheer volume of review suspensions isn't necessarily an indicator of a reviewer who needs to be permanently banned - whether because they learn to review better or they just give up on reviewing entirely - for the most part, the need for a review suspension seems to stop in most cases.

Right now, I'd argue that the need for reviewers is high enough on SO that giving people the chance to learn with the new UI is worth the potential assistance they may be able to provide in good reviews.

1
  • I very much agree with this. Let's see what the new review UI does to the amount of repeat offenders. Hopefully this solves (part) of the problem, and would take away a tremendous amount of work from moderators and volunteers alike. I remember that I personally spend time checking back a lot of reviews on SO, and reporting inappropriate reviews back to a chatroom for mods to take action on. Hopefully this all won't be necessary towards the future.
    – Luuklag
    Mar 30 at 20:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .