This post details the first in a series of releases related to the ongoing work on Review Queues. It focuses on changes to the ways in which users who have been suspended from reviewing are notified of this event. These changes are now live across the network.
“Review Ban” renamed to “Review Suspension” change
With this release, we are changing the terminology that is used to label a user whose privileges to review posts has been revoked. Up until now, this was called a “Review Ban”. It is now called a “Review Suspension”. Through this change we are seeking to soften up the language and emphasize the temporary nature of the suspension of privileges.
This change will be reflected in all of the notices described below, as well as in the admin interface that is available to moderators for applying review suspensions.
Review suspension notices
The current review suspension experience falls short when it comes to making sure that the user is aware of their suspension. While we currently remove the list of review queues from the review dropdown, no notification about the review suspension is visible there. There is a message on the reviews page while the review suspension is active, but it lacks visibility, and disappears as soon as the suspension is concluded. Furthermore, nothing points the user to visit the review page during the review suspension, so users often miss a message that gives them information on the cause of the suspension and resources for improvement.
This situation unintentionally makes it hard for the user to learn about what they did wrong and can lead to the user making the same mistakes once their review privileges are restored. It can also lead to moderators giving overly long review suspension terms, simply because that is the only way to try to ensure that the user will even see the notice during the review suspension.
Review Suspension notices within the /review section
The notice text that is included in review suspension notices is being improved in this release (in the below screenshot, the default text is the first line; there is also a more helpful explanation text given by default in the event that none was entered by the mod who applied the review suspension). We'll be creating new guidance and suspension notices as part of a future project.
While the review suspension is in effect, the message will be visible on the main
/review page as well as each individual review queue page and the system will track if the suspended user has seen the full review suspension message. During the review suspension, the “Got it” button won’t be visible.
If a user doesn’t see the suspension message while the suspension is in effect, the message will persist on all review pages until the user has clicked the “Got it” button confirming that they have seen the message.
Top bar review dropdown
During a review suspension, the top bar Review Queues dropdown will now show a notice informing the user of their review suspension. This is in place of the current messaging that just says “There are no review queues available to you”, without making reference to the review suspension or even giving a link to a page with the suspension message.
In the case where the review suspended user has not seen the full message on one of the
/review pages during the suspension period, the top bar review suspension notice will also persist after the suspension has expired until the user has clicked the “Got it” button as seen in the prior section. But, since they have access to review, the individual queue listings will appear in the dropdown again.
Other Changes, Metrics & Feedback
Some changes are being applied to the review queues listing found on /review:
- Each review queue will have a direct link to its Stats and History pages from the listing (currently you can only access this from the queue itself after loading a task).
- Users with review suspensions will be able to load the full list (though the queues themselves will be greyed out and will not be clickable), in order to give them an avenue to access Stats and History.
- All permissions notices on the page are restyled using Stacks.
- We have published a new help center article titled Why was I suspended from review queues?. This will be linked from the new default review suspension notices.
The overall goal of this functionality is to make the act of suspending a user from review be more effective in helping the user to learn and improve their reviewing skills. To that effect, among the metrics that we will be tracking to measure the success of this functionality are:
- Number of review suspensions applied over time
- Average length of review suspensions
- Amount of time from beginning of suspension until when the user sees the notice
- Percent of users who see the review suspension notice during suspension versus after suspension
- Repeat suspension rate for users
- Rate of reviewers who return to reviewing activities post-suspension
We are happy to read and respond to feedback and bug reports posted below relating to the areas covered by this release, for one month following the publication of this post (after that point, new issues should be posted as new questions).