This time last year, the Public Platform team announced an initiative to improve the review queues. Our goal was to explore new ways to better support our curator and moderator communities by revisiting the review queues used for community moderation.
With the help of the community, we have been able to accomplish a significant amount of work and made multiple releases including:
- Improvements to the review suspension experience
- New review queue help center articles
- Visual design changes to the queues
- Review queue onboarding
We’ve made a lot of progress with this project, but there is still work to be done. Today, we’re coming back to the community to discuss one of our last objectives: improving queue workflows and pathways. In parallel to our other releases, we’ve been taking more time to carefully consider what changes would make the most impact on the review queues. We’d like to share some new proposals and get your feedback.
The original plan
Our initial research helped us identify the following pain points in review queue workflows:
Problem #1: The Requires editing action in Triage was sending a surplus of unsalvageable posts to the Help & Improvement (H&I). H&I reviewers are then limited to either Skip or mark the task as Very low quality, having it return to the Triage queue and creating a frustrating endless loop.
Solution (implemented): We updated the actions available in the Triage queue and redirected tasks to where they may be better addressed.
- Needs community edit → Help & improvement queue
- Needs author edit → Close votes queues
- New flag action for other problematic posts
Problem #2: First Posts and Late Answers queues and combining those tasks into Triage. These queues were intended to give good posts a head start toward success and filter out bad posts, but tasks could see better, faster intervention.
Solution (not implemented): Initially, we considered deprecating these queues. When we got into technical discovery, we discovered that we couldn't simply move First Posts and Late Answers tasks into Triage because the Triage queue only accommodates question tasks.
We wanted to look at these problems again with fresh eyes. We revisited our internal conversations, reviewed some data, and reread the Meta feedback from previous posts to ideate on new solutions.
- Triage is a Stack Overflow exclusive queue and was created to quickly assess and provide feedback on questions that had been flagged low quality. This queue is often the root cause of other review queue issues. It lacks strong guidelines for reviewers to follow and assumes that they can identify good questions, both leading to inconsistent reviews.
- Currently, the Low quality post queue on Stack Overflow only handles answers. By deprecating Triage, we’d like to adopt how the Low quality posts queues on other SE sites function and route all posts (questions and answers) that have either been flagged as low quality or fall below the system-generated quality score threshold to Low quality posts.
Deprecate Help & improvement
- H&I hasn’t been successful in its intended purpose with issues from Triage diminishing the queue’s usefulness. Improvements to other queues could still satisfy the same need.
- H&I is our least productive queue. Over the last 90 days, only 7 tasks went into the queue each day on average, compared with 2,458 for First Posts (our busiest queue) and 219 for Reopen Votes (our second-to-last busiest).
Create separate First posts - questions and First posts - answers queues
- The First posts queue had good intentions in helping new users with their first posts, but the queue itself has no clear goal. Reviewers can take any or no action against a review task and is often not impactful or useful to the post.
- We want to separate questions from answer tasks and identify specific actions that may help new posts fare better as well as maintain content quality on the site.
Keep the Late answers queue
- The Late answers queue has been effective in removing low quality posts, repeat answers, and spam.
- Similar to First posts, we want to identify specific actions that help in resolving problematic answers.
Other: Prioritize aging review tasks
- Review tasks age out of the queues if they are not acted upon, so we’d like to ensure posts get the attention they need in a timely manner.
We will not be conducting 1:1 user tests or interviews in this release. Instead, we would like to open this up for discussion on Meta to gather a number of opinions. Please share your thoughts until April 28, 2021. Afterwards, we’ll analyze your responses and share new designs. We’re excited to read your responses and look forward to learning from your insights.