Visual design changes to the review queues

Question

The Public Platform team would like to announce another release in a series of planned work for the Review queues. This release focuses on refreshing the queue user interface in preparation for further improvements to the user experience. Starting today, we will roll out these changes in three phases across the network.

Roll-out plan

We have planned for these UI changes to ship in three distinct phases affecting specific queues across the network. We will update this post as each phase is rolled out.

	What	Where	When
Phase 1	• Home page (/review) • Close and reopen votes queue • Stats/History view • Steward badge	Meta Stack Exchange Meta Stack Overflow	Shipped! January 28, 2021
Phase 2	• Changes from Phase 1 • Suggested edits queue • Low quality posts queue • First posts queue • Late answers queue	SuperUser Ask Ubuntu	Shipped! February 16, 2021
Phase 3	• Changes from Phase 1 + 2 • Triage queue (SO only) • Help and improvement queue (SO only)	All network sites	Shipped! February 23, 2021

UI changes to all queues

All review queues will be adopting a new user interface that focuses on providing reviewers with context to make deliberate, thoughtful decisions on review tasks. We shared an initial design proposal in our product discovery phase and made sure to include that feedback into the final design.

Responsive/Mobile-friendly design

We utilized Stacks to update the visual design in the review queues. As part of this update, we were also able to make the queues responsive and utilize mobile-friendly UI components.

Changes

• Actions menu - We’ve removed the action buttons for a menu-style component. Selecting an action now requires a bit of pause and mitigates robo-reviewer behaviors we've seen in the past. Instructions are now always visible and associated with each action.

• More context - Users can now toggle between the review task and the necessary context (original question, answers, and comments) on the task page to assist in making educated decisions on a task. No need to open new browser tabs!

• Vote controls and follow feature - We heard from you that being able to vote directly on all tasks as well as the option to follow posts and receive updates on changes would help in the reviewing process. These features will be available in all queues.

• Filter button - We’re highlighting this feature (where it’s available) with a button and replacing the easily missed text-link.

• Review count - We’ve removed the badge progress bar in lieu of keeping track of how many tasks you can complete in a queue (don’t worry, the badge bar isn’t gone).

Review home page

The review homepage has received a Stacks update as well, setting it up for forthcoming plans to create better onboarding for new reviewers. This page now provides a bit more information to new reviewers.

Changes

• Only the queues available/earned by the user are clickable

• Includes additional text about necessary reputation to earn access to new queues

• Available queues are prioritized to the top and then sorted by the number of tasks in each queue

Additional updates

All aspects of the Review queues are receiving some much needed design attention. Here a few more changes you can expect to see:

• There is more incentive to continue reviewing. The Steward badge can now be earned multiple times for every 1,000 reviews in each queue. If you have already completed 2,000 or more reviews, we will backfill any additional Steward badges that you have already earned. You can expect to see them accrue over the next few days.

• History view - Added and rearranged a few UI components so that users can better navigate and utilize the page

• Stats view - Updated badge progress bar

Feedback

Please leave any feedback or bug reports related to this release below this post. We will be reviewing responses on this post for one month following Phase 3 (March 23, 2021). After March 23rd, please ask a new question to report any new issues.

Answer 1

feature-requeststatus-completed

In the old review queues, when you visit a completed review, you can just click Next to proceed. Now you have to check the radio button and click 'Submit'. Can this be just a single click please, just like 'Skip' also is in the new queue? My touchpad will thank you for that.

Answer 2

status-completed

Not sure if it’s a bug or deliberate change, but something that has been tripping me over and over while viewing the history is that the placement of the editor’s user card looks more prominent and can be mistaken as the OP.

P.S. Well, I think I have to get accustomed to now... (cue XKCD #1172)

Answer 3

bug status-completed

The Steward badge can now be earned multiple times for every 1,000 reviews in each queue.

The badge descriptions do not reflect this and need updating

Answer 4

Immediate reactions

(no posts in the queues, so going off screenshots):

• Love seeing the presentation of posts being brought back into line with question pages. In particular, the stats at the top are a lot harder to miss than the sidebar on the old UI.

• Love the responsive design - phones are an ideal interface for review, as it involves lots of small tasks; perfect for standing in line at the grocery store.

• I personally hate dropdowns for actions: they default to hiding what is available. On mobile they have the additional issue of being modal: I can't both select an action and scroll through what I'm acting on. And I use an Android phone, where they're actually kinda usable; from what I've seen, this will be a real problem on iOS devices, where they present a weird scrollable UI seemingly inspired by "The Big Wheel" from the old Price is Right gameshow.

  

  Remember: the goal here is for folks to make informed decisions based on a careful examination of the post being reviewed - anything that separates the action from the post risks working against that.

• I'm a little bit concerned about the position of the task description itself: putting it between the post stats and the post seems... Odd? It actually took me a minute to find it.

Other thoughts

The make-or-break for this is gonna be how folks actually use it. Voting in particular is potentially problematic: when we first tried that, it resulted in a MASSIVE skew for voting on Stack Overflow.

I'm happy to see that you intend to test this on Ask Ubuntu and Super User - they get enough traffic to potentially flush out some issues. I'm unclear on whether you're rolling out the phase 1 changes there before you do on Stack Overflow.

Regardless, it will be critical that you monitor behavior after this goes live. Not just how many reviews are done, but the breakdown of their disposition and side-effects. For example, if more First Posts are upvoted and later closed, there's something weird happening; if more posts going through the close queue are being downvoted and answered, there's an issue. Etc.

Looking forward to trying this out on my phone once it's live on a real site!

Answer 5

bug status-completed

When trying out the filter in the Close Votes review queue on Meta Stack Exchange, the system suggests me to filter on javascript or java. The queue is empty right now, but I'm pretty sure it will remain empty forever if I filter on that. Please use the current site's top tags or none at all.

Answer 6

bug status-completed

Could rollbacks in the edit summary show the revision number and not the raw text of the revision, as shown here (from the Reopen Votes queue):

So it looks like this:

That would be much more helpful than a bunch of characters.

Answer 7

bug status-completed

For duplicate closure, it seems the link to the duplicate is not shown on the completed Close Vote review item now. E.g. a random completed review which the question is currently a dupe of The Complete Rate-Limiting Guide

Compare to the current Close Vote review on other sites

Answer 8

bug status-completed

I suppose the "All reviews" should show all the reviews in that queue (privilege for >10k).

However, clicking it just reloads the page appending a ?skipped=false (if it doesn't exist) parameter. That button should be either removed or disabled for <10k.

Answer 9

bug status-completed

The revision comment is truncated. For example on this question (review) the revision comment is:

Active reading [http://stackoverflow.com/legal/trademark-guidance (the last section) ].

But what is actually shown is this:

Answer 10

feature-request status-declined

Make better use of the screen real-estate.

My favourite view to review in is the side by side view, usually the markdown version of it. As it currently stands the site is served with a max-width instead of being totally responsive. This is already painful on some long posts without many paragraphs, but in this review view it is even worse.

Take a look at this screenshot, of the entire width of my monitor:

The screen is divided in about 5 equal parts, of which only 2 out of the 5 carry meaningful content, namely the side by side diffs. When reviewing long posts this becomes extra painfull, as then half of the screen will be empty (The first and last column, and half of the column that carries the actions box):

Can we move the actions box back on top, so I can use at least 3/5th of my screen to view the content that is actually of interest.

Answer 11

feature-request status-completed

When hovering over the date the question was asked, I'd expect a tooltip with the exact date (like almost everywhere else you see a relative timestamp) but not in the new review queue UI:

Answer 12

bug status-completed

There's a small UI bug I noticed in Close Votes on SO. Below "Not suitable for this site", it says "subreason.InputTitle"; I assume that's not what it's supposed to say. This is only noticable with any of the "A community-specific reason" close reasons.

This bug is also visible on Math SE, and likely all other sites with community-specific close reasons.

(Thanks to KReiser for screenshot and confirmation this also affects Math.SE)

Answer 13

status-completed

This is implemented. If community-specific close reason votes exist on the item, they will be shown indented underneath the main close reason. These will always be shown (departure from current functionality where it is only shown if you click on a more button). - Yaakov

Not possible to see which community-specific close reason a question was flagged for in the close queue, if the question has close flags but no votes

In the old design of the review queue, if a question was flagged as Not suitable for this site (i.e. "off-topic", "A community-specific reason", etc.), it was possible to click the "more" link to expand the reason and see which specific sub-reasons under that umbrella the question had been placed into review for. This list included both sub-reasons from users with the close voting privilege casting actual close votes and those chosen by users who don't have enough rep to vote to close casting a flag to close:

^{(credit to Ryan M for the screenshot)}

However, in the new review interface, there doesn't seem to be a way to see the specific reason(s) which have been chosen by users without enough rep to cast close votes casting flags:

There is another way to view sub-reasons others have selected, by opening the closing dialog, but that only includes sub-reasons chosen by voters, not by flaggers, so this means that if there are only flags and no votes the dialog will be completely empty, and even if someone voted, there's no way to see if a flagger chose a different sub-reason from the voter(s).

Can a way to see which sub-reasons under Not suitable for this site have been selected including by recommend closure flaggers please be added to the close queue?

Answer 14

feature-request discussion status-completed

I've had a chance to play with the Close Vote Review Queue a bit now on Stack Overflow where the overwhelming bulk of my reviewing activity occurs. My main takeaway is that there are some good functionality improvements (yay for letting us up/down vote in the close vote queue!), the tradeoff is a very expensive one: screen space

I see a lot of (in my humble opinion) unnecessary/redundant content on the page, and the end result is that the actual content of the page is pushed down for me past the halfway point of the screen (reference: I have 24" monitors at 1920x1080 resolution):

(and I actually set the top bar to display: none for these screenshots, so imagine everything is even further down the screen).

As you can see by the arrows and horizontal lines, we have to track wayyy down the screen just to get to the point of queue; all the content outside of the question title and body is not going to change from item to item, and most of it is not stuff we need to see ever, so why are they taking up so much real estate?

Here's the same screenshot with number labels for a list of suggested changes below:

1. We don't need this link. If someone wants to go back to the questions page, they can click the existing Stack Overflow logo link about 50px above this and go back to the home page, which is /questions for anyone who is logged in. Likewise we already have the top menu option for review queues available if we want to go to the All Review Queues page. One extra click is not worth saving if the cost is 50px of valuable vertical screen space.

2. Same as above; we know what the close vote review queue is, and it's amply explained by other sections of the page as well... the flag/close banner, the URL, the 'review' UI, the 'Action' menu items on the right, etc.. I'd rather see users taken to the privilege page or a FAQ page the first time they click on the specific review queue than have this take up yet another 50-75px of screen space, if anything.

3. Why is this on its own line? Especially if you expand the size of the container to not be so small (see item #6) there should be plenty of space to move this up to the line above.
   

   3a. (Not pictured) Why is the "Filter" menu not a modal that appears outside the document flow? As it is, it pushes the page down unnecessarily when it's open. It's a small thing, but just feels like an obvious 'miss' in the execution here.

4. Not convinced we need this information... at least, we definitely don't need the question score, because it's shown on the very next line where I can actually vote on it/click to break out the full up/down vote information. I suspect that this entire row was just copied from the /questions page due to that being the most portable location for getting that info. All we need to see in this line is the question title. There is some use to seeing the number of answers (mostly in being able to see and vote on said answers from the queue)... so that bit I'm still on the fence about. The OP name and rep and ask date belongs at the bottom right of the question, but that's a separate bug/request here already being tracked/reviewed.

5. Again, we don't really need this info/option, at least not for the Close Vote queue; what answers a question has, or whether it has any in the first place, are irrelevant to whether the question is close-worthy. The only exception I can think of here is in the case of deciding which question should be duped as a target of another, but that's a case where you'll want to be looking at each question on the /questions/ page anyway rather than in the queue.

6. The width of the main container element is 1100px. But with the sidebar force-hidden by the new layout, we could easily expand this up to 1200px to reclaim some wasted gutter space and show a more "normal"-width container. I'm sure this change would be great across all review queues, but especially so for the suggested edit queue, where diffs are shown side-by-side.

7. (Not pictured) I also removed the "Review the following question" header element; again it's obvious we need to review this question, because we're in the review queue and this is the question on the page...

Taking all those things into consideration, I made a quick mockup with the Stylus add to demonstrate what this would look like (again, I'm focused on the Close Vote review queue here):

(NB: I have since re-added a grey HR/border between the question title and question body, but it isn't visible in this screenshot. Imagine there is one):

If you compare the 1st or 2nd image in a tab vs this 3rd one in a separate tab and swap back and forth, you can see there's a lot more space available in my modified version for the actual meat & potatoes of the page: the question being reviewed.

And just for those who might want to take that layout and run with it in their own user styles, here's the code I have that makes these changes:

div.container { /* making the page container a more reasonable width. */
    max-width: 1200px !important;
}
.s-page-title--text > nav { /* hide the nav menu at the top.  */
    display: none;
}
.s-page-title--text  p.s-page-title--description { /* hide the description of the vote queue. */
    display: none;
}
.s-page-title + div { /* set parent element to relative */
    position: relative;
}
.s-page-title + div > div.grid--cell { /* move the filter options to the top */
    position: absolute;
    top: -68px;
    left: 250px;
}
div.js-review-task div.js-review-content h2:not(.popup-title-container) { /* hides instructions on what this page is for. */
    display: none;
}
#js-review-filter-id { /* set the filter option menu to be a modal instead of pushing the screen down */
    position: absolute !important;
    top: 0px;
}
article.s-post-summary > ul { /* removing meta information */
    display: none;
}
article.s-post-summary { /* fixing row size due to grid properties */
    grid-template-columns: auto;
}
div.js-review-task > div > div > div:first-of-type + div:not(#panel-question) { /* hides the answers toggle menu option; unnecessary for the CV Queue */
    display: none;
}
div.js-review-task > div aside.js-review-instructions ~ div.grid--cell {
    padding-bottom: 5px;
    border-bottom: 2px solid #4a4e51;
}
div.post-signature { /* moves 'edited by' usercard to bottom-center of question instead of bottom-right to reduce confusion */
    flex: 3;
}
aside.js-review-instructions div.s-notice p { /* removing all the extra space for telling me I passed an audit item, part 1 */
    display: inline-block;
    margin: 0px;
}
aside.js-review-instructions div.s-notice button {  /* removing all the extra space for telling me I passed an audit item, part 2 */
    margin-top: 0 !important;
}

Answer 15

feature-request status-completed

In Reopen votes queue actions are not ordered in logical way.

"Reopen" and "Edit and Reopen" actions should be grouped together. "Leave closed" should be first action on top or last action on the bottom.

Current layout is rather confusing.

Answer 16

bug status-completed

On screens with widths less than 640px (e.g. my iPhone 11) the daily stats are hidden:

There seems to be enough space; this is how it looks like a few pixels wider:

Answer 17

feature-request status-declined

Increase the max-width of the review screen to include the width of the left-nav bar

Here is a screenshot of both the regular site, and the review page at full width, 100% zoom. As you can see the screen real-estate used for content of the review page is smaller. Both the review-page and the contentblock (including the right sidebar) of the mainpage have a max width set of 1100 px.

Seeing there is no left-nav on review screens can we please increase the width of the review screen with the width of the left-nav.

Answer 18

feature-request status-completed

We have come full circle on this. The notice has been moved above the question metadata, and the tab buttons (Task/Answer/Revision, etc) are now below the question metadata, directly above the area that the affect.

Based just on this UI mockup in the question, the box at the top with the "This question has been flagged as" message seems out-of-place:

Wouldn't it be better to swap that box and the post title? Right now, the post looks disjointed, with the post title and body/details separate from each other.

Can it be like this instead?

In the previous mockup in the post "Improving Review Queues - Design overview I: Onboarding and updating workflows, that's how it was shown. Was there are reason for the change in this current mockup? I think swapping it back would result in a more coherent read: "This is your review task, this is why this post is in the queue (because it was flagged as reason), then here is the post title and body/description."

Furthermore, after I mocked-up this swapping, it now looks like the vote counter is redundant, and could possibly be simplified into:

Answer 19

status-completed

With the new UI / visual design, when you go to a review page and there are queues that you aren’t allowed to access yet, it says “Awarded at 3k” (or whatever number is appropriate), with the trophy icon (the same one that’s used for the “Recent Achievements” dropdown):

     

This seems to be very peculiar and inappropriate language.  Nothing gets “awarded” when you reach and exceed the threshold for admission to a review queue.  The old wording, “You need at least 3k reputation to review close votes.”,

     

was perfectly fine.

---

P.S. In the announcement (i.e., this “question”), you say

Changes

• Only the queues available/earned by the user are clickable
• Includes additional text about necessary reputation to earn access to new queues
• Available queues are prioritized to the top and then sorted by the number of tasks in each queue

The first and third bullets are not changes — it’s always been that way (at least as far as I can remember).  And the second one, as shown above, seems to be outright false; the new design has less text about reputation necessary to ‘earn’ access to queues than the current design.

Answer 20

feature-request status-completed

Can we have the View Count show a shortened version (e.g., 190K), instead of all these numbers:

And then have a tooltip to show the exact number, to be more consistent with the look on the actual post? It’ll look better for posts with 1M+ views.

Answer 21

bug status-completed

On narrow screens, the user names and their review counts on the review stats (e.g. on Meta Stack Overflow) occupy the same space:

Answer 22

feature-request status-completed

The 'Next' option and the 'Edit' option have rather useless tooltips. The other options (at least in the Close Votes review queue) don't. Can they be removed?

Answer 23

feature-request status-completed

Let us vote on answers as well (they need some love, too)! Here's how an answer looks like from the "Answers" tab:

Answer 24

bug status-completed

In the Suggested Edits queue, when looking at a review task, the link to the post will always route to the parent question even if the suggested edit applies to an answer on that question.

As an example, on this review, clicking this text:

Takes me to the parent question instead of taking me to where I expect it should take me, the answer that the suggested edit was proposed on.

Could this link be changed to point to the answer instead?

Answer 25

This may not pertain to the intent of the current review queue makeovers. But I have been longing to see some functionality to enable a reviewer to directly link to a moderator, particularly in the suggested edit review queue. (What I am about to request, can be disabled for audit reviews).

I review primarily on [math.se], and occasionally on [matheducators.se]. I have encountered efforts to "edit a post" to threaten the OP of the post, including vandalism and outright hostile name-calling and threats, particularly offered from "anonymous user". Also, I notice a username repeatedly making exceptionally trivial edits. It would greatly simplify matters if there were a mechanism, within the review, to flag for moderator attention.

Currently, to contact a mod, I have to flag the post on which the attempted edit was made, and in my flag description, copy the review (requiring me to go to review history to get the link) into the description, etc, often multiple times for ongoing problems.

I very much like the changes thus far described, and do not want to ask too much of developers now, but I would be ecstatic if such a "flag moderator" functionality were added to the suggested edit review queue.

Answer 26

feature-request status-declined

Allow opt-in for sites that haven't rolled out the new UI to everyone

I'd love to be able to help test this out in the earlier phases, but there are virtually never any questions in the queue on MSO and MSE, and I don't have 3k rep on Ask Ubuntu or Super User.

Could it be possible for users on other sites to opt into the new UI (similar to the opt-in for the new editor) before their site is converted over for everyone?

Answer 27

bug review status-completed

(Before I was aware of this announcement post, I already posted this as a meta bug question here: Submit button not working anymore in new review queue layout)

I encountered a bug in the Triage review queue. I can click the Submit button when I select one of the options "Looks OK" or "Needs community edit" (which do not have suboptions) but not if I select "Needs author edit". In the past, a dialog opened in which I was asked to specify the detailed reason (e.g. "Needs details or clarity" or "Needs more focus"), but now, I am stuck at this point. So either the dialog is not displayed or it is located somewhere off-screen, I guess.

The only thing that happens is that the Submit button is enabled, but when I click it, nothing happens (both in Firefox on a Linux desktop PC as well as on Chrome on Android).

Some other users apparently do not encounter this behavior and have speculated that it might have something to do with their privileges (see comments under the linked question above).

Edit: Before, I reported that the same behavior occurred when selecting the "Flag" option. After double checking, I found that this is in fact not correct. When selecting "Flag", the dialog to specify the type of flag correctly appears after clicking Submit. So, the problem only occurs for the option "Needs author edit".

Answer 28

feature-request status-review

In the suggested edits queue, the username of the person who proposed the edit is written with tiny letters. Their reputation is also hidden.

This is some important piece of information and you should give it more emphasis by placing it somewhere in the top - perhaps here?

(after renaming "the following question" to "the following edit" - bug)

Answer 29

feature-request status-deferred

I think this would be the perfect opportunity to enhance the automated "review testing" system by adding a "minor" new button.

Right now, when the system thinks you made the wrong choice, it tells you that you messed up, and to do better in the future. Then there is a single button I understand that you have to click to continue on the review queue.

I suggest to add a button:

I understand, but object the automated decision (wording to be improved), with a text field to provide a comment why that is.

Rationale: we all know that the automated system is far from perfect, and it is really annoying when it presents you questions that it got wrong. When that happens, it frustrates the reviewers, and worse, it leaves a "stain" on their resume. The later might matter or not, but when it matters, it is on the reviewer to remember when the system messed up to be able to "prove innocence" (and when you really do A LOT of reviews, that is a real problem).

So, giving the reviewer an opportunity to object would enable:

• the system to "drop" bad test questions faster
• the system to keep track of objects, so in case there is ever a need for a human moderator to check the review history, those objections are more prominent

Addendum: the aspect of "this is a bad test question" might even be worth a button when the user passes the test. Happened to me repeatedly, too: to figure "this is a test question, and I should react that way, although the specific question at hand actually should have been treated in another way".

Answer 30

feature-request status-completed

When reviewing answers in the First Posts review queue, sometimes there's a tab 'Other answers (x)'. When you click it, the tab heading is repeated, while it's not for the other tabs ('Answer' and 'Question'). I don't think it's necessary and it's eating away a bit of screen real estate. Can it be removed please?