New Feature: Community Review Tasks - Now in Beta

Question

Please post new bug reports, feature requests or discussions as individual questions tagged with at least review in addition to the requisite

And thanks for all the great feedback so far!

---

We are currently soliciting feedback for an improved version of /review to help with the very important task of maintaining site quality. There are three targeted tasks we have built thus far and will continue to roll out others after we gain confidence these three are working:

• Low Quality Posts: This task targets posts that we've algorithmically determined to be low quality. Your job is to read these posts and:

  • decide if a post does in fact Look Good
  • Edit the post to make it better
  • Delete¹ the post entirely
  • or skip it if you are Not Sure.

  The minimum reputation to perform this task corresponds with the ability to edit any post (currently 2,000).

• Suggested Edits: This task targets posts edited by users who have not yet earned the reputation to fully edit. Your job is to read these edits and:

  • decide to Approve the edit as is
  • Reject the edit
  • Edit the original post and make your own edit
  • Delete the post (moderator only)
  • or skip it if you are Not Sure.

  The minimum reputation to perform this task corresponds with the ability to edit any post (currently 2,000).

• Close Votes: This task targets posts that have received at least one close vote. These posts generally need additional review to help decide if a post does in fact require closure. Your job is to read these posts and:

  • decide to Close this post by casting another close vote
  • Do Not Close² this post if do not agree with the reasons for closing
  • Edit this post to make it better
  • or skip it if you are Not Sure.

  The minimum reputation to perform this task corresponds with the ability to cast close votes (currently 3,000).

Each of the actions on the review task listed above may not always be available based on your current reputation or moderator status. We will be reviewing the data collected in this experiment closely to determine if these new review tasks are working and also reserve the right to pull these tasks out entirely if we find they are not. We have enabled these review tasks on Meta, Super User, Server Fault, and Stack Overflow.

To check out the new hotness visit: https://meta.stackoverflow.com/review

^{1 If you don't have enough rep to delete, you can "Recommend Deletion". This doesn't do anything right now except move it to the front of the queue for people who can delete it.
2 "Do Not Close" just removes it from the queue right now -- it doesn't affect close votes.}

Answer 1

When you did your 50 daily "Close Votes", the link is "blacked out" and a hint is shown: "You have no more close votes today; come back in 3 hours.".

But if you did your 50 reviews of "Suggested Edits", there is no such message. Not before you click "approve" you will get the "Out of votes - you may only vote 50 times a day." notice. (Especially annoying as there is no notion when to will have some more votes again.)

Answer 2

For the 'close votes' tab, it currently asks the following question:

Should this question be closed as [insert close reason here]?

And then has buttons:

• Close
• Do not Close

Clicking 'Close' opens up the close dialog and invites me to choose a reason. It would be faster for reviewing if clicking 'Close' automatically cast a vote for the reason given, that would save having to open up the dialog, selecting an option from the dropdown menu, and then voting to close again. I feel like this slows down my reviewing stride when I'm reviewing a lot of questions.

Perhaps this could be made quicker with another button, something like this:

Should this question be closed as [insert close reason here]?

• Yes (automatically votes for reason in question)
• Close for Other Reason (opens close dialog)
• Do not Close

Answer 3

I was trying the close votes review and noticed that I could be more effective if I could limit the selection to tags I am familiar with.

Its very hard to to judge duplicates and other stuff correctly if the language/tag is one you are not familiar with.

But it is easy to use and I really like it :)

Answer 4

• When improving an edit I'd like to be able to easily see what the question looked like before the suggested edit - helpful when improving borderline edits. Even just a link to the original post would be good, but it disappears the moment you hit improve. As far as I can see the only way to make the link to the original edit appear is to cancel the improvement.

• When reviewing edits more feedback on what the other reviewers have decided would be good, especially if they've entered a custom message.

Answer 5

Two improvements that I could see going a long way towards improving editing accuracy:

• Showing if a question/answer is Community Wiki in Suggested Edits
• Allowing editors to see comments made on questions (in Suggested Edits only).

The former allows one to take a double take on whether or not the edit improves the question as a whole, and can relax most fears about the edit being vandalism.

The latter allows for editors to indicate if an edit was made in direct response to remarks in the comments. I may be hesitant to spring for this, as the OP should update their question, but it can help in certain cases.

Answer 6

The new process seems promising, but I'm concerned it implicitly moves the goal posts for deleting Low Quality Answers.

Under Jeff Atwood's year-old dictum, deletion is for very low quality answers, unlikely to be salvageable, embarrassingly bad stuff.

The new process encourages reviewers to click Recommend to Delete if the answer under review would require substantial editing to fix (retyping > 20% of the answer and significantly changing its character) while a competing answer has already gotten it right, one way or another.

This might be intentional and/or a good thing. I'm just saying it's a change in criteria, and possibly an unintended consequence. The issue may deserve an airing in its own thread, but I wanted to throw it out here as it's early days.

Answer 7

In /Close Votes, I'd suggest to add the following functionality to the Do Not Close button: if the question gets closed, cast a reopen vote automatically.

Answer 8

The Suggested Edits page does not provide any way to get the context. For example I was reviewing a Suggested Edit that added a change to run a command in the original answer as root. To undertand whether that was warranted required reading the original question.

When reviewing Vote to Close questions, you can just click the question to get the context. This is absent in the Suggested Edits page.

I had to click the user, then activity, then find the question the edit was made to so that I could read the question and answer in context.

Answer 9

What little I can see looks cool now, but it would be nice if some other folks (for example, 40k users on SO who only have 1.6k rep here) could see it too... :)

Maybe I'm dense but I can't figure out what the row of gravatars after each of the tools represents?

Answer 10

I'm facing an issue related with the rejection of tag-wiki's:

More than once that I've came across and rejected the same tag wiki.

Seems like my previous rejection wasn't registered or isn't being recognized as "already rejected by me" when prepared for presentation.

Ps:
Nice work on this new review system, I'm an even more happy user :)

Answer 11

Can we please have a feature to Improve wiki tag edits? (clicking more doesn't add the improve option).

For example, in the edit presented below, the spacing and punctuation aren't very good, but the content is sound, however I wouldn't approve it till it is improved, at the same time I don't want to reject it as its not really invalid:

Also, I think it would also help if under the sections for rejection there was a "Edit doesn't match tag" option, specific to wiki tag edits, as I've seen quite a few edits that don't match (either cause they really don't match or there is a misunderstanding of the tag), and typing a custom reason each time is a bit laborious, but at the same time I don't feel that it falls under "Invalid Edit".

Answer 12

Could you please rename "Improve" → "Edit" in the suggested edits? "Improve" and "Approve" are just too similar words.

Just now I approved an edit that I wanted to improve … and mark it as not helpful.

Answer 13

While I'm not 100% positive it is a result of this new feature, as one who does not have the authority to edit posts directly, I have noticed a significant decrease in the time it takes for my suggestions to be reviewed. Generally within a few minutes, my suggestions are accepted, whereas a few weeks ago, it could have taken a few hours.

It also appears that the suggested edit queue has been kept very low, which also helps me, since I find it very frustrating to look at a horrible, needs-to-be-edited-now post, but I can't submit an edit because there are too many pending suggestions.

From my perspective, the new system works great!

Answer 14

Does the new system take into account favourite and ignored tags as say the interesting tab does.
The reason I ask is that when reviewing I will often ignore items that are not in my skills as I do not know enough and having to click on these will slow things down and put me off reviewing which is not what the change is meant to do.

If we do go to the new interface could we sort the items giving the favourites first , then other interesting.

Answer 15

When relative times are used elsewhere, the full datetime is included as a title attribute. This isn't the case for the times in the recent review list, or the "asked"/"answered"/"actives" times beside posts. I'd like it to be included.

Answer 16

If you take action on a suggested edit which has already been approved/ rejected, you get a popup saying (along the lines of):

This suggested edit has already been approved. Please visit the post and ...

However, after this, you are still sat on the same edit, albeit with the button you clicked removed.

What I would expect to happen is the controls being replaced by "Next" and that taking you to the next edit.

Answer 17

Prioritize flags

It would be nice to have if there was a way to filter out auto generated flags or some other way to prioritize flagged things. For example Low Quality Posts section has many posts auto-flagged incorrectly. This makes it a bit waste of time to go through all of them. The "contribute something useful" to "spent time" ratio is very low IMO on these.

I guess user flags are more likely to be accurate than auto flags so it would be nice if they are somehow prioritized and handled immediately.

Another idea is to also prioritize recently flagged posts. I guess in most cases it is much more valuable to immediately handle a recent flag rather then spend time on an 1 month old question.

Also auto-flager could be improved. One idea that comes to my mind is that when a question has many answers that are flagged because of length & content, those answers could be excluded from auto-flag?

An example of legit answer with many other legit(but flagged) answers:

Answer 18

It would be great if an option can be provided to filter the answers/questions for review using specified tags.

I cannot see any more progress towards my gold badges, like Copy Editor, Marshal, Proofreader, etc. It will be handy to have one.

Also in the new review system a reviewer cannot upvote/downvote the answer being reviewed (and flagging).

It seems that classification of flagged questions is very confusing. I.e. the classification should be split into groups according to actions to be taken of the question: delete (not a real question, spam), close, move (not belongs here). 3 options should be sufficient.

Answer 19

One task I'd like to be able to do is look for old questions and answers that are under review. It was difficult under the old system, but impossible under the new. While not everyone is interested in this sort of drudgery, this clean-up can be quite helpful, given the number of old duplicates and poorly written questions that crop up when trying to search the site.

Some have requested a feature to limit reviews to certain tags. My suggestion would be for an expanded version of this, with various filters that can be applied to reviews: tags, dates, the "Close Votes" and "Close Reasons" from the old review system, flags (user- & auto-flags).

Answer 20

Does editing from the close vote review page have to take the user away from the question once the edit is submitted? Sometimes I want to both edit and give my opinion on whether the question should be closed.

Answer 21

I would like a little addition to "looks good" I have reviewed a few answers where the quality of the discussion on the answer was high enough for me to make it a keeper, even though the answer was perhaps voted down. Perhaps an "answer is good" vs "answer and comments are important"

Answer 22

Should items that have already been edited show up in this list? I have seen both items that have been edited by >2k users (or edited by <2k and approved) in these lists. I also have seen items that already have an edit pending.

Shouldnt these items be filtered out so we are not reviewing the already edited?

Answer 23

Comment on review for low quality posts.

This applies to low quality answers only.

As noted in other questions we need to be able to vote on these.

We need to be able to see other answers as the action differs depending on these. If there is another good answer then the shown answer should be deleted. If there are no other good answers then it has to be improved.

Re recommend deletion - under the old system the only way to get answers deleted was to flag for a moderator but flags get "declined - flags should not be used to indicate technical inaccuracies, or an altogether wrong answer" so this is a totally new way of doing things. (although I would also note that according to this Q&A the mods might not be correct or consistent here) Thus I think the recommend for deletion really needs a full explanation.

Answer 24

Here's a bug:

When voting to delete an answer with a 0-score, in that case, this one, it says that I voted to delete it in the history.

… but it doesn't show on the actual post:

The delete (1) link is missing here.

Answer 25

The algorithm under the 'Low Quality Posts' retrieve too many items of the SQL section in my opinion.

SQL question are generally answered by a few number of characters representing the query, with little comments. All the answers are good, maybe they need more explanation, but that's it.

Maybe a filter by tag for this particular case should be an improvement.

Or mix the length of the post with the reputation of the poster. A 20 characters answer of a 2000+ rep user must be better than a 50 characters answer of a 10 rep user.

Answer 26

Is there any chance of going back to the review list upon saving an edit?

When a <2k user clicks on "More" under suggested edits why not show the pending edits without the option to interact? This would show what your <2k peers are editing and how (learn to edit by example).

Answer 27

For the Suggested Edits tab, I don't have the 5K rep needed to access the Suggested Edits queue on my main site (SuperUser). I just see it as always 0 suggested edits needed. I do have more than 2K rep on that site, so the link does show up for me, but why is it there if I can't access the queue?

Answer 28

Completely impractical. Very poor workflow design. Presenting reviews one at a time is just too slow a process. If there really are 58.1k 'close' votes pending, nobody is ever going to get to them all.

You need to present 20/50/100 to a page so people can pick their areas of expertise, and probably let people filter them based on tags etc, in fact let them use the existing search function.

And the vote-to-close process needs to be streamlined. For example if you agree with a prior 'exact duplicate' you shouldn't have to go through all the same rigmarole with about 5 mouse clicks.

I would also reconsider the wording of 'not sure'. I am often extremely sure that I don't have a view about specific reviews. How about 'don't care' or just 'next'? But with the change above the issue becomes moot.