What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 130 Stack Exchange communities.

I am concerned on the growing close votes review queue size. I know it has been discussed before.. more than once, and I simply want to propose some ideas...

Why not separate the single close votes queue into a few different close queues, or a custom filter within that review queue, like being able to choose which types of close reasons to view:

Based on the method of closing:

  • Flagged (no votes, only flags)
  • Fresh (one or two votes)
  • Vintage (three or four votes)

Based on the time:

  • New (added to the queue within the last 24 hours)
  • Young (added to the queue within the last four days)
  • None/Normal (between Young and Old)
  • Old (added over two weeks ago)
  • Ancient (added over a month ago)

This idea would let users more easily decide where to spend their votes, and also in my opinion help people start tackling the problem, the huge number which is now 56k, is daunting. This idea would work better if that number could be split into smaller segments.

Additionally this would help to filter out those questions which are stagnating and old, and help to improve response time on closing newer questions which are possibly still on the feed and being looked at by users.

I do not have a monopoly on ideas, and since this is tagged as a discussion, It would be great to have other people give their opinions as well.

share|improve this question

closed as off-topic by gnat, Martijn Pieters, Undo, Emrakul, animuson Apr 29 at 23:05

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "This question pertains only to a specific site in the Stack Exchange Network. Questions on Meta Stack Exchange should pertain to our network or software that drives it as a whole, within the guidelines defined in the help center. You should ask this question on the meta site where your concern originated." – gnat, Martijn Pieters, Undo, Emrakul, animuson
If this question can be reworded to fit the rules in the help center, please edit the question.

7  
A couple of notes: 1. I've noticed the queue growing abnormally rapidly in the past week or so, since changes were made to close reasons. It seems like a lot of people are trying out the new feature. 2. You can already filter on close reason and tags, which I think are more meaningful than the age of the close vote or question. –  Bill the Lizard Jul 1 '13 at 13:29
3  
I personally like the time idea, it would help a lot on Physics. –  Manishearth Jul 1 '13 at 13:32
    
I am honored by the (so far) 9 upvotes, But I am not asking this question to get reputation - I am asking it to get opinions, ideas, and responses.. Only if the community takes action will anything change. –  Inbar Rose Jul 1 '13 at 14:07
    
Note that, by default, newer items will have precedence in the queue over older items. I don't see a real reason to want to filter out new items and only vote on older ones, so that doesn't seem useful. –  Servy Jul 1 '13 at 14:58
    
@Servy The proposition is not just the filtering - but also being able to see how many you have filtered down out of the whole... I know it seems purely cosmetic, but it has an impact... :) –  Inbar Rose Jul 1 '13 at 15:01
    
@InbarRose I don't think people don't use the queue just because it has a lot of items in it. Having lots of items makes it more attractive to reviewers, not less attractive. From my experience, the primary reason less people use that queue is because reviewing items in that queue takes more work than any other. –  Servy Jul 1 '13 at 15:03
    
@Servy And if you could filter them down to just the ones you want to work with, you would feel happier to do them... like "I want to close questions that are possibly duplicates, that have 3 or 4 votes (because I want to be sure its not just some random suggestion) that happened within the last few days" for instance.... would be much better than just "any possible duplicate" –  Inbar Rose Jul 1 '13 at 15:06
3  
@InbarRose You can filter on duplicates, and the queue is already set to give preference to newer questions. As to only showing questions with 3+ votes, I see potential for abuse there as people will only go for questions to which they can vote with the herd on. We don't want people doing that. If we did that we may as well just reduce the required votes to close to 3. –  Servy Jul 1 '13 at 15:40
    
@Servy I admit that had crossed my mind - but I have another idea about that as well - regardless, I still so no reason why even the current filtering system will at least provide a numerical value behind it. So that I could see how many close votes are for each close reason, and also how many exist in different tags - which tag has the most question in the queue after the filter? As for the ability to look through questions which have already gotten voted 3+ times, users with 10k tools can already see that... as well as many other things with their tools that could be useful in the queue. –  Inbar Rose Jul 1 '13 at 15:49
    
@InbarRose Users can see how many votes are on a question, but they can't easily filter questions to just those with 3+ votes, which seems to be enough. I'm not saying that nothing should possibly be changed about the queue, I'm just saying that the majority of what you're asking for is already there, and those few bits that aren't are bits that are unlikely to be helpful. –  Servy Jul 1 '13 at 15:51
1  
I would like to see the answers of a question (and to be able to upvote them) from within the review dialog. This would save a lot of clicks - and a lot of time - when reviewing. Now we have to open the question in a new tab, and this reduces the throughput. –  user000001 Jul 1 '13 at 18:05
    
This is going to be controversial, but why not lower the number of required vots to four? –  Emrakul Aug 3 '13 at 15:32
    
possible duplicate of Huge close votes review queue on Stack Overflow –  Flow Sep 15 '13 at 15:22
    
the problem has been (at last) acknowledged and addresed by SE team: Enough fuzzying: let's let everything into the close queue and age out questions that don't reach a threshold –  gnat Apr 29 at 22:02

8 Answers 8

Make the filter button more obvious.

Most of the time these questions come up, it is suggested that we add some kind of filtering feature. Yes, most of us on Meta know where this is, but that doesn't guarantee that most users know about it. I, for one, only realized it was there when it was pointed out to me; I'm sure there are plenty of similar stories.

The default response for Meta users is to say "there it is, have a nice day;" the fact that we must continually point this out is indicative that something is wrong.

Let's make the filter button more apparent.

share|improve this answer

Not sure if this is appropriate as an answer, but here is the reason why I don't do a lot of close vote reviews: expertise (or lack thereof).

It seems that the items in the review queue are random in terms of subject material. I can reasonably judge whether a question should be closed if it deals with anything in the Microsoft stack (C#, SQL Server, IIS, etc). However, I often get questions not related to my areas of expertise (jQuery, Git, PHP, Java, etc). These latter sets of questions are harder for me to understand, and therefore I usually just skip them; if there are a lot of these, I feel I am wasting my time and just stop reviewing questions.

One option that I think would help is if questions could be prioritized by my tags, whether they be earned tags, or tags that I follow. Perhaps questions that have my ignored tags would be placed at the bottom of my review queue.

share|improve this answer
15  
Look for the "filter" link at the top of the review page (right next to Review | Close Votes). Click on that and you can review close votes just in a tag you specify. Couple that with a specific close reason and reviews go a lot faster, with less skipping. –  Bill the Lizard Jul 1 '13 at 17:52
1  
Excellent, thanks! –  chue x Jul 1 '13 at 17:53

Currently the review queue is huge and seems quite opaque - we have no idea what is in there! The goal of this proposal is to make the queue more transparent, and to give the reviewer more control. My thought is that this proposal is not a replacement for the existing review queue, but rather something that complements it. I should note that both the existing review process and my proposed process can share the same underlying question close "queue" (the data storage).

The Proposal

The implementation for the proposal borrows from the existing site UI - it just adds another tab called "Close". I've taken a few ideas from the meta site (including this question) and incorporated them into this new tab. Since the tab deals with close questions, only users with greater than 3,000 points should be able to see it.


proposed-close-question-ui


Some notes for each of the red boxes in the above image:

A) As you can see, this is similar to how the existing tabs display their questions. What is displayed are all questions that have been nominated for closing. Note that the number of close votes for the question is displayed in the orange box to the left of the question.

Sorting would be based on the age of the first close vote of each question. Questions when nominated for closing are placed at the end of the queue, and are displayed on the last page. As additional questions are nominated, the existing ones are pushed towards the front of the queue and thus shown on the front page. The intent with this is to put aged questions on the front page, giving them attention.

As with the existing tabs, questions are highlighted or de-emphasized depending on the tags; also, the display will highlight questions based on the selected close reasons (right side of screen). Clicking on the question title would take the user to the existing question close review screen. From there, the user can vote to close the question or leave it open.

B) This is the number of questions currently nominated for closing.

C) Favorite tags - questions are highlighted based on these. Clicking a tag will cause only questions in that tag to be displayed.

D) Ignored tags - questions are de-emphasized based on these.

E) Close reason - questions are highlighted based on these. I'm sure a designer can come up with a better UI for this. I just lifted it out of the current review filter popup. Perhaps there can be a parallel to the tags - allowing users to have favorites and ignored close reasons.

F) This is the navigation (paging) for the close questions. This allows users to work on questions that were recently put on the queue (on the last page), on questions that are about to expire (on the front page), or any time in between.

Expiration

As the questions on the front page expire, they are taken out of the queue. Currently questions are taken out of the review queue depending on one of these reasons:

  • A question is closed by 5 voters
  • A question is viewed a certain number of times

I would like to propose a 3rd rule. I think questions should also be aged out of the queue (expired) based on time. This has been proposed by others as well, such as here. I think we should age out a question that has been in the queue for longer than 6 months (perhaps a year, exact time to be determined). It is unlikely that a question will be closed after this amount of time - why not just take it out of the queue? At the same time, the close votes from the expiring question should be removed.

Changes to the current review queue

The existing review process would not have to change much at all. The only thing I might suggest is to have the most recently nominated questions for close be reviewed first. (I am unsure, but I think this may already be happening.) This would focus the current review process on newer questions, allowing those questions to get an initial close review.

Benefits

  • The new tab makes the closed question queue more transparent - it allows people to focus on new questions, old questions, or any time in between.
  • It allows people to focus on certain tags or question close reasons
  • The front page of the "Close" tab puts the focus on those questions that are about to expire.
  • Questions are not in the queue for an infinite amount of time - this gets rid of questions that are likely to never be closed.
share|improve this answer
    
Wonderful idea. My only concern is that once you select a question from this wonderful tab, you mentioned that the user is taken to the same review page as normal, from there I assume that once the review is completed the user will remain within the normal review queue view. Which means that only the first review will be using your wonderful idea. I have no thoughts on how to resolve this issue. –  Inbar Rose Oct 13 '13 at 6:50
    
I suppose in that respect, it is like any other question tab. You have to open the review link in another window (or tab) if you want to keep the "close" tab open. –  chue x Oct 13 '13 at 18:38

+1 to separating by time

In the past I'd have loved see a time categorization because I never liked to close unclear questions from new users without giving them a little chance for correction. When I found a very recent question as tried to comment (if there was no comment already) but never a close vote.

With the new "put-on-hold-close-after-a-while" system I don't feel so bad holding recent questions, but it would still be a nice feature.

share|improve this answer
2  
Unclear questions are the questions for which it's most important to close right away, not questions that you should wait to close. –  Servy Jul 1 '13 at 18:53
    
But there is more to it than that. When a user posts a question, he is usually online and monitoring it for responses (for at least a few minutes to an hour) And can sometimes improve upon that, which happens often - first comment says something like "please add your stacktrace" or "show us the code" or whatever - silly things like that... but by the time the user does this, other users came and saw a horrible question and voted to close it. –  Inbar Rose Jul 2 '13 at 6:40

I originally posted this as a feature request, but it appears to be appropriate as an answer here.

I would like to be able to additionally filter/narrow down the off-topic close reasons.

At the moment we can filter by tag and by close reason, but off-topic has 7 sub-close-reasons

enter image description here

It may improve the workflow a bit to be able to narrow it down to one or two sub-close-reasons.

share|improve this answer

At the first revision, the following list is copied from another answer, and is here for reference. This is a better place for it, as it is a conglomeration of suggestions for the close vote queue.

I have made this answer community wiki; please add links as you see fit for other suggestions not already covered by this list.

(Additionally, while I don't have time to do this now, this list should probably be sorted by votes or something. I'll do it later if I have time, though.)


Let me choose when I use my close votes
Huge close votes review queue on Stack Overflow
Let close voting from the review page not count towards your daily close voting cap
Close votes daily limit increase
Is the close system working as expected?
How slow is the Close Votes queue being processed? And how large should it get daily when the backlog is gone?
Could we split up revision queues to get counts closer to zero?
Why is the close vote review queue so full?
getting things done: make the number posts to review more manageable for a single reviewer
Close votes policy review
How can the Close Vote review queue be improved?
Strongly separate duplicates from "the rest" in close votes review queue
Stack Overflow review tasks: close votes
Increase close votes in /review
Do we have a problem with follow-through on close votes?
How to make close-review queue less thankless
Should we raise the daily cap for close votes?
Close vote review queue reorganization proposal
Why are there so many close votes to review and so few of the others?
Stats of close votes review queue for SO?
Automatically close flagged questions What can be done about the amount of close votes on SO?
What can be done about the massive Close Votes queue on Stack Overflow?
Ideas for the close queue?
Allow 1k users to review close votes, but count them only as 0.25 actual close votes
Can we know the rates of adding and reviewing close votes?
Reduce close vote expiration restrictions
Would raising the review limit help with the Close Votes queue?
Suggestion: close votes review queue
From bull to bear - what can be done to decrease the mass of close votes?
Close Vote Review, additionally filter off-topic close reasons
Automatically filter the Close Votes review queue to active tags for newbies
Daily Close-Vote review queue
Close Vote Funnel and Metrics
Sort Order on Close Vote Queue
Daily close votes queue limit
Resort the Close Votes Review Queue
A proposal on how to influence the number of questions that are pending to be closed
Handling the close questions queue: weighted votes
New queue for moderators: questions that appear stuck in close votes review at SO
Proposed contest for close review queue posse
An alternative approach to the close queue
Give a broom to anyone who completes 10 close queue reviews successfully in a 24 hour period
Tweaking close requirements to manage the close queue
Fuzzy the number of questions in the close review queue, a dopamine for the shutterers
Get Community involved in the close process
How to handle ever-increasing number questions pending closure in the review section

share|improve this answer
2  
Maintaining two of these lists would be a bad idea. I suggest we just stick to the other one. –  Dukeling Nov 8 '13 at 14:38

The queue is definitely gaining momentum. 83.2k today.

It would be nice to tie the users with a gold badge from close votes into the solution. However, the gold Steward badge is for any type of review and Edit is the most awarded. As a result, 2 of these badges should be required to ensure one of the more difficult versions has been earned.

I suggest these changes, numbered by priority in my opinion

  1. Give users with 2 Gold Steward Badges a +1 close vote weight

  2. Remove the daily limit for close vote reviews

  3. Automatically filter by the user's top tag

(the user could still view all by removing the tag in the filter feature)

share|improve this answer
2  
I think that those who reach the gold badge for any review queue should be able to close question without a daily cap. Or alternatively, have the golden badge be awarded multiple times for every 1000 close votes. –  Inbar Rose Oct 9 '13 at 6:37

One thing I notice is that there are a lot of questions that seem to have maybe one close vote (often something like this question - a question that realistically is not great, due to the lack of effort on the OP's part, but for whatever reason the rest of the community of that tag don't mind it), thus ending up in the close queue. That question will probably hang around getting looked at very occasionally for a long time, but clearly the 'community' that is the tag was okay with the question (whether or not it may deserve to be closed from a site-wide point of view).

As such, perhaps we can expire questions from the close queue after X days (5,10, something long enough to get a good look or three but still pretty short) if they only get one vote? Or, alternately, have a proactive button to enter your disagreement with the close vote, and once there are several of those (3? 5? However many would remove it from the close queue normally) remove it from the queue and/or remove the close votes.

share|improve this answer
5  
Close votes do expire after a few days, provided that there are a minimum number of view on the post during that time (so that a vote isn't expired if nobody is viewing the question at all). You can also vote to leave open on the close review queue, and with enough votes the question will be removed from the queue. –  Servy Jul 2 '13 at 19:52
    
Good to know. I wonder if that minimum is too high for some tags, though - I mean, sas for example probably has 10 or so viewers of close vote level. Is that vote ever going to expire? The 'vote to leave open' is what I was suggesting making proactive - ie, not just on the 'queue' but as an option for anyone looking at the question. –  Joe Jul 2 '13 at 19:57
    
As you've said, there are only a few people with close vote privs. Do you want several low-rep people visiting a question to be a sign that the question clearly shouldn't be closed? Yes, the min view limit means that the votes simply won't expire on many low-visibility tags. Those questions need the close vote queue to get enough eyes from high rep users to make a decision. Removing them from the queue without there being enough attention just means we're ignoring questions that may very well need to be closed. This is exactly why the queue exists in the first place. –  Servy Jul 2 '13 at 19:59
    
That's why I was hoping for some sort of 'leave this open' option for non-queue folks; there's a big difference between 8 of us or whatever looking at the question and not caring about it enough to do anything, and 3 or 4 of us actually voting 'please leave open', isn't there? –  Joe Jul 2 '13 at 20:07
2  
That's a feature that's been requested a number of times. I've heard rumors that it's planned for implementation, but I don't have any official reference as to whether or not that's the case. –  Servy Jul 2 '13 at 20:08

Not the answer you're looking for? Browse other questions tagged .