What can be done about the massive Close Votes queue on Stack Overflow? [duplicate]

Question

There are (at the time of writing this; see edits at bottom of post) 54.2k questions with close votes on the review page on Stack Overflow. Unlike the other review tasks, which all usually return to 0 quickly, the number of questions with close votes continues to rise. According the this Meta Stack Overflow post, there were 51.8k close votes as of April 25 2013. In the previous five weeks there has been a deficit of about 400 votes.

Isn't there something that should be done about the close vote system?

As opposed to related question Why is the close vote review queue so full?, which was about why is the queue so full, this question is about the growing queue and what can be done about it.

Date                Questions in close vote queue

2012-08-12           55000
2012-08-19           54900
2013-03-03           49300
2013-04-25           51900
2013-06-12           54600
2013-06-14           54800
2013-07-04           57900
2013-07-06           58200
2013-07-11           59700
2013-07-19           61600
2013-08-02           66100
2013-08-20           70400
2013-08-30           73800
2013-09-02           74100
2013-09-10           76500
2013-09-18           79600
2013-09-19           80000
2013-09-30           82100
2013-10-02           82800
2013-10-24           86400
2013-11-12           91900
2013-11-13           92900
2013-11-14           93500
2013-11-27           96900
2013-12-03          100100
2013-12-04          100500
2013-12-12          105400
2013-12-13          106000
2013-12-17          102300
2013-12-18          100700
2013-12-20          100100
2014-01-09          103800
2014-01-10          104400
2014-01-13          104600
2014-02-03          113900
2014-02-25          121300

Using the above data:

Average daily increase in vote queue per day (as of 2013-12-03): 94

Average disregarding the two outliers (two data points from '12): 184

Answer 1

The review queue lengths express the community's collective opinion on the work asked from us.

Some queues are always near zero:

• First Posts
• Late Answers
• Low Quality Posts
• Suggested Edits
• Reopen Votes

This has to mean that the community thinks it is all right to chip in a little work to help reviewing these things. The workload is obviously acceptable, and so are the various conditions for each queue: Some only require a single review, others more, and the work required is reasonable, ranging from a click (a vote) to editing if needed/worthwhile.

The close vote queue on the other hand, is a proposition that the community is turning down. In the beginning you could say that it was huge because of all the old close votes hanging around in the system. Now, however, we have numerical evidence that it is growing on a daily basis, and can conclude that it is simply not handled in any meaningful way by the community, compared to the other queues.

I can see no other explanation than the community is rejecting the deal on the close vote queue. If we really wanted to go through it, we could. But we don't go through it, so it must mean that we somehow collectively don't think it is right.

My own personal analysis of the problem is that the close vote queue represents a grossly uneven deal between Stack Exchange, the company, on the one side, and the community, us, on the other side.

The imbalance I see is this: In a typical scenario a likely not very experienced user asks an unresearched, badly formulated and unfocussed question. He (or she) has spent very little time and almost certainly no effort in asking the question.

What is expected of the community under the deal with Stack Exchange is: Five of us spend time and effort to try to understand what the poster is asking, and, at least in the case of a dupe, research the topic for the OP and then classify the problem in five different categories.

Personally, I don't think this is a square deal. Not only are we asked to do the poster's work for them, but we're supposed to produce a whole team (five people) to do it, each one independently - and cross check the other reviewers' work in the process.

That simply doesn't sound like a reasonable proposition, and so the queue is left to slowly grow as questions get a single flag or vote, which nobody cares to decide on.

Answer 2

Maybe we should reconsider the requirements to moderate that queue.

Is the difference in experience between a 3K and a 2K reputation user (or even a 1K user) really distinguishable enough to warrant separate access levels for these queues? Can we just lower the bar of entry?

Perhaps tenure, or activity considering lurkers, could factor into review queue rights.

On a discussion based community typically such a task would be performed by vetted moderators, as proven knowledge of the rules is required for the task.

Could we allow lower rep users to vote on a series of close requests and then, as we have the community vet posts, I would suggest we review a sample of their decisions to see if they are on target with Stack Overflow's standards?

If they passed approval allow them access to the Close Votes queue without moderation - relying instead on the existing system of requiring 5 votes.

Obviously we would want to restrict such a system by reputation/activity to some degree to avoid new users wasting peoples time - also this would require another queue, "Close Vote Approvals" or something, available to higher rep users.

Just spitballing.

TLDR; Mor dakka.

Answer 3

Why not have a close vote reviewing event?

• Raise the 40 reviews/day to 50
• If you finish the 50 reviews, you get an event-specific bronze badge.
  • Finishing the 50 reviews for 5 consecutive days gets you the silver event-specific badge
  • 10 consecutive days gets you the gold event-specific badge
• 3 failed audits during the event disqualifies you for 4 hours from reviewing close votes
• Until the close votes reach 0.

After the queue is cleared, change the 'Steward' so it can be earned multiple times for close vote reviews. (e.g., 1k, 2k, 3k, etc.)

Answer 4

As I understand it, Most questions that make it into the close vote queue actually end up getting removed from that queue.

I don't think that we need to encourage a massive amount of reviews so much as work on the margin between posts coming in and posts going out.

There are a few things that I can think of to deal with this

• Allow unlimited reviews for people who have already earned the gold badge.
• Give high-rep users more weighty close/leave-open votes.
• Make the queue more conspicuous so that more people find the review queue.
• You can have a criteria to remove questions from the close-votes queue(for instance, if it has an and accepted answer with a certain threshold of upvotes)

You can also introduce another gold badge for reviewing some 5000 or so reviews. This would probably be the most effective option, but it would also attract the most badge grinders.

the point is that we want to whittle the number of extra reviews per day to 0, and you really shouldn't need more than 500 extra reviews per day for this.

Answer 5

I think the best thing to do about it is to clear all questions from the queue as a one-time thing. Once the queue is down to a reasonable size, you'll be more likely to get people to actually pay attention to it.

Granted, that means that there will be 50k+ closed-voted questions that won't be reviewed. But at least new ones will be more quickly dealt with.

The purpose of the close queue is to draw attention to bad questions in tags that don't draw as much attention. In high-traffic tags, bad questions are usually shut down within 5 minutes. In low-traffic tags, they are not.