31

Summary

As many of us are aware, the number of questions in the close vote review queue has been increasing at what appears to be a faster pace than ever. During the month of December, I observed that the number of questions in the close vote review queue was in the neighborhood of 100K, we're now at approximately 118.6K! Quite a big jump in a short period of time. So after giving this much thought and reading through the other discussions and proposed solutions, I thought I would take a stab at coming up with some simple ideas as to how we can approach the issue.

Problem

The number of questions in the close vote review queue is growing at a rate faster than the Stack Overflow (SO) community can currently keep up with.

Reasons this issue is escalating

  1. There are not enough qualifying1 members that are assisting with the close vote review queue.
  2. The number of close vote reviews a qualifying1 Stack Overflow member may perform is 40 per day. AFAIK, moderators may perform as many as they wish to.
  3. The number of questions being added to the close vote review queue is greater than the number of questions being removed from the queue.
  4. A close vote ages away after only 4 days for questions with more than 100 views.

Proposed Solutions

My reputation on SO is currently below 3K, and as much as I want to help by performing close vote reviews, I do not believe that lowering the minimum reputation required to perform close vote reviews is a viable solution to this problem. By lowering the minimum reputation to perform close vote reviews, I think it would negatively impact the quality of the site and make it harder for the community to remain focused, unless we can come up with a way to efficiently pick members that are under the 3K rep level that we feel would do well with the close vote reviews. Until that happens though, I believe focusing on simple modifications to the existing framework of the site will help reduce the number of questions in the close vote review queue.

Notes:

  • All proposed solutions assume that a qualifying1 member starts with a base of 40 close vote reviews per day, let's call this b.
  • Please do not focus on the numbers I chose in my solutions, they are just arbitrary numbers I chose to illustrate my points and can be changed
  • m refers to the maximum number of additional close vote reviews we may wish to grant
  • n refers to the number of questions in the close vote review queue
  • v refers to the total number of close vote reviews granted to a qualifying member.

Solutions:

  1. Variable Close Vote Review Limit - Every midnight, a script is executed to calculate the number of additional close votes reviews that can be awarded to qualifying members, this is based on the number of questions in the close vote review queue at that time. Since this number is calculated every evening, and is based on the needs of the site, the calculated additional close vote reviews should decrease over time and eventually go to zero when the number of questions in the queue reaches a nominal point. In my solution, the nominal point is n = 10000 This is the most important aspect of this feature because it removes the worry that some members have about members gaming the site to earn badges since the vote limit reduces over time; however, it can still help the site greatly reduce the number of questions in the close vote review queue while the site is in need of the heightened voting, now and again in the future if this issue returns. Calculation of v is as follows:

     v = b {n < 10000}
 v = b + ⌊n ÷ (100000 ÷ m)⌋ {10000 ≤ n < 100000}
 v = b + m {n ≥ 100000}

  2. Badge Privileges - Individuals that have earned specific badges are awarded additional close vote reviews. Since these users present a lower risk than those users that have not earned these badges, providing them extra close vote reviews is very helpful at reducing the size of the queue:

    1. Those individuals that have earned the Steward Badge for the close vote review queue: v + 25
    2. Those individuals that have earned the Marshal Badge: v + 5
    3. Those individuals that have earned both of the badges listed above: v + 30

    NOTE: I do have a reservation about this solution in that I am unsure if the site is currently capable of tracking how many close vote reviews a particular individual may have, as opposed to setting one value that applies to all qualifying members. But I thought it would be nice to at least share this idea.

  3. Increase close vote expiration - Allow close votes on posts that have more than 100 views to expire after 7 days; instead of 4 days as it currently stands. My feeling is that 4 days is just too short of a time span to allow votes to expire. In situations where we have long weekends or holidays, or simply not having enough people visiting the site to work on the review queues, close votes may be falling off for the wrong reasons.

Outcomes

Based on these solutions, I foresee that roughly 19,979 qualified members will have the ability to perform a minimum of 75 close vote reviews when m = 35, or 100 reviews when m = 65, with additional close vote reviews awarded to members that have already earned the specified badges. The 'Badge Privilege' users will greatly influence the reduction of the close vote review queue at the beginning since they will have at most 105 close vote reviews (m = 35 and solution 2) or 130 close vote reviews (m = 65 and solution 2).

With these extra close vote reviews being performed and the increased close vote expiration, the close vote review queue should become very manageable in a short period of time. It is due to the nature of the calculated additional votes that people will remain limited in how many votes they cast, while still meeting the needs of the site in the event that the close vote review queue increases again in the future. While I do understand many are worried about the gaming aspect of the close vote review limit, I do not think that takes a higher priority to the close vote review queue because that queue directly influences the quality of the content that we allow on the site. If questions remain open, it sends a message to users that they can continue to ask lower quality questions without any ramifications.

Data

Assumptions:
(as of 1:12pm, February 17th, 2014)

  1. There are currently 827 individuals that have earned the Marshal Badge
  2. There are currently 19,979 individuals that have a reputation of 3,000 or higher - Number of users with the specified minimum reputation
  3. Using the Stack Exchange Data site, I was unable to determine the number of users that have attained the Stewerd Badge for the Close Vote Review Queue.

Notes

  1. The minimum reputation needed to cast a close vote is 3,000. See Reference #1

Related Discussions:

  1. Huge close votes review queue on Stack Overflow
  2. Would raising the review limit help with the Close Votes queue?
  3. Increase close votes in /review [duplicate]
  4. Close Votes review: I'm going on a strike!
  5. Daily close votes queue limit
  6. Close votes daily limit increase [duplicate]
  7. Increase vote to close count when higher rep [duplicate]
  8. Should the Daily Close Limit be Higher?
  9. Lowering rep requirement for reviewing activities
  10. Reviewing Close Votes, is just too painful, make it easier [duplicate]
  11. What can be done about the massive Close Votes queue on Stack Overflow? [duplicate]

References

  1. Cast close and reopen votes
  2. Close Vote Expiration Updated
Improve this question
14
  • 28
    I think the actual problem is that reviewing isn't fun and nobody cares to do it. If we could change that then the queue would be empty in no time.
    – juergen d
    Jan 26, 2014 at 10:00
  • 3
    Thank you for the feedback. @juergend, it may not be fun for the majority, and while I do agree making it fun would help, but at least in the meanwhile we can empower those that do spend the time reviewing with a little bit more working space. Thanks again. Cheers.
    – Anil
    Jan 26, 2014 at 10:05
  • 4
    Excellent suggestion. Personally I tend to avoid close-voting questions with unfamiliar tags, so perhaps we could reverse this and allow users who have been awarded tag badges extra votes to close questions asked under those tags?
    – Roger Rowland
    Jan 26, 2014 at 10:15
  • R.e. "The number of close votes a qualifying Stack Overflow member may cast is 50 per day". I think it is actually 40.
    – Joseph Quinsey
    Jan 26, 2014 at 14:26
  • 5
    The problem isn't the review cap. The problem is that the vast majority of people eligible to review never hit that cap. I'm not sure how this post is much different than the previously declined attempts to raise the limit.
    – Geobits
    Jan 26, 2014 at 15:32
  • 1
    @JosephQuinsey, I knew that topic would come up. On this page, stackoverflow.com/help/privileges/close-questions it states 50; however on the close review stats page, I noticed most users cap out at 40, and a few users on occasion had 50. So I was unsure what the actual cap was, so played it safe and went with what was stated in the help doc.
    – Anil
    Jan 26, 2014 at 15:37
  • 3
    You can cast 50 close votes per day max. In the close vote queue you get a maxium of 40 review tasks. (Close votes are a different thing than review tasks). I can do my 40 review tasks without casting a single close vote. Moderators can do as many review tasks they want. There might be an unconfirmed glitch that might allow normal users to do a couple of more review tasks.
    – rene
    Jan 26, 2014 at 15:49
  • 1
    @Geobits: The difference is that the limit increase others have suggested were more static, or permanent in nature; whereas, my suggestion matches the cap based on the needs of the site and reduces that cap as the queue is reduced. The badge privileges should add the extra boost to keep the queue under control. Also, with the increase in review limits, the people that currently assist with the queue could make sizable dents every day in the queue. Based on my formulas, imagine if just 20 people could cast 100 close votes everyday, luckily we have more than 20. ;) Cheers.
    – Anil
    Jan 26, 2014 at 15:56
  • 3
    Right, but IIRC, the others weren't declined for being static. They were declined because we don't want a small group of people doing more reviews, we want more people doing reviews in general.
    – Geobits
    Jan 26, 2014 at 15:59
  • 1
    @Geobits, And yet while nothing is being done, the queue is getting out of control, so I proposed an alternative to just raising the cap that can help keep the queue under control, while providing a sliding scale to auto lower the cap since many were worried about keeping the cap so high. I am failing to understand why empowering the people that work on the close vote review queue is looked down upon. That last statement wasn't directed at you, but just the general community. Cheers.
    – Anil
    Jan 26, 2014 at 16:31
  • 11
    How about adding a Jimmy Whales type appeal banner in orange, visible only to the +3000s, saying "If each of you cast 25 close votes, we can eliminate the entire queue in one day", and maybe offer a reward like an increased rep cap for a week or so.
    – Kerrek SB
    Jan 26, 2014 at 17:50
  • 2
    amount of Steward badges (as of Dec 2013) can be estimated using data provided by Shog here "...About half of ~500 users who have gotten golden badge at 1000 reviews indicated an interest to continue using queue even without badges."
    – gnat
    Jan 27, 2014 at 7:30
  • 1
    We're now a little over 112K, 2K more than when I posted 3 days ago. :(
    – Anil
    Jan 30, 2014 at 5:02
  • Whoever wants to champion this change has the burden of showing that a majority of reviewers are currently constrained by the limit on the number of votes that can be cast daily. Otherwise, this is akin to premature optimization.
    – Louis
    Jan 31, 2014 at 15:56

2 Answers 2

Reset to default
10

The reason so few close votes get cast is not limits. Very few people reach their limits. Raising the limits won't help. The reason is that the close vote queue is just not as much fun as the others. I don't know why, exactly, and I don't suppose any of us do. I made a reasonably popular feature request (Please streamline VTC-as dupe in the review queue) to speed up at least one part of it, but it never happened.

In the other queues I think you can occasionally find things that make you happy: first posts and late answers that are from people who really want to help. Low quality posts that can be improved - either immediately with an edit, or eventually with a good comment. You are not just wading through endless mud. But the close queues - even if questions don't deserve to be closed, which is by far the exception, they are usually pretty crappy. And they've attracted all these comments about how crappy they are, and responses that are all Y U MAD BRO and Can't We All Just Get Along and after about 5, I really need to stop. You can raise my limit to 100 or 1000 and it won't make any difference at all.

No idea what to do about it. Declaring bankruptcy and starting over won't help - it's not that nothing ever closes or the number doesn't get smaller, not for me anyway. It's that the stuff I am reviewing is so depressingly bad. I suppose trying to get more people to join me in this pursuit is an option, but not one I'm going to use.

Improve this answer
2
  • I can review 100% of any of the other queue, while drinking one cup of tee and feal like I have done someone useful.
    – Ian Ringrose
    Feb 3, 2014 at 22:23
  • 2
    Crushing the dreams of others is what makes the CVQ the most satisfying of all
    – OGHaza
    Feb 15, 2014 at 14:38
2

The number of questions in the close vote review queue is growing at a rate faster than the Stack Overflow (SO) community can currently keep up with.

I agree that this is a problem that only seems to be getting worse, and I don't know if the SE/SO team is really serious about resolving this. There have been so many recommendations and input from the community, yet I have not seen any changes targeted to lessen the increase in questions in the close vote review queue, let alone trying to get it back to under 50,000.

Some recommendations :

1) Give people with over 10K rep double the current number of close votes in the review queue. So instead of 50, it will be 100 per day.

2) Give everyone over 5K an extra 20 or so close votes in their arsenal. (I often times run out of close votes early in the day, and unfortunately run across DOZENS upon DOZENS of new questions that should be closed when I'm searching to answer questions later on in the day.)

3) Questions that are 60 days old or older, should only require 3 close votes to be closed. These questions will likely not be getting very many views anymore, thus attracting additional close votes from (good Samaritan) passerby's is highly unlikely. Also, the question is very unlikely to be edited to make it salvageable at this point.

Improve this answer
1
  • Options (1) and (2) both seem to be about getting the small number of committed users to do even more than they already do. While I wouldn't object to users being given more votes I certainly wouldn't want it pitched as the solution to the problem
    – Richard Tingle
    Mar 1, 2014 at 12:46

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .