16

After a few months, the Close Votes queue's large backlog is slowly climbing down, but in the short term we still see regular upward spikes.

I'm curious, when it reaches a manageable size, how manageable will it probably be? Will it ever reach a manageable size?

I ran a naïve-and-slow-as-hell query on the Data Explorer. It tries to fetch the daily amount of questions in and out of the queue, and shows that the number of questions entering the queue each day is usually larger than the number of closed questions on the same day. That means the queue would never empty.

But my query does not account for close vote aging, or questions leaving the queue for gathering enough "do not close" votes. I'm not even sure that data is available on SEDE. So my conclusion is probably not true.

Does anyone have reliable statistics on that?

14
  • 3
    It should eventually reach zero, like the other queues, if folks keep working on it.
    – user102937
    Commented Jan 15, 2013 at 16:37
  • 4
    @Robert I think the OP is looking for some more convincing evidence or explanations. I'm not saying you're wrong, just that your comment is kind of pointless (which is why it's a comment and not an answer, I imagine). Commented Jan 15, 2013 at 17:11
  • 3
    @jadarnel27: Sorry, I left my crystal ball at home this morning.
    – user102937
    Commented Jan 15, 2013 at 17:13
  • 2
    The point here is: the backlog makes me sort of anxious, so I was wondering if I could get more information on how it's going. No crystal ball needed, just data, and perhaps some inferences. Reviewing close votes is heavy work (at least to me, I have to open many posts in a new tab, and go through them), and not seeing the queue's size reduce significantly is a bit frustrating.
    – bfavaretto
    Commented Jan 15, 2013 at 17:16
  • @Robert I apologize if I came across as rude, that was not my intent (although using the word "pointless" was rather careless on my part). I just meant that your comment didn't contribute to the discussion because it lacked justification. Commented Jan 15, 2013 at 17:24
  • 4
    @jadarnel27: It's been going down gradually over time, so if people don't lose interest, I estimate that at the current rate the queue will clear sometime in 2016.
    – user102937
    Commented Jan 15, 2013 at 17:26
  • I'll just be honest; I'd prefer that the devs focus on more important things than satisfying your curiosity. They're the only ones likely to have any meaningful statistics.
    – user102937
    Commented Jan 15, 2013 at 17:31
  • @RobertHarvey That's a fair enough answer. I was hoping the statistics already existed, even if a little outdated, as the team seems to be keeping a close eye on the review system.
    – bfavaretto
    Commented Jan 15, 2013 at 17:33
  • Side question: How did it get so large to begin with?
    – user206222
    Commented Jan 15, 2013 at 18:15
  • 4
    @Telthien: Many of those posts are legacy, occurring before the current review queues were put into place. A lot of them didn't get closed because they were not interesting enough to get enough views to attract the necessary number of close votes.
    – user102937
    Commented Jan 15, 2013 at 18:17
  • @RobertHarvey Thank you! Interesting site history.
    – user206222
    Commented Jan 15, 2013 at 18:24
  • 2
    @jadarnel27 The "convincing evidence or explanation" I would cite are the smaller Stack Exchange sites. Server Fault's close queue cleared much faster (by virtue of being much smaller), and now it's rare to see more than 20-30 messages in it awaiting review. It's routinely at (or near) zero. I see no reason Stack Overflow wouldn't behave the same way once you guys get through the backlog.
    – voretaq7
    Commented Jan 15, 2013 at 20:00
  • I've forked the query to parameterize by date data.stackexchange.com/stackoverflow/query/116036/…' Stats for may looks like many more go in than out Commented May 23, 2013 at 19:14
  • @RobertHarvery yes. Because 82,000 definently is less than 60,000.
    – Cole Tobin
    Commented Sep 28, 2013 at 21:06

3 Answers 3

14

According to Huge Close Votes review queue on SO, the total was 55K questions to review back in October. 7,000 questions have been removed from the queue since then, so I think that's pretty good progress. If it keeps up at the same pace, we should have it cleared in just under 21 months from now, so around October of 2015.

I've been watching the stats for that queue for the past several weeks, and I noticed that there's a little bit of overlap between the daily and all-time leaders. This tells me two things:

  1. Some people do stay interested in reviewing questions for closure beyond the point where they've gotten a gold badge for it.

  2. Plenty of new people are also helping out in review.

Those two things make me hopeful that the pace will at least stay steady (if not increase) for a while to come.

19
  • 7
    Hey, my magic 8 ball estimate was pretty close!
    – user102937
    Commented Jan 15, 2013 at 17:34
  • @RobertHarvey Yeah, not bad at all! :) Commented Jan 15, 2013 at 17:35
  • Thank you @BilltheLizard, I wasn't aware of the discussion you linked to.
    – bfavaretto
    Commented Jan 15, 2013 at 17:38
  • 3
    From seeing how Server Fault's queue dropped, it does seem to accelerate as the queue gets smaller (once our queue got under 1000 posts people really started attacking it), or when approaching/passing landmarks like 10-thousands, 5-thousands, thousands, hundreds...)
    – voretaq7
    Commented Jan 15, 2013 at 20:01
  • @voretaq7 There surely is a psychological component to how people deal with the queues - which motivated this very question, by the way.
    – bfavaretto
    Commented Jan 15, 2013 at 22:39
  • 1
    more than 2 months passed since your answer, and current trend seems to go rather strongly against your past observations. Per my reading, your reasoning is based on the fact that queue went down from 55K to (55-7)=48K in 3 months prior to your post. 2 months after it though, queue only increased to 49.8K. Is it about time to re-check the data and reconsider estimate?
    – gnat
    Commented Mar 19, 2013 at 12:05
  • @gnat "If it keeps up at the same pace..." I'm not going to try and hit a moving target. Commented Mar 19, 2013 at 12:38
  • @BilltheLizard agree, that "if..." provision in the answer saves it from becoming plain wrong in the face of recent evidence. Using words "moving target" to describe stall queue size sounds funny :)
    – gnat
    Commented Mar 19, 2013 at 12:41
  • 2
    @gnat I meant the queue-zero date, but that's still a little awkward-sounding I guess. Commented Mar 19, 2013 at 12:47
  • yeah. Date never as a "moving target" sounds funny, too. Is there a constant for “end of time”? :)
    – gnat
    Commented Mar 19, 2013 at 13:05
  • @gnat Haha, I'm a little more optimistic than that. There is an end date, we just don't know when it is yet. :) Commented Mar 19, 2013 at 13:09
  • 2
    @gnat I've given it some more time, and that thing is just not going down. That will teach me to try and extrapolate from two data points. :( Commented Apr 11, 2013 at 19:51
  • @BilltheLizard I see. #3 in top at CV reviewers page indicates that you invested some effort into this
    – gnat
    Commented Apr 11, 2013 at 19:54
  • @gnat Yes, I've spent almost as much time in there recently as I have in the mod queue. I'll keep chipping away at it, but even my super-powered diamond close votes don't seem to make a dent. Commented Apr 11, 2013 at 20:03
  • @BilltheLizard while we're at it, let's recall that this "damn hard thing" is about "300 freaking easy golden badges that could drain 40,000 items from the queue in less than a month, and so little interest in these among 10,000 eligible users - why?" :)
    – gnat
    Commented Apr 11, 2013 at 20:06
5

Extrapolating a little what Bill the Lizard said:

  • We have around 3k reviews a day (based on today's stats)
  • It probably takes around 7 reviews per post to remove from queue

So we have:

  • ~400 posts leave the queue daily
  • ~24k posts left the queue in 2 months (since October)
  • ~24k - 7k = ~17k posts entered the queue in two months
  • ~280 posts enter the queue every day

Looks reasonable!

4

21 months wow. I am not questioning this estimate but I would want to figure why should it take that long.

Let's see, we've got more than 10 thousands users above 3K rep who could work in this queue.

Assuming 7 reviews per item, about 300,000 review actions are needed to handle 40,000 items.

  • Now, let's look at above data from badges perspective. 300,000 review actions mean 300 golden Steward badges.

How easy is to obtain these badges? Dead easy if one filters items correctly (in my experience, it takes about 10-20 minutes of work a day after first few dozens reviews) and rather difficult otherwise.


Okay so what do we have? There are 300 golden badges laying in front of about 10,000 eligible users. These badges could be obtained by spending 10-20 minutes daily for less than a month (1000 review actions, limited by 40 per day -> 25 days).

300 freaking easy golden badges that could drain 40,000 items from the queue in less than a month, and so little interest in these among 10,000 eligible users - why?

Above reasoning shows that observed speed of processing the queue is much much slower than it could be. We are getting 20+ months estimate while something like 2-3 looks entirely possible.

I believe this is because the way how close vote queue is presented to users makes review flow look harder than it really is.

5
  • 1
    I kind of agree with your points, but a) this isn't an answer to the question, and b) no matter what you do to reward them, adding time to most people's daily requirements will be seen as a chore.
    – Rory Alsop
    Commented Feb 17, 2013 at 10:20
  • @RoryAlsop answer updated to address your point "a)". As for the point "b)", I do not understand it sorry; would appreciate if you elaborate
    – gnat
    Commented Feb 17, 2013 at 13:00
  • 1
    Filtering doesn't make it as easy as you make it sound. There are still a lot of hard questions to review, either because they're borderline closable (take longer to decide), or long questions / have a lot of comments (take longer to read through. And skipping a lot of harder questions makes reviewing boring IMO. So the badges are not so rewarding. Also, I think it would still take longer than your estimate, as we have lots of questions (off all types and with multiple close reasons) coming into the queue every day.
    – bfavaretto
    Commented Feb 18, 2013 at 2:15
  • @bfavaretto the points you make don't match well with data I observe. I mean, we're talking about guys who already closed - let me check - 195,751 questions (not counting deleted ones). And that's been mostly without queue and badges. I find it hard to believe it would be that difficult to them to handle 1/5 of that, with queue and badges. Once again, we're talking about 20+ months estimate
    – gnat
    Commented Feb 18, 2013 at 6:06
  • 1
    @gnat posted in Feb. Ever since queue only got bigger... All the estimates here are worthless. What can we do?
    – Shai
    Commented Aug 7, 2013 at 9:00

Not the answer you're looking for? Browse other questions tagged .