-9

This idea has already been proposed in response to one of many questions about the size of the close vote queue on Stack Overflow. So here it is again, as a separate feature request:

Reset the close vote queue.

Pick a date, and mark all the close-votes that are earlier than that date as "hidden", and only show the newest close votes. At the point of enacting this, and picking the date to be "now", the queue size should go down to almost zero.

Watch the size of the queue for a week or two. If it has grown far beyond zero, then this experiment didn't work, and you can unhide the old questions and return to the status ante bellum.

But if the queue size does remain near zero, then it means that people are more motivated to contribute where they feel they can make a large relative impact. Achieving "zero" is a feel-good goal, while achieving "91225 (down from 91240)" doesn't feel that big of a deal. And if the experiment does work and we manage to keep up with a fresh queue, then we can slowly layer old questions into the mix, money-laundering style. Nobody will ever notice :-)


Let me offer another angle on this suggestion.

I want to analyse the claim that there is a motivational difference between a full and an empty queue. Specifically, the claim is that people are more motivated to work on a small queue, for psychological reasons. It feels like you're making a greater impact in making a small problem resolved than in making a large problem a tiny bit less large.

I have no proof that this effect exists on Stack Overflow, only circumstantial evidence coming from the observation that all the other queues consistently go back to zero regularly.

We could obtain further support for or against this claim by looking at historic numbers of queue throughput (which I'd like to see, if anyone can pull this data out); and/or we can run an experiment. Either way, I believe it would be valuable to find out for sure if there are psychological measures that would improve user engagement that we're currently leaving on the table. I'd be perfectly happy to find out that nothing of this sort is the case, but I'd like to know what we know.

17
  • 3
    Would be interesting to hear the counter argument from any of the 8 people that have downvoted so far. Commented Nov 23, 2013 at 13:09
  • @MartinSmith: Yes, indeed. I know there's a considerable "established" opinion that the close-vote queue is "working as intended", but given the choice of leaving it as it is or thinking about options, I thought I'd surface one of the suggestions.
    – Kerrek SB
    Commented Nov 23, 2013 at 13:29
  • 11
    This just sweeps the problem under the rug. The problem is that we have nearly 100,000 questions that need reviewed. Hiding them doesn't change that. If you want to reach a "feel-good goal" use a tag filter. I clear both the facebook and android tags out of the close queue about once or twice a week. Commented Nov 23, 2013 at 16:05
  • @BilltheLizard: Yeah, people could use filters. But do they actually? You could also say "people should just feel good anyway", but that's neither here nor there. The suggestion is to try something that may have a significant psychological and emotional effect, and see if it could actually help. It might not, but then again, it's not a destructive experiment and could easily be rolled back. I just wish we had some real insight in how people behave.
    – Kerrek SB
    Commented Nov 23, 2013 at 16:17
  • 2
    Telling people to use the existing tool isn't the same as telling them to "just feel good anyway." The tool was created for a purpose. If people don't use it, why should more steps be taken to address the same issue? Commented Nov 23, 2013 at 16:22
  • 1
    @BilltheLizard: OK, here's a sanity check: Can we figure out how many items, in absolute counts, get processed in each queue, and how many items get added to each queue, say on a daily basis?
    – Kerrek SB
    Commented Nov 23, 2013 at 16:25
  • 1
    You can approximate the daily numbers from the monthlies that Shog9 posted here: meta.stackexchange.com/q/208311/1288 (I have no idea what happened in July, but that's when the deficit exploded.) Edit: Shog9 just reminded me, that's when all the close reasons were changed and close flags were redirected to the close queue instead of going to the mod queue. Commented Nov 23, 2013 at 16:35
  • @BilltheLizard Actually, I think eliminating the close queue will reveal that the current process won't keep up with incoming rate of questions that need to be improved-or-closed, which I think is the more serious issue we face. Commented Nov 23, 2013 at 17:19
  • 2
    No. No no no no no. There are still plenty of bad questions that can be closed. Just clearing the queue doesn't change the situation.
    – Makoto
    Commented Nov 23, 2013 at 17:22
  • 1
    @Makoto: I didn't say "clear". I said "hide temporarily".
    – Kerrek SB
    Commented Nov 23, 2013 at 17:31
  • Six of one, half a dozen of the other from my point of view. You're still essentially masking the amount of questions that have been nominated to be closed. Yes, the number of questions to be closed can be disheartening, but hiding the number of questions that need to be addressed only provides a false placebo in that the problem is getting solved faster than it is.
    – Makoto
    Commented Nov 23, 2013 at 17:34
  • 2
    @Makoto: A curious analogy - the placebo effect has been demonstrated to have real positive effects...
    – Kerrek SB
    Commented Nov 23, 2013 at 17:35
  • 4
    @PeterAlfvin The growing number of questions in the close queue already demonstrates that the current process isn't keeping up with the incoming rate of questions that need to be improved or closed. We don't need to hide it to show that. Commented Nov 23, 2013 at 17:41
  • 1
    @BilltheLizard Some people believe that the only reason we're not keeping up now is because people are discouraged by the queue size. Although that's a factor, I think the situation is more fundamental. More importantly, until the existing queue is "removed" in some fashion, we won't be able to see and deal with the specific problems that are holding up timely and efficient resolution of the incoming flow. The analogy in lean manufacturing is "lowering the level of water in the lake to expose the rocks below". Commented Nov 23, 2013 at 17:49
  • @BilltheLizard: It seems like we just did and everyone is loving it...
    – Kerrek SB
    Commented Mar 3, 2014 at 15:33

1 Answer 1

9

Once upon a time, three people were living in a flat. All of them were pretty busy, and quite fond of pizza, to the point it made most of their meals.

At first, everything was under control, but after a while laziness, procrastination and consumption of strong beverages took their toll, and empty pizza boxes started to pile up in the living room. It went to such a level that nobody could find enough motivation to tackle the seemingly insurmountable task of putting all that to the trash downstairs, and things only went worse from there.

When the critical point was reached, one of our protagonists had an idea: resetting the pile. By quickly stuffing all those boxes in an unused room, they could restart from scratch. Naturally, from now on, they would reliably put the new boxes into the trash, so they could take some of the boxes out of that room from time to time and also dispose of them. Problem solved.

Alas, a few months later they realized the situation had not improved at all, mainly because they still felt inclined to leave empty boxes around in the living room, and they now had quite a few others locked up in another room developing their own fauna. From there on, the situation degraded very quickly, and the eventually necessary cleaning ended up involving sweat, tears, breathing rags, angry neighbors and a very messed up flat.

(Apologies for not coming up with a better analogy.)

Close vote reviews were introduced to put our empty pizza boxes (questions with pending close votes) in our living room instead of another room somewhere. Thanks to them (the reviews, not the pizza boxes), the problem is in our face all the time (the increasing number of related questions on Meta demonstrates this), and we're arguably able to measure it better.

Your suggestion amounts to putting all the boxes back into the other room, negating the purpose of close vote reviews, and assumes less boxes in the living room means a better incentive to dispose of them. I'm afraid that incentive, if it does exist, will not be enough, mainly because we have a lot of boxes, and most of the time it takes five of us to take a single one to the trash. We will be back to square one eventually.

Instead of trying to hide at least part of the problem, we can start getting more involved and get organized on our own if we want to.

11
  • 1
    Those are some heavy pizza boxes.
    – animuson StaffMod
    Commented Nov 23, 2013 at 15:57
  • @animuson, yet you can lift one alone, that's why I said "most of the time" ;) Commented Nov 23, 2013 at 15:59
  • Do you actually have proof that close votes are such a natural problem? As far as I know, we have never put that to a test, because the queue started out heavily loaded by inheritance when it was first introduced, didn't it?
    – Kerrek SB
    Commented Nov 23, 2013 at 16:10
  • @KerrekSB, the point is that it does not matter. Close votes cast before reviews were introduced are not a burden that reviewers inherit from others, they're the reason why close vote reviews exist in the first place. Commented Nov 23, 2013 at 16:12
  • 2
    I'm sorry, but that point totally matters: The entire claim of my post is that the visible burden of the legacy is what stops people from doing what they would otherwise do if they couldn't see that burden. I know that's speculation, but I think it's plausible enough that we shouldn't dismiss it without actual evidence.
    – Kerrek SB
    Commented Nov 23, 2013 at 16:14
  • 1
    Well, I can assure you I personally vote to close something like a dozen questions per day on average these days, sometimes more, and half of them are still open when I double-check a few hours afterwards. To be sure, we would have to query the close votes and measure the progression of "new" close votes (since reviews began) versus the amount of "old" ones. A SEDE query, maybe? Commented Nov 23, 2013 at 16:17
  • Yes, please - see my above comment to Bill. I think we could have a simple sanity check here to show whether there's a "motivational" difference between the full queue and the empty ones.
    – Kerrek SB
    Commented Nov 23, 2013 at 16:26
  • Regarding "getting involved": I appreciate those efforts, but somehow I feel that they only really engage those people who are frequent visitors of Meta, or who feel above-averagely motivated to Do The Right Thing for the site. I believe that that's not nearly everyone who's eligible to vote, and that a lot of the reviewing is done "incidentally" by people who just want to burn a few idle minutes here and there.
    – Kerrek SB
    Commented Nov 23, 2013 at 16:55
  • @KerrekSB, strength lies in numbers, and "burn a few idle minutes in the interest of the network" scales very well. Commented Nov 23, 2013 at 17:00
  • Yes, exactly, which is why I think the best bang-for-buck comes from engaging with those "idle numbers", rather than with the few "I close 40 questions a day" Meta users.
    – Kerrek SB
    Commented Nov 23, 2013 at 17:02
  • 1
    +1 for making me hungry for pizza (and I also approve of your idea).
    – Jamal
    Commented Nov 23, 2013 at 17:52

Not the answer you're looking for? Browse other questions tagged .