104

I've been putting this off for a long time...

Close vote aging - the deactivation of votes that haven't resulted in a question being closed - is a critical part of the vote-to-close system, but has something of a troubled history: all too often, it has been more annoying than useful, capriciously blocking the closure of awful-but-obscure questions. We've spent the past couple of years experimenting with changes to make the rest of the system operate more effectively, but now it's time to bite the bullet and hammer this last piece into place.

My proposed change is simple - skip to the last section here if you wish; I know you're all tired of reading my overly-long essays about obscure parts of Stack Exchange, but whether you realize it or not this poorly-understood system has a profound effect on how these sites operate. Therefore, I humbly ask that you bear with me as I try to explain the rationale behind this proposal.

The importance of vote aging

Closing can be both fair and effective when a question is quickly brought to the attention of multiple trusted users, who may then opt to vote or to refrain. Problematic questions can be shut down quickly or fixed if possible, and discussion concerning problems with the question can be conducted.

But when this doesn't happen, things get dicey: problems may go unaddressed and the direction of moderation may be unfairly influenced by a subset of privileged users. One can easily imagine situations where, given sufficient time, any question might be closed due to the gradual accumulation of votes from a tiny minority of voters, and indeed this does happen.

Vote aging is designed to minimize this, by creating a ticking clock that demands a decision be made promptly and otherwise considers the matter resolved. But after a couple of years, a problem with this system became apparent...

The rationale behind the current rules for aging

These are the current rules for vote aging:

when the aging task fires it just looks at every post that has at least 100 views where all outstanding close votes are older than 4 days.

Then it deletes a close vote. A day later, it runs again and does the same thing, until either another close vote shows up, or all the close votes are deleted.

Simple enough, right? Except for that "100 views" thing. There is a long tail of questions on these sites that take months or years to garner that many views; sure, some of them are awful, but there's also a treasure-trove of hard-to-find information among them - surely lack of popularity isn't a good reason to damn a question?

This requirement exists because back in 2011 it was getting really hard to get enough people looking at lousy questions to close them. Review didn't exist yet; there were 10K tools to aid in reviewing close votes, but not enough people using them to really matter. And folks were getting frustrated; in many cases, it was more effective to flag and ask a moderator to close than it was to vote! The community of trusted close-privileged users was becoming ineffective; the bulk of closing was being done by moderators, acting at the behest of anyone who could flag.

The change in how aging works worked - at least, it kept votes around long enough for someone to act on them. But it didn't really address the underlying problem, and it made vote-aging considerably less effective.

Figure A: Close vote results over time

Here's a chart of close votes by month, along with some annotations for various events. Unfortunately, we hard-deleted close votes before the summer of 2011, so I don't have accurate data with which to illustrate the problem that this 100-view requirement addressed - you'll have to just trust me that it was a real thing, and that this did help. I can demonstrate how it failed to address the underlying problem though...

Figure B: Close vote effectiveness by question view-count, summer of 2011

That chart needs some explanation. The X axis is views, the Y axis is the percent of close votes that resulted in closing (5th close votes and moderator closes are excluded here, as they're instantly effective 100% of the time). Each line represents a timeframe: 4 days, 16 days, 30 days, 60 days and all time. The data reflects the four months following the addition of the 100-view requirement for vote aging, but the actual view counts are current.

The key observation here is that the 100-view aging threshold made time irrelevant for the questions that fell below it: there's very little difference in the effectiveness of close votes after 4 days and 60 days, and not much more for all time. But once that 100-view threshold is past, effectiveness drops like a rock...

An obvious question here is why the all-time effectiveness for even questions over 100 views is pretty high: well, remember, I'm looking at the view count now - a big chunk of the questions closed at over 100 views started out with less, and didn't garner more until after they'd been closed.

And that's where Review comes into play...

How Review affects this

Refer back to Figure A in the previous section. See where the new review system was introduced? Suddenly, we didn't have to rely on chance or even personal taste to give votes nominated for closure a look from privileged users. Close votes - and actioned close votes - exploded! Now compare Figure B to...

Figure C: Close vote effectiveness by question view-count, summer of 2014

It took us a couple of years to work out all the kinks; scaling up something like this presents some rather unique challenges... Didn't help that we changed all the close reasons half-way through: we suddenly had a LOT more people reviewing a LOT more questions with unfamiliar and occasionally confusing options. And the old aging rules still in effect... The result was chaos: instead of benefiting more obscure tags/topics, the views threshold ensured that the smallest problems became the biggest clogs in the pipeline, throwing them up for review again and again and again until they would finally get closed - meanwhile, questions that garnered lots of organic views dropped out fast no matter how problematic.

We had built and refined a vast system for democratizing justice... and then put it to work enforcing parking tickets.

Isn't this just a Stack Overflow problem?

No. Actually, right now this is less of a problem on Stack Overflow - a few months back we turned on a system for prioritizing reviews there, something of a twist on the old aging scheme: just kicking stuff out of the queue when it clearly wasn't making any progress, focusing the attention of reviewers on areas where they stood to do the most good and least harm. That... Actually worked pretty well:

Figure D: percentage of initial close votes actioned in Y days, by month

The summary of this chart is: 90% of questions that get closed on Stack Overflow now get closed in under 2 days; 50% get closed in less than an hour!

This isn't true for other sites though. Here's Figure D for Ask Ubuntu:

Figure D.AU: percentage of initial close votes actioned in Y days, by month

And remember Figure A? Here's figure A on Ask Ubuntu:

Figure A.AU: Close vote results over time

You see the same problem developing there as on Stack Overflow: a growing number of unactioned close votes that aren't aging. We could try to follow the same path there as on Stack Overflow, prioritizing the close queue - but that's a bit of a band-aid (it still doesn't avoid votes trickling in organically over extremely long periods of time), and has to be heavily tailored to the site. Also, there's one more problem with aging...

You got ONE SHOT - better make it count...

Remember the original problem that the 100-view threshold was intended to address? Well, what happens today if you find a seriously problematic question in the obscure tag that you follow, you vote to close it, and... Nothing happens. Maybe your vote hangs around forever, or maybe enough random viewers trickle in from Google to hit the view threshold and your vote ages away.

What can you do? Not much. You can't re-cast a pending vote, and you can't re-cast an aged vote. Your proposed closure was put before the committee and... ignored. Maybe there were a lot of parking tickets that day.

Some folks raise a flag when this happens, but moderators are often reluctant to intervene on topics they aren't personally familiar with unless the problem is truly egregious - they signed up to be exception-handlers after all, and a question no one cares about isn't all that exceptional. Others lean on the Very Low Quality flag to give questions a second shot at review - but strictly-speaking, this is an abuse of VLQ and will probably get harder as we continue to develop review.

What you'd like to be able to do, upon coming across a problematic question you've previously voted to close, is just bump it back into the close queue. Once upon a time, this was actually the recommended way to handle vote aging on questions if you kept tripping over them - but at some point, it was disabled, probably because folks abused it to keep harping on some personal annoyance.

Proposed changes

Finally, we're here. That bit above was hard; this is simple:

  1. Start aging votes after 14 days, even if the view count is too low. That is, if the newest vote on the question is 14 days old, aging will begin even if the view threshold has not been exceeded. If the view threshold is exceeded sooner, then the existing time threshold (4 days since newest vote) will be used instead.

    This should be site-configurable on the off-chance that we need it to be greater somewhere (or decide that even this is too long to wait), but realistically this should be sufficient - the vast, vast majority of questions that do get closed are closed in under half this time. We should keep the view threshold in place for now just in case I missed something crucial here, but should plan to reduce it to some token number in the near future - just high enough to protect against insane edge-cases that get missed entirely, but not so high that it's in effect on a significant number of questions.

  2. Allow re-casting votes that've aged away after 14 days. That is, 14 days (use the same site-configurable value used in #1 here) after your vote has aged away, you're free to cast the same vote again.

If your first try doesn't get any attention, and you keep tripping over the same problematic question... You should get to have another go at getting it reviewed. A 14 day waiting period means you don't get to spam reviewers with every question you personally don't like, but does give you that opportunity to correct oversights without abusing flags for the purpose.

For consistency, both of these changes should be applied to Close Votes, Close Flags, and Reopen Votes.

That's... It. We discussed lots of crazier, more complicated ideas over the past couple of years, but kept coming back to the idea that we were fixing the wrong problem - and indeed, we were. When Review works, vote aging can be pretty simple; when it doesn't, there's a deeper problem that needs to be fixed. At last, it's time to rip off the adhesive bandage and see the moisting flesh underneath...

33
  • 9
    The tl;dr here is: close vote aging exists to serve as a limit on the process for evaluating proposed closures, however it no longer fulfills that purpose due to other changes in how close votes are processed. This change will have an extremely small effect on the number of questions that get closed, however it will have a disproportionately large effect on how question closure is prioritized in keeping with my long-term goal to get questions closed fast or not at all. Please post requests for data porn as answers; this is long enough already. @Tim
    – Shog9
    Feb 26, 2015 at 6:34
  • 3
    Reopen vote behavior should be kept consistent with close vote behavior, @Famous, if for no reason other than to simplify documentation. In practice, the aging rules matter even less for reopen votes than for close votes - in fact, reopen vote aging was completely broken for several years and no one noticed.
    – Shog9
    Feb 26, 2015 at 6:48
  • 7
    Will this change be retroactive?
    – Braiam
    Feb 26, 2015 at 13:27
  • 3
    Yes, finally! On smaller sites (particularly small betas), or when dealing with questions that aren't on the front page, votes too often age away before a question that should be closed is closed. This is made worse by the fact that a mod "leave open" vote in the review queue kicks it out of the queue -- so even though that mod vote doesn't cancel the close votes, it vastly reduces the number of people who will see it. We want the community to handle these things, so I'm glad that we're going to stop hobbling the community. Feb 26, 2015 at 14:41
  • 7
    "capriciously blocking the closure of awful-but-obscure questions" -- that's so very true. Watching my votes go away when cleaning up "looking-for" garbage at SO was... not the most pleasant experience
    – gnat
    Feb 26, 2015 at 15:25
  • 4
    What do you mean, @Braiam? Not going to try to re-write history if that's what you're asking; this would go into effect for current and future votes.
    – Shog9
    Feb 26, 2015 at 15:57
  • 1
    I've talked about it before elsewhere, @gnat - it's a remarkably inaccessible, ineffective way of exacting revenge, very few people bother, and those that do tend to shoot themselves in the foot. It's even more trivial to detect than other forms of revenge voting, so if it becomes a bigger problem as a result of this change we can and will adapt as-needed.
    – Shog9
    Feb 26, 2015 at 17:59
  • 1
    adapt as-needed I see. That makes sense, thanks. "14 day waiting period... means you don't get to spam reviewers with every question you personally don't like" - this throttling seems to make this approach reasonable. Two-week-breaks mean it gotta take quite a while for it to develop into a serious issue, and introduce a lot of cooling to prevent it from happening in the first place
    – gnat
    Feb 26, 2015 at 18:21
  • 1
    Yes, Actioned means the question was closed, @Josh (it does NOT mean that it stayed closed - this is not a measure of the accuracy of votes, merely whether or not they had any real effect on the question). I've edited to clarify your second point.
    – Shog9
    Feb 27, 2015 at 2:28
  • 5
    Dup-hammer helped (for duplicates), but the biggest win was just bringing down the number of unlikely-to-be-closed questions in the close review queue. The more people we have looking at a smaller set of more obviously-problematic questions, the faster it goes. @Roombatron5000
    – Shog9
    Feb 27, 2015 at 3:06
  • 4
    ...cont'd: Now I'll pass by questions that I feel should be closed and not vote because they won't have any CVs, and/or I know if four other people don't come by soon, my vote won't count anyway. Takeaway: if you give us the vote, then our vote should always count.
    – TylerH
    Mar 2, 2015 at 15:21
  • 2
    @Shog9 You can't just say that editing/downvoting are more effective at closing a question than close-voting. First, that's the whole point of close-voting; if there's such a need for something that you make a tool just for that thing, it stands to reason that it should be the best and most powerful tool for that thing. Second, many questions that receive CVs aren't really salvageable; they're just flat-out off-topic (OT is the reason that receives the lion's share of my CVs). Third, downvoting doesn't work to close the question, if it has upvotes, if it has activity, if it has answers, etc.
    – TylerH
    Mar 2, 2015 at 15:51
  • 2
    I said "moderation" not "closing" - they're not synonymous, @TylerH. If you see a question that clearly needs to be closed, vote to close it - it will get attention from others, now more than ever. But if you're on the fence - or you're looking at a post that you doubt many others will want to close - then consider the other options for indicating or addressing problems with it.
    – Shog9
    Mar 2, 2015 at 16:01
  • 2
    @TylerH A lot of the stuff where votes just aged away are questions that many would look at and kind of scratch our heads as to why they should be closed. Sure, some of them might be clear cut, but they've reached the point where nobody cares about them any more. It's better to drive people to stuff where consensus is far more likely to occur relatively quickly, and without contention, and continue to do that going forward. I've had a diamond since 2011 and I have seen so many things with 1 or 2 close votes on them seemingly coming out of left field, you can't optimize for that.
    – user50049
    Mar 2, 2015 at 16:29
  • 2
    @TimPost I don't think permanent CVs on questions that take a long time to close (months/years) count as wasted/squandered/problematic. The SE team does, it seems. I don't know why you/they do, but this difference in opinion is irreconcilable, at least for now.
    – TylerH
    Mar 2, 2015 at 19:31

4 Answers 4

29

This is now live.

On Stack Overflow and here on MSE, a hearty portion of old votes and flags (over half of them on questions that were long ago deleted) have been invalidated already; other sites will see the effects of aging start tomorrow.

I'd like to take a moment to address some of the concerns raised regarding the second portion of this change: the ability to re-submit questions for close review...

Time and time again, we've seen that it's possible to encourage certain actions and discourage others by adjusting their difficulty... But as soon as you make a given action impossible, folks give up and start looking for work-arounds.

That's not something I want to see more of here. If you feel strongly enough about a question to keep burning your votes on it, so be it - the system should provide a well-defined, well-constrained path for you to do so rather than forcing you to seek out dodgy alternatives.

Keeping in mind, once a question has been closed and reopened you're still blocked from voting in the future - the close "wars" of the past will remain firmly in the past.

There's already been a fair bit of craziness when it comes to close votes. The limited number of votes per day stops being a motivation to prioritize what you vote for when your vote essentially lasts forever - even if no one else cares, you can't really feel like you've wasted it when it's still hanging there, ready to mean something as soon as four others show up. Well, that's no longer a possibility: either your vote will mean something, or it won't, and you have to decide up front whether a given question is worth it.

6
5

I like this. Honestly, though, the main reason I like this is the first point:

Start aging votes after 14 days, regardless of view count.

I can only begin to imagine how many questions there are that have a couple of close votes sitting on them that will never get to 100 views. Since people's close votes will age off of questions more quickly, I'd hope that this would potentially encourage people to downvote instead of using VTC as a "super downvote"—downvotes never decay! :D

The other part I'm actually a little more skeptical about.

Allow re-casting votes that age away after 14 days.

This is... nice, I guess? For the questions that have close votes that truly age away, I find it extraordinarily unlikely that camping out on the same question and re-voting to close it just to give it another round in the queue is really going to become a common occurrence. Most of these questions will be lost to the system.

This isn't really that awful, though questions which aren't closed cannot be Roomba'd. I'm just skeptical that anyone is really going to make use of this feature unless they miraculously stumble upon the same question weeks later by complete accident.

When it comes to obscure tags, the only surefire method I've found of getting a question closed it hopping into chat and bugging people to until it gets handled. Perhaps I'm wrong, but I can't fathom this making that any less true.

5
  • I lose your point. So you will have, instead of a Q with 3 VTC and score -1 a Q with 0 VTC and score -4. What have you gained?
    – yo'
    Feb 26, 2015 at 11:38
  • 14
    I can't count the number of times I've searched for the same question and found the same stupid duplicate instead. These days I can just fix that - but if I had to vote and then see my vote aged away every time I came back, I'd be pretty bummed. We've seen a fair number of flags from this.
    – Shog9
    Feb 26, 2015 at 15:57
  • @Shog9 Yeah, it doesn't seem like a destructive change in any way, and I can see it being nicer than the alternative, I just don't know really think it feels like a very major adjustment. Feb 27, 2015 at 6:15
  • 10
    There are lots of times when we encounter questions in the Stack Overflow review queue group that we can't VTC because we did once, the vote expired, and the question found its way back into view of a discerning user.
    – TylerH
    Feb 28, 2015 at 2:59
  • This would also happen much more on the smaller sites, you will come across the same question.
    – Tim
    Apr 11, 2015 at 16:33
-3

As I read the question, I wondered:

If a question is never read, why does it matter whether or not it's closed?

I'm pretty well convinced that huge swaths of questions can be safely ignored. They don't need to be closed because they just aren't being viewed. In fact, the close review cycle means that casting close votes gives many questions more attention than they deserve. If so, there's really no reason protect close votes from aging based on some arbitrary number of views. Part 1 of the proposal makes a ton of sense in that light.

Part 2 is odd. One of the nice things about voting (of all sorts) on Stack Exchange is that you can fire and forget. If you see an awful answer, just downvote and move on. If you see a question that just isn't clear enough to answer, vote to close and move on. If you think you can help improve the post, you can edit and/or comment, but that's entirely optional. You've had a chance to make your opinion known. If others agree with you, the answer will get pushed down the page and the question will be closed. If others disagree with you, there's no benefit to worrying about it: you've done your duty.

But by giving people a second (and third and etc.) chance to put a question up for review, there is something else you can do about a close-worthy question: come back and vote to close every fortnight. In other words, the second part could undo the benefit of the first part if a small number of people take the change as licence (or even obligation) to keep a personal "hit list" of closable questions.

It doesn't even have to be that calculated. If someone monitors the unanswered questions tab of a tag, they might routinely re-encounter a question they've voted to close in the past. Instead of the mild annoyance of finding out that their close vote has expired worthlessly, this change could potentially lead to what some might call insanity. It could be difficult to differentiate that behavior from abuse.

You mention duplicates as particular annoyances. The saving grace there is that close votes generate a comment pointing to the duplicate. That comment doesn't age away, which means that there is still a (admittedly feeble) breadcrumb trail to follow. Duplicates also are a special case of closing that could easily be handled other ways. For instance, I'm intrigued by the idea of using an upvote of the "possible duplicate" comment to move the process of closing forward.

But in the end, I think the package is a positive improvement in the close system. Any problems we see with allow re-votes to close should be easily discoverable and can be addressed if they actually do occur.

2
  • 16
    "If a question is never read, why does it matter whether or not it's closed?" It matters for a few reasons. 1) Because some of us want to keep Stack Overflow clean, and help to guide users' behavior by closing poor questions. 2a) Because Stack Overflow puts a heavy emphasis on quality, and so 2b) I shouldn't have to wade through fifteen pages of crappy questions to find the one good one; I should be able to close fourteen pages-worth of those questions so that, in the future, a new user or passerby can find the good content much faster/easier.
    – TylerH
    Feb 28, 2015 at 2:57
  • Been here, done this, accidentally. When monitoring an unanswered list, I close-voted a question I'd hit previously & forgot. Was surprised by receiving You have already voted to close this question, but you may vote again in X hours. See Apparently I'll be able to close-vote the same question again.
    – Mogsdad
    Jun 30, 2015 at 19:50
-9

Please, allow VTC recast only on questions that have a low score (whatever does that mean, a threshold of <= +1 or <= +2 would be good IMHO, but may well depend on the site).

Without this, you may have people who dislike a certain user or a certain question and will try every 2 weeks to close the question. Given how careful are reviewers sometimes, they can easily make it through.

1
  • 25
    using close votes as a form of harassment is abuse. If someone makes a nuisance of themselves this way, they'll get the same treatment revenge-voters get.
    – Shog9
    Feb 26, 2015 at 16:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .