80

There have been several proposals for how we’re ever going to get a handle on the ever-mounting close-vote queue. Dupehammer has been especially successful, and there have been several proposals since Mjölnir for awarding more close votes one way or the other, including one to give extra close votes, only accessible via /review and one to have a closehammer.

Here is a simple proposal to grant more close votes without any fancy logic.

Dr Seuss Cartoon of Tammany voting corruption

Proposal:

We should make close votes scale the way delete votes scale. Currently, you are granted 5 delete votes with the privilege, and then 1 more per 1,000 reps up to a maximum of 30 per UTC day.

With close votes, upon reaching the privilege you are granted 24 on most sites and 50 on large sites. But then—nada.

I propose that close votes be made to scale the same way that delete votes do. Your initial award should be same as the current one (24 or 50), but then you would be granted another close vote for every 1,000 reps past that.

Providing for voter fatigue

This sliding daily maximum should probably top out at some reasonable maximum so that Jon Skeet doesn’t end up spending all his time close voting instead of answering. We’d never get to 100,000,000 answers that way.

But to what maximum? More than it is now, perhaps 50 for sites getting below 50 questions per day on average or 100 for those getting above that.

Maybe.

See, I don't know for sure that it makes sense to have a different maximum for large sites than for small sites. Even though I'm sure the 24/50 split must have been triggered by something, I wonder whether it would be good enough just having more close voters as a large site would and have the maximum be the same everywhere, whatever that number is.

Don't worry about the details

The proposed figures given above are all just wags; it doesn't exactly matter what they all are. Maybe the initial award should be 20, or maybe the max by 100 and 200, or 40 and 80. That doesn’t matter. The important idea is that this be made something which scales with rep.

Postscript

Something vaguely like this has been proposed before, but only in an answer and only roughly. That suggestion was “chunkier”, and had no upper limit for Trusted Users. This one is more fine-tuned to smaller steps, the way delete-votes work.

  • 1
    related topic: meta.stackoverflow.com/questions/306126/… – rene Sep 17 '15 at 20:13
  • 1
    Shog wrote something about how no one uses all of their close votes. If that's the case increasing the number isn't going to make much difference... – ben is uǝq backwards Sep 17 '15 at 20:16
  • 18
    So the funny thing with that is that very few people use all their close votes, and almost no one uses them all regularly... But a fair number of people still hit them occasionally. So there's a big problem increasing the number of votes in response to voting (which is what was proposed there) in that you're not reducing frustration for folks who only need a lot of votes occasionally but you are encouraging burnout for the folks who are already at risk for that... Kinda counter-productive either way. This proposal has a few other advantages too, chief among them being that it is simple! – Shog9 Sep 17 '15 at 20:19
  • 15
    I can't judge for other sites but on SO the major problem is not the number of close votes per user, but the number of users that use their close votes. I will support any proposal that fixes that behavioral issue but if you only expect me to spend even more time handling that flood of crap I don't see how this would really help. – rene Sep 17 '15 at 20:20
  • 8
    @benisuǝqbackwards you better keep in mind that Shog knows very little about how things are at smaller sites (note this proposal mentions "24" votes limit, not 50 as at SO). Stats at Programmers show that many voters hit the limit - and these stats are terribly incomplete in that these don't account for users who force themselves to keep their last 1-2 votes for just in case if something awful pops up – gnat Sep 17 '15 at 20:20
  • 1
    @gnat The proposal does mention 24-vs-50, and considers whether there should be a different figure for large sites but has no answer to that. The other proposals were all complicated enough that it would be hard to justify the dev time to produce them, so I wanted to stay clear of too much complexity. But a largesite-vs-smallsite “spiking” multiplicative factor would be possible. I still wonder though. – tchrist Sep 17 '15 at 20:24
  • 1
    @Shog9 your observations are only for Stack Overflow, totally irrelevant here (note this proposal mentions "24" votes limit, not 50 as at SO). Please avoid making conclusions based on luxury features available only at the "main" site in the network – gnat Sep 17 '15 at 20:25
  • 1
    @tchrist at SO, this issue likely impacts quite a tiny fraction of users who are both regularly very active in close queue and as regular contributors in popular tags. They get only 10-15 votes left after the queue, that's like half of what's there on smaller sites. I don't know how they manage (maybe not bad at all, I myself have built sort of a "feel" after 10-15 thousands reviews in The Queue when I just spend what I have the best I can and stop worrying). But for content quality matters, making them "powerless" after runs over the SO close queue may be... sub-optimal, softly speaking – gnat Sep 17 '15 at 20:39
  • 6
    @gnat, this is what close voting looks like pretty much everywhere - there are variations, but there's no site where the right edge of the graph approaches the magnitude of the left. As frustrating as this is daily for a few people on a few sites, in practice it simply doesn't matter to most close voters on most days. That doesn't mean it isn't possible (or wise) to reduce the frustration for those few people... but that should be what we set out to do then; anything else will only lead to further disappointment. – Shog9 Sep 17 '15 at 20:44
  • 2
    @Shog9 could you look at the proportion of active (cast any vote within the past week), eligible close voters on a site to the proportion of them that use greater than 75% of their close votes during that week? I.e. If you have active 283 users who can cast close votes on the site is quite different than if you have 25,000 users who can. (283 active users on P.SE with 3k rep; 25k on SO). As an aside, could you generate that graph for Programmers.SE? – user213963 Sep 17 '15 at 20:46
  • 4
    @gnat yeah, the SOCVR room is picking up momentum and the number of cv-pls there combined with my CVQ handling makes that I don't touch the things like the triage queue or any of my regular tags with close votes at all. – rene Sep 17 '15 at 20:50
  • 3
    I'll write up a full report for this eventually, @MichaelT - hence the review status tag. The Progse graph looks like a tipped-over hangman game. – Shog9 Sep 17 '15 at 20:54
  • 3
    @TylerH I’m not trying to reduce anything, so the starting point would be the same as now. Clarified that the proposal is to award more votes with more rep, but not to start off with fewer of them than anyone has now. – tchrist Sep 17 '15 at 21:37
  • 3
  • 4
    @Shog9 Now that we’ve passed the proverbial 6–8 w̶e̶e̶k̶ month implementation timeframe, has there been progress on this status-review proposal — and if so, would you mind letting us all in on any results of those deliberations? Pretty please with fresh tomatoes on top of it? – tchrist Jun 11 '16 at 17:36
29
+100

I support this idea. I'm one of the highest-rep users on EL&U, and I vote to close when I see bad questions. But currently I stay away from the queue for two reasons:

  1. There are simply too many questions now that need closing. I look at the mile-high mountain of crap in the queue and then at the tiny little shovel I have, and I wonder what good it would do just to burn all my close votes and still have most of the mountain yet to shovel.

  2. On the occasions when I do burn through all my close votes, invariably I run across something really, really awful that I wish I could vote to close. But no can do.

Both circumstances make me cross.

Edited to add a third category

  1. The queue gives you questions in no particular order. If I were able to look over all the questions and choose the worst for closing, I might feel different about this. But since I might potentially spend my votes on the shallower end of the pool and have nothing left by the time I reach the deep end, I am forced for prudency's sake to conserve, and therefore to close questions in the wild instead of the queue. Yeah, I know, mixed metaphor. They happen.

A Little Extra

I just realized I have 30 delete votes on EL&U, but only 24 close votes. That seems strange. Surely delete votes should be rationed more carefully than close votes?

  • Are you referring to the ELU close queue, or the SO queue? ELU's seems fairly small to me... 200-odd right now, and only one or two reviewers have finished their daily reviews. – Nathan Tuggy Sep 17 '15 at 22:34
  • 5
    @NathanTuggy: ELU. 200 questions is still an order of magnitude more than I have votes to deal with. – Robusto Sep 17 '15 at 22:49
  • 5
    @NathanTuggy When you only get 24 votes and the review queue eats 20 of them, you are awfully darned leery of running the queue. So it isn’t right to say that most people haven't "finished". I run out of close votes constantly, but I almost never do so through the review queue. And I say that as the top CVRQer, too. Who wants to run out of all their votes in one pop? ELU has a daily close rate that is under half its overall close rate, perhaps for lack of close votes, perhaps for other reasons. – tchrist Sep 17 '15 at 23:08
  • @Robusto: Well, sure, but being able to take a good solid whack at fully ten percent of the total backlog doesn't seem so terrible, even if you're only contributing about a quarter what's needed for each of those. – Nathan Tuggy Sep 17 '15 at 23:27
  • 5
    @NathanTuggy: The point is, when you only have chump change you can't afford all the projects you want to take on. It can be dispiriting. – Robusto Sep 18 '15 at 0:37
  • Don't the queues give you questions sorted by how many reviews they've already had, and therefore generally the most known to need closing? – Nathan Tuggy Sep 18 '15 at 22:28
  • I don't find that to be the case. Sometimes the worst ones are farther in. It's software, it's not perfect. – Robusto Sep 18 '15 at 23:23
  • @Nathan IIRC that only happens in SO. – M.A.R. Sep 18 '15 at 23:50
  • @inɒzɘmɒЯ.A.M: I believe it's enabled everywhere, but the only ref I can find is not terribly specific: meta.stackexchange.com/questions/217849/… – Nathan Tuggy Sep 19 '15 at 0:16
  • @tchrist, robusto: Judging by the aged-away close reviews, it looks like all you guys need is 1 (one) more reviewer throwing 20 reviews (and therefore about 15 close votes) at the queue every day for a month or less, and you won't have a queue any more. It's awfully tempting to solve that with a pep talk, not a software change. – Nathan Tuggy Sep 20 '15 at 1:22
  • 2
    @NathanTuggy: We write software in part so that we don't have to rely on pep talks. – Robusto Sep 20 '15 at 2:35
  • Oooh #3 is a really good point. Perhaps we could be shown the lowest-scored posts in the queue first. That might help make it a bit more fun. – Lightness Races in Orbit Feb 14 '16 at 12:08
20
+50

TL;DR

enter image description here

(Access to the close queue should remain at 3 k reputation.)

Motivation

Let me begin with quoting two of Shog’s comments¹²:

So the funny thing with that is that very few people use all their close votes, and almost no one uses them all regularly... But a fair number of people still hit them occasionally. So there's a big problem increasing the number of votes in response to voting (which is what was proposed there) in that you're not reducing frustration for folks who only need a lot of votes occasionally but you are encouraging burnout for the folks who are already at risk for that... Kinda counter-productive either way.

 

this is what close voting looks like pretty much everywhere - there are variations, but there's no site where the right edge of the graph approaches the magnitude of the left. As frustrating as this is daily for a few people on a few sites, in practice it simply doesn't matter to most close voters on most days. That doesn't mean it isn't possible (or wise) to reduce the frustration for those few people... but that should be what we set out to do then; anything else will only lead to further disappointment.

As apparently a lot of people are willing to cast a few close votes per day, a better way to tackle big close queues would be to increase the number of people that have some close votes to begin with. Have the number of close votes increase with reputation (as the question suggests) but instead of starting with 24/50 at 3 k, end there.

Users without the close privilege can only flag closeworthy posts they encounter in the close or low-quality queue (similarly for Triage and H&I, AFAIK). By giving them a few close votes, a little bit more weight is put on their reviewing work and they can slowly familiarise themselves with the close privilege. Something similar applies to medium-reputation users who just monitor/browse a site/tag.

By slowly increasing the amount of close votes, possible abuse by users with lower reputation (i.e., less trust) is limited and responsibility increases gradually with reputation.

Proposal

  • Give a small number of close votes (say, three) to users who acquire the access review queues privilege (500 reputation on graduated sites).
  • Increase that number of close votes continuously until 24/50 at 3 k reputation.
  • Do not grant the access to close and reopen queues before 3 k reputation.

Optional: Making things complicated

I am aware that keeping things simple is a central desire, but here are a few more complicated ways to address possible concerns, in particular in respect to abuse:

  • Do not allow more than two close votes on a question from users with less than 3 k reputation.

  • Allow access to these close votes only from the review queues.

  • Make the whole feature hidden, i.e., do not change the interface in any way, but count closure flags from users with less than 3 k reputation as close votes (if they have close votes left for that day).

  • 2
    That's not a bad idea, either. – tchrist Sep 27 '15 at 16:47
  • 2
    I think we've got a solid candidate for bounty reward. :) Not that I support this idea (yet) - it is too novel to me. But it makes a fantastic food for thought – gnat Sep 27 '15 at 18:03
  • 1
    A good idea; it makes for an excellent broadening of the base of moderating users' pyramid. – Deer Hunter Sep 27 '15 at 19:30
  • 1
    Interesting. I think this deserves serious analysis. – Nathan Tuggy Sep 27 '15 at 22:40
  • Have you considered the implications of what this would do to /review if you had people adding more questions to it? Alternatively, given Stack Overflow's experience with triage, consider incorrect 'leave open' reviews (from a population base that outnumbers the 3k+) and how that would make it even harder to close crap (now you have to dig around in 10k tools or find it). Third complicated point is way to inconsistent and surprising in what would be shown ("I flagged these two, but show up on that close list but not this one an hour later") – user213963 Sep 28 '15 at 19:50
  • @DeerHunter consider that that pyramid is going to be adding significant numbers of votes from people who currently say "looks good" on a StackOverflow triage post that is completely unclear (rather than flagging it as 'unclear'). – user213963 Sep 28 '15 at 19:53
  • 3
    Have you considered the implications of what this would do to /review if you had people adding more questions to it? – The suggestion would not empower anybody to add items to the review queue, who does not already have it (everybody can flag to close). If making this power more visible has a big effect, we should rather worry that users do not know about it now. – Wrzlprmft Sep 28 '15 at 20:13
  • 6
    given Stack Overflow's experience with triage, consider incorrect 'leave open' reviews (from a population base that outnumbers the 3k+) and how that would make it even harder to close crap – Note that it is an explicit part of the proposal not to grant anybody below 3 k access to the close queue and thus the power to cast leave open “votes”. – Wrzlprmft Sep 28 '15 at 20:14
  • Fair 'nuff on those two points. Though given that the vast majority of users aren't using their close votes currently, and the vast majority of 500 or so rep users are severely miscategorizing posts with the access to the review queues that they have, I do not have great confidence that the users granted close votes through this process will substantially improve the underlying deficiencies in community moderation. Though what would be most telling is stats from SE about the rate of flags from these users, and the percentage that are helpful. – user213963 Sep 28 '15 at 22:42
  • 1
    @MichaelT: A few years back, 73% of individual close flags were correct on SO. 98.28% were helpful by resulting in at least one vote. Presumably this has oscillated a bit, but the general trend seems favorable. – Nathan Tuggy Sep 29 '15 at 3:23
  • 1
    @NathanTuggy: I wonder how many of those flags that were not correct just expired. So does 73 % correct close votes mean 27 % incorrect close votes or much less? – Wrzlprmft Sep 29 '15 at 6:04
  • @Wrzlprmft: Probably less, although it's hard to be sure. (And, of course, the number of wrong close votes on existing reviews is probably in the double digits; single wrong votes don't do much.) – Nathan Tuggy Sep 29 '15 at 6:15
  • I like this idea. For example I don't need a higher cap since arriving at the cap is a nice signal for me to stop doing this and doing something else (and I only seldom hit the cap). I would rather do a few close votes regularly then a lot at once and I doing a lot every day - I would feel overworked. This is nice for getting to know the close vote review system. – Trilarion Apr 30 '16 at 13:24
6

Make votes cast on tags where there are relevant tag badges only cost a fractional part of the vote pool for a user.

  • A close/reopen vote cast by someone with a relevant silver tag badge costs 0.75x votes
  • A close/reopen vote cast by someone with a relevant gold tag badge costs 0.50x votes
  • A close/reopen vote cast by someone with a close/reopen vote reviewer badge costs 0.9x (in the queue only?)
  • A close/reopen vote cast by someone with a close/reopen vote steward badge costs 0.75x (in the queue only?)

If someone has a gold tag badge and silver reviewer badge, votes within that tag would only cost 0.45 votes.

This acts as an outright grant for people with the close and reopen vote queue badges. Also gives the people who are active within a given tag more ability to cast votes within that tag.

While not extremely significant, it gives a few more close votes each day. Furthermore, since this works with the existing 5 votes to close, it is easier to tweak the cost factors (compared to other proposals elsewhere to increase the vote strength and leading to possible inconsistency of tallied votes at a later time).

  • 1
    The accounting is also not exposed to anyone except the voter, so it only has a chance of confusing those with the most system experience. – Nathan Tuggy Sep 29 '15 at 3:18
0

Wrzlprmft's recommendation is interesting. I'd suggest the following modified form:

  1. Start with a single close vote at 1000 points, when the user becomes an "established user".

  2. Add a single close vote for every 100 points, until 2900 points, when the user will have 20 close votes.

  3. At 3000 points, give access to the close queue plus 5 close votes. (Total votes: 25)

  4. Thereafter, add close votes at a different rate. (Should it be faster? Slower?)

Another idea:

  • At some number of points, allow multiple close votes on a single question, but max of 3 votes for a single question by a single user. Votes may be added one at a time or all at once. This way, questions that clearly should be closed aren't left open after getting stuck at 4 close votes. Also, abuse would be somewhat limited because at least two people would be required to agree to close the question.

  • For a question with less than five users voting to close (if multiple votes per question is allowed), then at least 6 votes are required to close. Examples:

    • 1,1,1,1,1 ⇒ 5 votes, 5 people ⇒ close
    • 2,1,1,1 ⇒ 5 votes, but only 4 people ⇒ need one more vote
    • 2,2 ⇒ 4 votes, 2 people ⇒ need two more votes
    • 2,2,2 ⇒ 6 votes, 3 people ⇒ close
    • 3,2 ⇒ 5 votes, 2 people ⇒ need one more vote
    • 3,3 ⇒ 6 votes, 2 people ⇒ close

    • 1,1,1,1 ... ?
      ⇒ a new voter can add one vote (1,1,1,1,1);
      ⇒ existing voters can add two more votes (3,1,1,1) or (2,2,1,1)

  • This basically amounts to...

    if (votes >= 6) or (voters >= 5); then close
-3

If the queue size fluctuates wildly it may be more convenient to simply let people "borrow" their close votes from tomorrow's or next week's vote allowance:

  1. I don't think that empowering fewer users with more privileges is better than broadening the base of actively moderating users.
  2. Not every community is like SO/ELU/Maths. These are the big critters in the Stack Exchange zoo, others have much lower queue and community sizes.
-10

How about we just ban new questions from <5k rep users until this is sorted out?

Then, with a clean slate, open it back up again whilst coming down hard on off-topic posts using the dupehammer and a closehammer feature, to stop the queue getting out of control in the future?

I'm serious!

  • 16
    Why not just ban all the users? No questions, no close votes. Perfect. – Shadow Feb 14 '16 at 12:28
  • @ShadowWizard: yourlogicalfallacyis.com/slippery-slope – Lightness Races in Orbit Feb 14 '16 at 12:28
  • Not more than yours. – Shadow Feb 14 '16 at 12:29
  • @ShadowWizard: Eh? That doesn't make any sense... – Lightness Races in Orbit Feb 14 '16 at 12:29
  • 1
    You think that banning all users is a slippery slope, I think disallowing questions from <5k (why not <100k? <1000000k, by the way?) is even more of a slippery slope. It's against everything Stack Overflow is and is just so ridiculously wrong on so many levels that I'm out of this useless discussion. Let's just agree to disagree. Cheers – Shadow Feb 14 '16 at 12:32
  • 1
    @ShadowWizard: No, I'm saying that your counter argument is an invocation of the slippery slope fallacy and is thus invalid. I never suggested disallowing questions from <100k or <1000000k. But I appreciate the polite, calm and constructive way in which you addressed my proposition. – Lightness Races in Orbit Feb 14 '16 at 12:45
  • This would work, but has no chance of ever getting implemented. – bjb568 Feb 14 '16 at 14:51
  • @bjb568 why? If this answer will have 1000 upvotes, even Shog will surely take a serious look into it. Who knows, SE has made crazy things. So far I'm the only one thinking it's a bad idea. – Shadow Feb 14 '16 at 14:55
  • @ShadowWizard Shog is a moderate anti-deletionist, there's no way he'd support this. It's unlikely that this answer will have a positive score, let alone one as high as 1k. – bjb568 Feb 14 '16 at 14:57
  • @ShadowWizard: It's not the first time I've suggested this, so I know it won't get implemented. But I stand by my suggestion :) Unfortunately, all the arguments against it are reasons that we have such a big close vote queue, so, like, what else can you do? – Lightness Races in Orbit Feb 14 '16 at 15:27
  • 3
    I have suggested before that questions on ELU start out closed and need to be manually opened. It's ridiculous, but some days I think it could work. – curiousdannii Aug 29 '18 at 15:10

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .