Proposed contest for close review queue posse [closed]

Question

Many hands make light work.

I propose that we offer a coveted Stack Exchange coffee mug to the top 25 people who work to clear the close queue in the next 45 days, and a USB rocket launcher to the top reviewer.

(image from Hyperbole and a Half)

This will do two things.

1. It will clear the close queue. Yay!
2. It will renew interest in community moderation, which I feel has been lagging a bit lately.

Accuracy of reviews can be assured by the degree of concurrence with other reviewers. No SE programming is required, only a few database queries.

Answer 1

No one can win this contest. The limit of 40 reviews per day will cause approximately one bazillion of us to arrive in a dead heat.

Answer 2

I've all but given up on the close votes queue on Stack Overflow. It's psychologically damaging attempting to make a dent as you can't tell whether you actually are or not.

At the time of writing this question has 39 upvotes and 226 views. 39 people can make a dent of around 17.5k in the queue over 45 days, which'll keep it under control. 226 can clear it completely and deal with any increase.

Based on the assumption that people are going to read this answer, I'm going to make a start. I promise to use up all my close votes every day I have access to a computer for the next 45 days - mugz or no mugz (though I do need some!). We don't need anything special to do this, we just need to get over the fear.

Join in and let's beat the people polluting the site or save those who aren't.

Close all the things!

_{P.S., the people who win the mugs will be the people who vote to close the most questions get deleted the same day; giving people with more than 20k an unfair advantage...}

Answer 3

The current queue is 82.2K

It is growing at roughly 10K per month (observational, may not be accurate)

So you're looking at about 97K in 45 days (let's rough up to 100K)

So at 45 days, 50 reviews per person, that's only 222 (5 votes per post) people beyond what is normally closed in a given day. I'd say it is definitely doable.

(And this doesn't even take into account the fact that many posts will have close votes already - assuming an average of 2 existing close votes, that brings the number of people down to 133 of 17,000+ eligible close voters).

Answer 4

Firstly, I am a lo-o-ong way from being able to participate in closing posts. I am still something of a noob around here.

Secondly, I have wa-a-ay too many mugs.

In my thinking, the system got like this because of a flaw in the system design, so short term fixes will be just ... short term.

The key change is to automate more of the close process. From my observation many questions on the queue (how many?) have no answer marked correct AND the question is down voted. So I suggest to auto-close

• All posts with > 2 close votes AND no answer marked correct AND the question is down voted AND question is > 1 year old.

• All posts with > 3 close votes AND no answer marked correct AND the question is down voted AND question is > six months old.

This change would close the (mostly) no-brainers, and it still relies on the community to flag and down vote posts. The exact parameters, such as the question age, can be fine-tuned for optimum results. And the change should be phased in with gradually tightening criteria over six months, so that mods are not swamped by complaints.

Auto-closed posts should marked as such, so that the OP knows why and can challenge the closure (pun).

Answer 5

How about no reward at all?

Temporarily drop the rep limit to 2,000.

Provide notes on what is expected of a Close Review.

Allow people who have a mind to to just plough into them (in a considered way, after reading the notes provided).

Nothing towards badges. Nothing towards anything. Close Review returns to 3,000 once some manageable number are outstanding.

No incentive for robo-anything. Simple mistakes in review are reduced in effect through requirement for multiple votes anyway. The standard of the notes provided (pops up when hovering somewhere, or a click, or whatever) reduces the number of simple mistakes.

If going ahead with any strategy, some investigation into what the major parts of the problem are, and which type of questions (if such can be decided) should be tackled first would be a big help, and motivational to see the length of the queue start dropping at the beginning of the task.

And not to forget, analysis of how it came about in the first place, and remedial action on that. I've seen a comment that the backlog existed, just was not exhibited, before the Review Queue system existed. Fair enough. But that does not explain 25,000+ increase in the last eight months or so. Or does it?

Without something to stop it happening again, it'll happen again. Then expecting The Community to do anything about it will be really demotivating.

Answer 6

I think a contest is the right approach, but it needs to be done in a big way.

First, set an amount of time, I was thinking two months, though 45 days is fine. If you do that starting next month, you can end things at the end of the year, but that's not crucial.

Second, you give out one-time permanent badges for the contest commemorating their achievement forever. Let's say for 250, 500 and 1000 closes in that timeframe (obviously you should probably remove the review limit for the duration of the contest).

Third, you give out either rep or even specialer one-time ranked permanent badges for the top 3 closers.

Make it as big as the Winter Bash Hat thing, go wild.

EDIT: Now that we're into the 2013 Winter Bash, we can really see the incentive even just a temporary award creates. The close queue has really been decreasing. Lets plan for a big closing contest this next year with some of the permanent rewards I mentioned.

Answer 7

I think envolving the comunity in this kind of event is fun and good for the "community feeling". If the queue goes down some 10~20K its already a common victory.

I write this to ask, again, to raise the close limit during the contest. There will still be audits "on", to control the quality (If the audits stop or not Robo-reviewrs is another discussion). This will also be a good opurtunity to test if raising the limit is useful to get down that >80K number by checking the stats after this.

Contest? Yes!
Raise the close vote limit during the contest? Yes!

Answer 8

The problem with the accuracy factor: doing an "edit" will technically mean that everyone disagrees with you. Will there be a special case for editing or something like that?

Answer 9

Bad idea for one main reason:

Users are already robo-approving for essentially no reward.

What would happen if you give them actual stuff?

We'd end up with a whole lot of questions that are either open and should be closed, or closed and should be open.

And honestly, I doubt it really matters if you resolve ties based on accuracy or what. Many users will just try regardless. Some will get banned, sure. But will enough get banned quick enough before they do too much damage? Probably not.

Admittedly I don't really know if robo-approvers are a problem on the close-vote queue, if it were, the queue probably wouldn't be so backed-up (although, a few robo-approvers probably won't make such a big difference in the numbers), but, if it isn't, this suggestion might cause it to become a problem.

Answer 10

I love the idea spirit. What worries me is possible unwanted side-affects of a contest.

• Already there are complaints about too many robo-reviewers. I suspect badges have a major part in the robo-reviewer behavior.

I'm afraid that an outright contest may bring out the worst of that kind of behavior.

• But, I agree that creating motivation is key. With over 18,000 3K+ rep users on SO, if they were motivated to perform an average of 2 review tasks each per day, the close queue could probably be cleared within a couple months.

Answer 11

I'd propose another metric to calculate the posse leaders, and the award...
well, the awkward is whatever the staff comes up with.

The idea is: in a extended period of time, say 6 months,
whoever casts the most final close votes, number 5 out of 4, leads,
and I think this should work out of the review queue.

For that we need a query like this: Get unclosed questions with closed votes.
_{And I'd love to know how to fork it to also filter by tags and body length, gimme teh codez, plz!.}

---

Of course this proposal favors me because I'm already doing that :P
_{^{of course not, rules start when rules start}}

Out of 50 daily votes, I spend about 40 in those results.
Average optimistic closure 200/week, 800/month.

IF the Question IS (out of my are of expertise) AND IS (long and/or have code in it), I skip it.
Otherwise, IF:

• "I have this idea -some explanation- what to do?..., hmmm, donnow

• What does $foo means in -any language-..., hello?, RTFM

• Plz, help me -because yes- and gimme teh codez bcz -more yes-..., ok, good luck

• Recommend me -where is the nearest WalMart- or sheiz..., ...

immediate closure.

---

Well, more than anything, I wanted to share the 40 daily-close-killer-votes technique.