What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

I am under the impression that increased participation in the review queue is, on the whole a good thing. It encourages people to feel an ownership in the community and partially responsible for the site as a whole.

On Programmers.SE, I've seen what I believe to be an uptick in flagging to close (non-audits with no votes in the close queue). I believe that to be a good thing as it indicates to me that there are at least a few on P.SE who may some day become more active participants in the site.

On the other hand, I don't often see too many new names in the "recent reviews" page. There are often new names scattered about on close votes of recent questions which suggests to me there is a population of 3+k users who don't review, but do close recent questions.

While the close queue at P.SE is completely different than the one at SO (we start wondering about it in chat when it hits 20 and occasionally motivate people to help smash the queue back to 0), I was wondering if there wasn't something that could be done to help direct everyone possible to review (or at least be aware of it - there was a comment at one point of a 10k user who wasn't aware of anything in the /review link after clicking it as a new user and not seeing anything in it that could be done by him).

The suggestion

Next to the review link display the number of reviews that user is capable of doing at this time, akin to the way the 10k banner looks when there are flags or edits outstanding.

Many people perceive numbers as "there is something here that needs my attention". Inbox size, alerts on mobile device apps, notifications, etc... Using this it would be possible to point people to the review part of the application and help them become a more active participant in the site and the community as a whole.

This number would be:

min(first post queue size, first post audits doable) + min(late answer queue size, late answer audits doable) + ...


There seems to be some confusion about what this would be and what the numbers on it would be.

I have used my considerable art skills (sorry, no pie) to make a mockup of what this would look like based on the current snapshot of what I see on P.SE

Review on P.SE

"Wait," you say... "why is that a 20? Shouldn't it be 24?"

Nope. Because on P.SE I can only do 20 close vote reviews a day (smaller queue or something of that nature). If you hit it with someone who hasn't reviewed most of the queue, you'll actually see that its 60. And still the answer should be 20.

And so, min(60,20) + min(0,20) + min(0,20) + min(0,20) + min(0,20) (those are the other queues there too) is 20. And that's the number that should be shown.

This should be something that you can get to 0 every day. Making it something that is an unmanageable number makes it something people will ignore, and that's not what this is suggesting.

share|improve this question
1  
And us SO users would be stuck with the six-figure close queue all day. Way to go. –  michaelb958 Dec 14 '13 at 1:05
6  
@michaelb958'saSleigh I think you misunderstood that number. If you can do 40 close reviews in a day, the number would be min(106123,40) + ... which would show how many you can do, not how many there are total. If you did all 40 close reviews in the day that part would be min(106123,0) + .... I absolutely do not want anyone to see a 6 digit queue size. I want them to see something that the can see and deal with and make an impact on. –  MichaelT Dec 14 '13 at 1:21
4  
Oh. Oh, yeah, that makes much more sense. +1 –  michaelb958 Dec 14 '13 at 1:24
    
@michaelb958'saSleigh with that edit, does it make it something a bit easier to see? –  MichaelT Dec 14 '13 at 1:38
1  
Yep. Couldn't miss that if I tried. –  michaelb958 Dec 14 '13 at 1:59
add comment

4 Answers 4

up vote 27 down vote accepted

As of a few minutes ago, we are replacing the pending suggested edit count in the top bar with the number of pending reviews for diamond moderators and folks with the "moderation tools" privilege (i.e. 10k users on graduated sites and whatever that level's at on betas):

10K users ONLY

We are excluding the Close Vote queue from this on Stack Overflow, and there's a minimum of 10 pending reviews that have to be in the system before the indicator will show up.

It is also cached, so may or may not be very accurate depending on the amount of ongoing activity in /review at any given time.

share|improve this answer
1  
Nice work, Anna - I can already see it having an effect. –  Shog9 Apr 29 at 22:22
1  
@Shog9 new indicator seems to lie, no way on Earth can I have 58 review tasks at Programmers as of now (I checked all review queues and everything I could see in 10K tools). Looks as if it doesn't exclude questions that already have my CV –  gnat Apr 29 at 22:28
5  
Correct, @gnat: this is everything in review, even stuff that you, personally, can't review. Super-expensive to personalize this - so the alternative is simply not having an indicator. Think of it as... A reminder to encourage your peers to review. –  Shog9 Apr 29 at 22:29
2  
@Shog9 I'm curious what the long trend of this is (or even the reviews/hour after a few hours... ok, I'm impatient). As I said, we're trained by email and instant messages and the like to keep numbers in boxes at zero. –  MichaelT Apr 29 at 22:53
1  
Nine hundred and seventy four flags?? (Yes, that is what I take away from this post.) –  WendiKidd Apr 29 at 22:58
2  
@WendiKidd Welcome to moderating Stack Overflow. –  Anna Lear Apr 29 at 22:58
1  
@MartijnPieters Nope, it's on. If you're talking about meta, looks like it's back under 10 pending reviews, so the indicator doesn't show up. –  Anna Lear Apr 29 at 23:29
1  
@AnnaLear it is bugged. See meta.dba.stackexchange.com/questions/1112/review-queue-oddness –  FreshPhilOfSO Apr 30 at 2:01
1  
@FreshPhilOfSO Not bugged. The topbar count is every review in the system, not just the ones available to you (ie ones you haven't skipped or already otherwise acted on). –  Anna Lear Apr 30 at 2:06
12  
Oh. Terrible. Will drive me mad having a number there I can do nothing about at a certain point in time. Many thanks for replying! –  FreshPhilOfSO Apr 30 at 2:08
8  
This is utter crap and I find it extremely disruptive. Please discontinue this insanity immediately. If I can't act on something, I don't need to be notified about it. Period. Who comes up with ideas like that anyway? –  SvW Apr 30 at 7:47
8  
But the OP said, "This should be something that you can get to 0 every day. Making it something that is an unmanageable number makes it something people will ignore, and that's not what this is suggesting." I think I can appreciate the tradeoff you've decided to accept here, but now you have many questions to answer: meta.stackexchange.com/q/230742/246931, and meta.stackoverflow.com/q/252843/2829605, and meta.dba.stackexchange.com/q/1112, and meta.drupal.stackexchange.com/q/2973, and probably more to come. If it's good in the long run, more power to ya, but... –  Nick Stauner Apr 30 at 18:24
2  
This isn't exactly status-completed on the OP, and it's creating a very visible and confusing side-effect. I think the community would appreciate a little more explanation of what it has to accept as a result of this decision when you have the time. –  Nick Stauner Apr 30 at 18:26
2  
Not quite sure what you're after, @Nick - but read this and let me know if you have further questions. –  Shog9 Apr 30 at 20:35
2  
@MichaelT: Fair enough! I'm not unhappy with the change either, FWIW. In hope of making it more understandable and reintroducing some of its original responsiveness (i.e., when it was just for suggested edits), I've made an initial feature request about it. I'd be curious about other ideas for souping it up too. –  Nick Stauner May 1 at 1:53
show 17 more comments

You're absolutely right that we can do more to get new reviewers active in all of our review queues. We're exploring a bunch of possibilities, but it will take some time to figure out what the right approach is; the ideas we like most are much more technically complicated than the ideas we could do fast, so we're trying to find a good balance.

There are a couple of problems with just displaying the number of reviews that a user is capable of doing at this time. The most obvious one is that as your reputation and access to review queues increases, so does the number of items you're eligible to review; our highest-rep users (some of whom are among the most active reviewers) would have a notification that's always on and that they can never clear to zero. This means they'd likely wind up just ignoring it.

The second big issue is scale; with over 100k items in the close review queue on Stack Overflow, for example, that number obviously affects the perceived impact a user thinks s/he can have. Ideally, we'd find a way to alert users of urgent and actionable review items – i.e., notifying them of items they're likely to care about and have an impact on, and show them a number that they can clear completely, or at least see go down substantially as a result of their actions.

The notion here is a good one, but I don't think your exact suggestion is the best implementation. I'm not yet sure what is, but I wanted to let you know we're working on it.

share|improve this answer
    
Thank you for providing an answer that I can quote to the next guy who thinks the SE team is sitting on the problem or trying to bury it. –  Frédéric Hamidi Dec 13 '13 at 21:30
5  
You tell those people we're only sitting on the problem to keep it warm until it hatches, @Frédéric. –  Shog9 Dec 13 '13 at 23:42
7  
I suspect that I didn't make myself clear in this suggestion when reading the other comment to me (above). The number I would like to see displayed is the number of reviews that person can do. For close reviews this is 40 if none are done and 0 if 40 are done. That part of the 'total reviews doable would be min(106123,40) or min(106123,0) -- certainly not 6 figure numbers, but rather one or two digit numbers. –  MichaelT Dec 14 '13 at 1:23
6  
Now that... Is a refreshingly novel idea, @MichaelT. –  Shog9 Dec 16 '13 at 4:54
7  
@MichaelT Ah, yes, I did misunderstand your intent. That's an interesting idea; will definitely consider it in our planning. –  Laura Dec 16 '13 at 19:06
add comment

Meta discussions about site standards and accepted practices help to lay the groundwork for useful reviewing by explaining the philosophy and history behind the process. Since reviewing methods change over time, reviewers should be regular users of Meta so that they can keep up with the changes.

If we push a newly-anointed user into the review queue without that background, we run the risk of generating bad reviews. Proper training is important to ensure that the reviewers are in sync with the site goals and practices.

If we implement a strategy to get more eyes in the review queues, I think it should be accompanied by a strong push to acquaint reviewers with the Meta documentation on the subject.

share|improve this answer
1  
I agree with your points. The counter-point would be the suggested edits queue. A quick review of the custodian badge shows a number of low-rep, non-meta participating users being pushed to the queues. More training on sites norms is a good thing, but there is precedent to draw them in as MichaelT suggests. –  GlenH7 Sep 16 '13 at 17:44
2  
IMHO a good reviewer needs nothing else to do than applying good discretion and common sense and wise judgement. Rewiewers who dont visit meta and are more focused on the topic of the site than politics do often a more reasonable and fairer job in the review queues. Community moderation works best when people think about and judge thing independent and according to their best knowledge and judgement. If this results in differences to site policiese those policies are not good for the community.... –  Dilaton Dec 13 '13 at 23:38
add comment

If you want this feature for yourself now, you can install a user script to make it happen.

The following script that I wrote can be used in Firefox with Greasemonkey installed or in Chrome with TamperMonkey.

http://userscripts.org/scripts/show/184677

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .