8

Currently review limits are set at:

20 reviews per queue per day if the queue size is less than 1000
40 reviews per queue per day if the queue size is 1000 or more

If there is a an increment of 20 votes for 1000 threshold, then isn't it reasonable that we have further increments for larger review queues?

If so, what would be reasonable?

Currently the SO close vote review queue sits at 8.7k

enter image description here

The question may arise that more delete votes may be needed for closed questions, although automatic deletion may alleviate that pressure.

If this is acceptable to the community, we could use the answers here to form a .

This question Scale daily review caps more effectively to bring even busy queues to 0 instead of 1000 discusses a similar concept, but has focus on scaling review limits between 0-1000 in an attempt to keep them closer to zero and has a specified algorithm. I am asking if the community does want to increase the reviews, then how should we increase them, example, an extra 10 per day per 5k?

| improve this question | |
  • There is no solution handed for the main problem: the number of votes. What makes this proposal better than the previous one? I am still in doubt whether to upvote or downvote this post since I think the problem should be addressed, but I wonder how this will make things better. – Patrick Hofman Feb 2 '16 at 15:31
  • @PatrickHofman well I have looked and there were multiple posts asking for the increase to 40, and it seemed like it would never happen and it did, it's just not enough to knock the longer queues on the head so to speak. As for comparing it to the previous one, this is not a proposal of an algorithm, but asking for feedback on how to formulate it. Whether it be a clear extra 20 per 5k – user310756 Feb 2 '16 at 15:34
  • If you're looking to tackle the SO CV queue, keep in mind that current review rates are sufficient to handle only about 2/7 the total incoming flow — the rest are quietly aged out of the queue before receiving many reviews. Increasing average reviewer load by 3.5 times or more is not, I think, practical. At all. – Nathan Tuggy Feb 2 '16 at 19:33
  • (Source of review queue numbers, from Sep 2015: ~850 handled, ~1700 aged out every day, with the latter slowly increasing.) – Nathan Tuggy Feb 2 '16 at 19:45
  • @NathanTuggy thanks for the stats. I don't think increasing it by 3.5 is wise, I was thinking more like 10 - 20 extra reviews per 5k – user310756 Feb 3 '16 at 3:10
10

I think it is clear the community wants it, there are numerous posts how to fix the issues with the review queues, especially on Stack Overflow.

The issue is, there are more than enough users to do the review tasks, but they just don't want to. There are a few users doing their share in reviewing posts, and that number is too low on Stack Overflow to get through the entire queue.

Whatever proposal is brought in to fix the queue, it has to start with commitment from the community. Do we really want to let the few reviewers out there do all the work? When we should elevate the number of reviews allowed, no matter what algorithm is used.

Is it fair to those few reviewers, and the rest of the community, to let them do all the work? How should we reward them? How can we get the part of the community that doesn't moderate involved to do their job?

2016-03-16: It seems they are running some test on Stack Overflow with increased reviews and close votes. Is this going to happen after all?

| improve this answer | |
  • 1
    I'm much more in favor of finding ways to involve more users in using their close votes. We need to scale out... – rene Feb 2 '16 at 15:45
  • Indeed. That is my problem. You can increase the reviews, or get the users involved that don't moderate at all. – Patrick Hofman Feb 2 '16 at 15:46
  • The thing is, yes it would be great to have more users helping, but we also need to keep the community that is striving so hard to achieve this the support and means to do so. The votes can always be peeled back if we get more community support. Trying to make a dent on that queue can feel futile. – user310756 Feb 2 '16 at 15:48
  • It would be great if we could expand on the questions you ask in your answer. Even if other users address these points in other answers here. – user310756 Feb 2 '16 at 15:51
  • Okay, what would be the key part of the question? 'How to get users involved in reviewing?' – Patrick Hofman Feb 2 '16 at 15:51
  • @PatrickHofman to me like an uphill battle on par with getting my teenage children to do housework :D recruiting reviewers that is. Maybe we have to accept that there is some limiting algorithm as sites grow, where the percentage of active people in terms of moderation doesn't grow in proportion with the site?? I don't know – user310756 Feb 2 '16 at 15:52
  • Sorry I didn't answer your question this > Do we really want to let the few reviewers out there do all the work? When we should elevate the number of reviews allowed, no matter what algorithm is used. – user310756 Feb 2 '16 at 15:59
  • Let me be clear about that queue size: it is already manipulated with the aging and filtering taking place. It is easy to get it close to zero, simply use different aging/filtering rules. Keep in mind that the team-up in the SOCVR only started to have some coordinated effort on a few tags. And that was in the time when the queue had 120K items and growing, not the wiggling around 8-9K it does now. I think that queue will never be at zero again. I do think having better filter / coordination of using close votes will make queue handling worthwhile. – rene Feb 2 '16 at 16:00
  • @rene it shouldn't have to be so difficult, that's my point.. I know it was much larger. We need to see a chart of it over time and with the increase of reviews per day to 40. – user310756 Feb 2 '16 at 16:12
  • 2
    Note that it appears SO is not the only site with trouble keeping the CV queue under control; dozens of sites have started seeing problems with scale, some with only one or two reviewers for queue lengths of 100+; CV is almost always the worst off, or even the only queue suffering scaling problems. Examples, last I checked, included Android, ELU, AU, SU, SF, and others. – Nathan Tuggy Feb 3 '16 at 3:22

You must log in to answer this question.