96

More and more frequently in the review queues, I am spending time either fixing a post or considering whether an edit is valid, etc, only to make changes and have them denied/not count because someone else has blindly hit the approve button in the meantime.

My proposal is for the system to "lock" a post in the review queue once it has been presented to someone for review. Do not let anyone else (in the review queue) do anything with it until it is released by the last user it was presented to (or a certain amount of time has elapsed).

By "lock" I mean that it won't show up in anyone elses list until released. I did NOT mean that it would be in multiple queues and be unactionable in some of them. This would fit well as the queues only present one at a time anyway.

This will allow people trying to make good decisions the time to do it and prevent someone else from making that effort a waste of time.

As support, look to some of the published reasoning behind the auto-question ban. A large portion of it is that bad questions waste the time of the community. Taking the time to ensure you are making a good decision in the /review queue only to have it not count because someone was badge-whoring and clicking the approve button as fast as they could also wastes the users time... arguably the more (for lack of a better way to put it) valuable user who is willing to invest some time and thought into making the site better.

14
  • 1
    I understand your grief, but the solution will cause annoyance to other editor (not reviewer) also. You probably can copy and paste your edit and go to the post directly to fix it.
    – nhahtdh
    Oct 9, 2012 at 14:20
  • 5
    Ah, my post wasn't clear, let me make a little change... I meant that it would not even be presented to anyone else until the last user it was presented to was done.
    – Barak
    Oct 9, 2012 at 14:22
  • 1
    It may not be accessible from review queue, but it is still accessible from the post directly. There is also the problem of balancing the locking timeout, so that genuine editor will not be denied of their work (if edit for too long), and malicious one cannot lock the post from editing forever.
    – nhahtdh
    Oct 9, 2012 at 14:25
  • 3
    Ah, just saw you disambiguated editor from reviewer. I'm more concerned about the /review queue... stuff is getting approved that shouldn't be, and I would at least like my "no" vote to show if the item is presented to me in the queue.
    – Barak
    Oct 9, 2012 at 14:26
  • 2
    I'm talking about editing in review queue, since it is the most annoying thing when you have done an extensive editing, then get the message that the suggested edit has been approved.
    – nhahtdh
    Oct 9, 2012 at 14:28
  • 3
    @nha, I didn't say it would be easy, and some experimentation would certainly be required to determine those things, but the system today is very discouraging for those who want to put some effort into it... this could be a step towards fixing that issue, and I think some effort should be made towards it.
    – Barak
    Oct 9, 2012 at 14:29
  • 2
    @nha... yeah, I get that and hate it as much as you do, it's what actually prompted me to post
    – Barak
    Oct 9, 2012 at 14:31
  • 1
    While I wholeheartedly agree on fixing the issue, I just don't feel like your suggestion will work out.
    – nhahtdh
    Oct 9, 2012 at 14:31
  • 3
    Heh, I guess we agree to disagree then. I cannot see how it wouldn't be a vast improvement. There would be issues as you mention, but I think the magnitude would be far less than what we have today with the current system. How many questions have you seen here on Meta ranting about this very thing? We can sit here and shoot down ideas all day, but if we don't try something, nothing will ever change. "Paralysis by analysis" I believe would fit here. (Aside: wow, I didn't think MSO even had the "move the discussion to chat" warning after seeing some of the comment threads on other posts).
    – Barak
    Oct 9, 2012 at 14:39
  • 1
    If this get support from other users, probably it will be implemented. Let's hope that I'm the only one so negative about this idea. (As a side note, I got the move discussion to chat with this message - the threshold seems to be higher on meta).
    – nhahtdh
    Oct 9, 2012 at 14:41
  • 1
    "It would be interesting to try exclusive review period to remedy this. By this, I mean that picked [review item] is taken off the queue for 2-3 minutes so that no one else can review it until timeout expires. This would guarantee that my decisions (at least quick ones) could only clash with those of the user(s) who held the [review item] longer than mentioned timeout - which would weed out mindless click-through robo approvers..." (quote source)
    – gnat
    Dec 25, 2013 at 19:07
  • 2
    possible duplicate of meta.stackexchange.com/questions/180216/…
    – Cole Tobin
    Dec 25, 2013 at 19:22
  • 2
    Related meta.stackexchange.com/questions/209349/… if the intention is to reduce effort which is in the end is needlessly expended. Dec 26, 2013 at 10:06
  • 1
    possible duplicate of The robo-approvers are killing my will to review edits Apr 4, 2015 at 3:52

4 Answers 4

28
+500

I agree completely, there's absolutely no reason good editors and reviewers trying to improve the Stack Exchange community should be overridden by stupid robo-reviewers looking to get a shiny free internet points badge. It's also really frustrating to pore over something editing it and then to have it say 'nope, people already reviewed this'

However, this may cause a few logistical problems, as well as slow down the reviewing process significantly. When as many reviewers are online as there are reviewing, and one person starts reviewing marginally before the other, chances are that someone will get that review while another person has it. I'm not sure how easily that could be fixed, but I'm sure that it could be fixed.

Also, it would slow down the reviewing process for smaller review queues. If only one person is allowed to review, and everyone else has to wait for a certain amount of time or until everyone else is done reviewing, there will be a lot less available reviews in the queue. While this wouldn't be a problem for most queues, I'm afraid that this would cause the CV queue on SO to spiral even further out of control..

Actually, upon further consideration, no this wouldn't. It wouldn't really affect the size, and could possibly speed things up for the CV queue, as more posts would be getting close votes at the same time, as opposed to a fewer amount of posts getting more close votes.

I think that a blockage of 3-5 minutes would be adequate for some queues, and 10-15 for others. You obviously need longer for edits, but if you're just flagging and deleting, you won't need much time. What would be better is it is locked until 1 or 2 minutes of inactivity by the reviewer.

Also, one thing that may work is that when someone is editing a post, it can't be reviewed for a longer period of time, 20 minutes for example, but if they're just flagging it, lock it for 5 minutes

I'm not sure if this is how it works already, but perhaps what we could do in the CV queue is rank posts by number of close votes, so the posts with 4 close votes get reviewed first, and therefore, get closed faster, reducing the number of posts overall in the queue. Of course, the best way to do it is just to get people reviewing as fast as they can, or to have more than one CV reviewing party.

3
  • 4
    This wouldn't slow any queue that was far from empty. While one question is locked for review there are thousands more which are unlocked and waiting Jan 15, 2014 at 12:26
  • 9
    "If only one person is allowed to review, and everyone else has to wait for a certain amount of time or until everyone else is done reviewing," - untrue. everyone else would only need to review something else, that's all. Unless queue is almost empty, but then slowdown is not a problem.
    – Mołot
    Jan 15, 2014 at 14:13
  • 2
    I was about to suggest the same as the OP - in order to speed up the process. Because the time spent by each reviewer leads to results in a percentage much closer to 100%. In the CV-queue I estimate that 50% of my time gets lost because I was reviewing a question that afterwards turned out to be already closed. Same holds for the edits: By the time I have reviewed the changes some other reviewers have already fixed that post, so the time is lost.
    – chiccodoro
    Mar 7, 2014 at 12:27
17

I am all for having some reasonable exclusive time to complete review.

Still, I would like to point that unrestricted locking - "until released" - comes at price. And this price may be quite high, especially in Suggested Edits and Close queues. Unrestricted locking means that particular reviewer can block everyone else from reviewing the post for an hour, or day, or more.

Think of an impact this may have on, say, suggested edits. Imagine a perfectly reasonable post that looks poorly formatted because of minor mistake made by poster, imagine an edit suggested to fix it being blocked by a reviewer who just opened it and went to lunch or sleep.

As time goes by, readers see all the same broken post and vote it down. When blocker-reviewer gets back at last and approves the fix, how much effort is needed to recover the post to "fair" score? Are all who voted it down expected to get back and undownvote? Isn't this too much of a hassle just to make it comfortable for a lazy SOB who just didn't care to perform their duties in time.

Similar damage could be there in close votes review. Imagine a close-worthy post that has a potential to be improved by editing into a better shape. Slow reviewer "holding a lock" may block others from closing it for an hour or day or more. While it hangs there open, it may get answers which in turn will make salvaging edits impossible. As a result, post that could be saved by quick close-edit-reopen, gets locked in a bad shape.

Summing up, I would rather prefer a review lock to be unconditionally released after some specified timeout. If reviewer can't complete their job in 3 (or 5 or 10...) minutes, lock is released anyway and other reviewers get a chance to handle it, too.

For the sake of completeness, timeout approach isn't completely painless, either. I sometimes stumble upon posts that take me a long time to review - timing the lock out means I will need to somehow complete review actions outside of the queue. These cases seem to be rare though and I believe it would be a reasonable compromise to let them happen.

3
  • 3
    I have basically stopped reviewing edits because of this (I did a few recently when the little counter started appearing). I think 5-10 minutes would be enough for the queues I have access to (<3k). Dec 26, 2013 at 10:08
  • 8
    Here's an example: stackoverflow.com/review/suggested-edits/3665130 I got to this 20 seconds after proposed. Grammar not improved. Code-formatting not significantly improved (if something of that insignificance removed the scroll-bar, OK, otherwise...) Took about 10 seconds. Reject. Gets me Already approved, go do some more work on it.. I don't mind not getting the extra review-count, but I do mind others getting the credit whilst I just get to spend more time on it. Dec 26, 2013 at 10:24
  • 4
    @PompousQWoodger that's understandable: currrent approach essentially punishes those who prefer to put thinking into reviews. I'd be comfortable with 5-10 min, too. I probably could handle 3 min and maybe even 2, but having like 20-30 seconds to complete review as of now sucks. Just sucks
    – gnat
    Dec 26, 2013 at 10:25
7

Tasks in some review queues are "locked" to a reviewer for a period of time:

  • First Posts
  • Late Answers
  • Suggested Edits
  • Triage (currently enabled only on Stack Overflow)

It should not be necessary for other queues, as a larger number of reviewers is generally desirable in order to complete the task.

All this locks is the dispatcher though; while no other reviewer will be assigned the task for [several minutes], there's nothing stopping anyone from visiting the question directly and taking action on it that invalidates the review, nor reviewers previously assigned the task from returning to it and completing it. This is, I think, a reasonable compromise between the previous behavior (which resulted in many review collisions) and a harder lock (which might easily result in tasks getting "stuck").

5
  • Is this also true for audits? Looks like two of us got the same audit at the same moment. This was in the CVQ...
    – rene
    May 6, 2015 at 19:04
  • Close queue doesn't lock, because it takes a lot of reviews to do anything... Normally. Audits may be a problem.
    – Shog9
    May 6, 2015 at 19:56
  • To be honest, at least here on MSE, it cause more damage than good, in my opinion. I rarely get to see suggested edits anyway, and once there is one, it's locked when someone first reach it. Would be great if it will be disabled here. Aug 27, 2017 at 6:58
  • That's not gonna be less of a problem without locking, @Sha - just means more instances where you spend time reviewing an edit only to have your review discarded because the edit was approved/rejected while you were thinking about it.
    – Shog9
    Aug 28, 2017 at 16:05
  • @Shog only on sites with very high number of reviewers, which currently is only Stack Overflow. On smaller sites, including here, there are only few reviewers at any given moment, so such thing is very unlikely to happen. e.g. on a smaller site it took two weeks for my edit to be reviewed. Here it takes about an hour in average. Aug 29, 2017 at 7:09
3

Agreed. What you describe is basically a model of pessimistic locking - which prevents reviewers from spending time on a post that afterwards turns out to be "done" already, which is a waste.

The CV queue would become much shorter for each reviewer, which is awesome, too.

A timeout as suggested by gnat would be necessary. 3 minutes are fair enough. It might take longer in some cases , but we would improve the situation for all the other cases.

Additionally I would suggest that there are actually N locks with a separate timeout each. Only when 5 concurrent locks are on the review it disappears from the other's queues. N would equal the number of votes required for the particular queue. (E.g. if 4 votes are required to close a question, 4 concurrent reviewers should be allowed).

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .