15

I just went through an item in the Low quality posts review queue in Stack Overflow en español. It sounded strange to me to have it there, since its timestamp was from around September. In any case, I voted to close and then went back to see what others had done, to confirm it wasn't anything weird. The thing is... it was! Not in the sense I thought about, but because I was the 15th person to review the item, as you can see in https://es.stackoverflow.com/review/low-quality-posts/230955

enter image description here

Since this was unexpected, I tried to see what post this happened to, but I cannot get more information. For this, I reviewed another item and the behaviour was the same:

enter image description here

Vote to close and now when I check the page it shows a lot of people in the review list. Apparently I was the 21th to review, but it still shows as 'needs more reviews': https://es.stackoverflow.com/review/low-quality-posts/214264

enter image description here

The question in the review queue has apparently gone through different Close votes, but the links in its timeline doesn't provide further information about them (it just says 'This item is no longer reviewable.' in one case).

3
  • 1
    This is actually a design quirk/flaw of this queue. I'm posting an answer to explain why. Nov 5, 2020 at 11:37
  • It's not an audit?
    – Mast
    Nov 5, 2020 at 12:01
  • @Mast I thought so, but it does not show any signal about being it. In fact, the original post is still open, so it is very unlikely to have gotten into the LQP review queue as an audit. Nov 5, 2020 at 12:03

1 Answer 1

16

tl;dr: this appears to be happening because no one is reviewing the item in the Close Votes queue, and not enough people with close voting privileges are reviewing or voting to close the post.

When it comes to reviewing questions in the Low Quality Posts queue, there are three actions one can take: either find that it looks OK, edit it to fix problems with it, or recommend it for closure (maps to a close vote if you have enough rep to cast close votes).

As per When is a review task "completed" or "invalidated"?, the number of Looks OK reviews required to dismiss the task with an outcome of Looks OK is equal to the number of active "very low quality" flags from users (which don't expire), plus 1 (2 on English SO). Also, only a single Edit review is required to dismiss it. But the answer doesn't state anything about the number of Recommend Close reviews required. And that's because those reviews don't actually contribute toward dequeueing the post.1

This is also confirmed by this excellent flowchart published by an employee a while ago, which is still quite accurate to this day (with only one exception for most of the time, and two as of last week's revamps to Triage). I've zoomed in to the relevant part of the flowchart below2:

"Stays in LQ until final review met, but also sends to Close Review"

Notice what the circled box says: if a post there is reviewed as Recommend Close, it "Stays in LQ until final review met, but also sends to Close Review". In other words, it has no effect on its dequeueing from the LQ queue. However, it does have another effect: it puts the post in the Close Votes queue. The option has the effect of casting a standard close flag on behalf of the reviewer. That queue treats it that way as well.

Close flags and votes expire after 14 days, and close review tasks become invalidated if all active close votes and flags expire.

Let's have a look at the second question as an example. We'll examine the question's timeline, and piece the events together with the review task's events. Before continuing, I should mention that due to a bug, the review queue conflates the display of "Recommend Close" and "Close" reviews:

  • On May 26, someone reviewed the post as Recommend Close. This spawned a Close Votes review for the item.

  • 14 days later, on June 9, as no one else had cast any close votes or flags, that flag expired, and the review was invalidated. You can see in the LQ review's history that there were no close flags cast within that 14-day period.

  • On June 28, the post received a second Close review, which spawned another CV review, which received no reviews.

  • The LQ review continued to get more and more close reviews in the LQ queue, along with possibly external close votes or flags, at a rate of at least one every 14 days, enough to keep that second CV review alive and not invalidated. In the meantime, the CV review task didn't receive any reviews from anyone.

  • On October 24, the second close review was finally invalidated after a 14-day break in flags.

  • One hour ago, you made another Close review, and this should soon spawn a third CV review task.

So, to put it simple, despite the heavy preference toward closure from <3k users, not enough people with 3k+ rep came in and voted to close it, and of the very few who did, their close votes kept expiring. As Recommend Close reviews in Low Quality Posts don't count toward dequeueing the item in any way, the review stretched on indefinitely with nearly two dozen reviewers.

I...think this is a rather rare edge case coming to bite. This definitely shouldn't end up being drawn out like this, and there should be something to stop it. I think there should be a flag to let moderators know if the number of reviews for closure in LQ exceeds a threshold, with no closure.

However, based on the way the system is currently configured, there's no issue here, this is how the queue is designed.

Update: After having looked through the database for other examples of such reviews, I found one example that had been reviewed by 25 people in favor of closure, but received a single Looks OK review at the very end. That single Looks OK review was enough to dismiss the task with a Looks OK outcome, even though there were 25 other people who didn't think so. I'm aware that there's an automatic moderator flag raised if a similar situation happens for answers, but based on the fact that no moderator took any action toward the post, I can't say certainly if the same is raised for questions. If it isn't, that's a massive, gaping flaw, and should be fixed.


1 For a time, that post said that six Recommend Close reviews tasks would dequeue it, but that's not actually the case. I edited that text in back in March 2018, just after I gained the ability to review Low Quality Posts anywhere on the network, and my knowledge of the network was more limited then.

2 This flowchart does oversimplify two details, first, that first posts can also go through that queue in addition to First Posts, and second, that not only questions with low quality score go to the LQ queue, but also questions with high enough score that are flagged as "very low quality" - that's basically telling the system, "hey, you were wrong about judging this as being of good quality enough, this should be treated as not meeting the threshold". But those aren't relevant here.

3

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .