8

TL;DR: Split the low-quality queue in two, separating posts with possible quality issues (autoflagged posts) and posts that may need to be deleted (NAA, gibberish), as having both of them together causes confusion and mistreatment of posts. To keep things simple, reorganise the FP, LA and LQ queues, while you are at it.

Looking at the triage and help-and-improvement queue over at Stack Overflow and the Meta posts on that, I sometimes think that this is planned already, but then again sometimes statements make me think that the idea has never crossed anybody’s mind. So, here I am, suggesting it anyway.

Three notes:

  • I will start of ignoring questions for this at first, as this only complicates the explanations.
  • I will also keep my proposal as close to the status quo as reasonably possible (except for the restructuring). This is not about the details, this is about the general direction.
  • This is not written from the perspective of Stack Overflow with its special needs. I do not see any argument why it could not apply there though.

Proposal

  1. Create two review queues:

    • Help and improvement (HI)– The interface and processing of posts in this queue are identical to the first-posts and late-answers queues with the only exception that a post may need more than one review to be dequeued.
    • Not an answer queue (NAA) – Interface and processing are identical to the low-quality queue except for the wording (which should be along the lines of what I suggested here).
  2. Introduce a needs improvement flag for answers saying something along the lines of:

    This answer can be severely improved by editing or the author needs guidance on answering.

  3. The queues are fed as follows:

    • First posts, late answers and answers flagged with needs improvement are sent to the HI queue and so are answers that would be autoflagged and sent do the low-quality queue nowadays due to heuristics. If a post is sent there for more than one reason (or is already in the queue), the number of required reviews is increased (details need to be determined, this request could be considered).

    • Posts flagged not an answer are sent to the NAA queue.

  4. Retire the first-posts, late-answers and low-quality queue as well as the very-low-quality flag or morph them into one of the above.

What does this actually change?

First posts and late answers receive the same treatment as before – only that one person cannot review the same post in both queues (and the low-quality queue on top). Correctly flagged not-an-answers also receive the same treatment. Answers whose only problem is a lack of quality, however, get sent to a queue that is not about deleting them but about improving them. Also, the very-low-quality flag gets removed, as it is pointless in its current form anyway (see below). Instead there is a needs improvement flag, which essentially sends a post to the late answers queue.

So essentially, I could have proposed a new separate queue for low-quality posts, but then I would have received complaints about yet another queue or wanting to reintroduce the 10 k queue. Also, restructuring seems like a good idea.

Depending on how the queues are organised internally, this may be rather easy to implement.

Why do we need this?

The current low-quality queue is about deciding whether a post is actually an answer and about improving low-quality posts. This is the cause of a lot of problems. The wording of this queue is mainly aimed at improving posts, but if a post is just low quality (e.g., a short and unelaborate answer), there is no proper action to take. Thus a post that gets autoflagged (due to heuristics) may be deleted because only the original author can improve it and nobody wants to click Looks OK. This leads to a lot of confusion, disappointment and disgruntling of new users, who sometimes just need a little bit of help to write proper answers. I wrote about this more extensively in these answers of mine.

Therefore posts that are autoflagged should first be sent to somebody willing to improve them in a queue that is not about deleting them (HI queue). If such posts turn out to be not-an-answers, they can still be flagged such there.

The very-low-quality flag was probably intended to say this post needs improvement at some time, and apparently some sites and people use it as such (see here and some of the comments and and answers on this question), but the current wording does not reflect this:

This answer has severe formatting or content problems. This answer is unlikely to be salvageable through editing, and might need to be removed.

According to my understanding, unsalvagable severe content problems are cat-on-keyboard or unparsable English – but those can as well be flagged as not an answer. I have no idea how unsalvagable formatting problems even look like; if I can identify formatting as the source of a problem, I can also solve it. So, there is no need for this flag in its current form. What we might need, is a low-quality flag that sends salvageable posts to those who have the time, temper and expertise to improve them – which is the purpose of the needs improvement flag.

Noteworthy issues

  • At what reputation shall the queues be available? The most straightforward solution would be 500 for the HI queue (same as first posts and late answers) and 2 k for the NAA queue (same as the low-quality queue). The main effect of this would be that autoflagged posts and posts flagged very low quality/needs improvement would now be sent to a queue which is available to users without editing privileges. I have no problem with this, as first posts often require edits as well and often for the same reasons, but one could also make these posts available later in the same queue (just as tag-wiki edits only become visible in the suggested-edits queue at 5 k reputation).

  • Answers that are autoflagged or flagged needs improvement are sent to a queue that allows to vote. I do not consider this a big problem, but if it turns out to be one, this can be fixed by not uniting the queues or disallowing votes for those posts specifically.

  • The number of queues would be reduced by one and so would be the number of times a given reviewing badge can be earned.

  • Autoflagged posts are not directly sent to a queue that is about deciding their deletion anymore. Thus, utter junk detected by autoflagging may take a tiny bit longer to be deleted (this may be counteracted by having a flag from the HI queue count like recommend deletion/delete from the LQ queue, if the user has sufficient reputation).

  • The needs improvement flag may do more harm than good as it allows users to lazily delegate the responsibility for a bad post they found instead of editing it themselves. On the other hand, sometimes you really lack the expertise, time or temper to edit a bad post yourself, in which case the flag would be appropriate.

What about questions?

The triage and help-and-improvement queues on Stack Overflow pretty much go into the same direction as what I propose, in particular by separating improving questions and weeding out questions without any value. Moreover, the problems described above do not apply that much to questions anyway, as the correlation between low quality and closeworthiness is much stronger for questions, the close queue and low-quality queue are separate and the very-low-quality flag for questions is hardly used in my experience (but even more pointless than for answers, IMHO).

It is however uncertain whether and how those two will be implemented on sites other than Stack Overflow. However the mentioned stand-alone implementation of the help-and-improvement queue on other sites pretty much corresponds to my suggestion, I would just unite it with the first-posts queue, as the two are mainly about the same thing anyway.

12
  • 1
    There should also then be a recommend deletion flag. And I'd argue that algo-chosen posts in the LQRQ should be deleted much more than edited.
    – bjb568
    Commented Aug 30, 2015 at 22:04
  • @bjb568: How would the not-an-answer flag differ from your recommend-deletion flag? As for autoflagged posts: That may strongly depend on the site. On the sites I am active on, most of them are just bad answers. Of course, one could have the algorithm choose the queue, e.g., send posts that look like a question to the NAA queue.
    – Wrzlprmft
    Commented Aug 30, 2015 at 22:23
  • There wouldn't need to be another flag for answers, but there should be a recommend deletion for questions.
    – bjb568
    Commented Aug 30, 2015 at 22:33
  • @bjb568: For what purpose? We do pretty well without one today. The VLQ flag for questions is only a long way to closure, as you cannot delete questions from the LQ queue directly (see this question of mine).
    – Wrzlprmft
    Commented Aug 31, 2015 at 4:54
  • VLQ speeds up the close->delete process for questions, and an option to also delete from queue would be even better.
    – bjb568
    Commented Aug 31, 2015 at 11:33
  • 1
    VLQ speeds up the close->delete process for questions – How is this faster than directly flagging for closure? This would very well answer this question.
    – Wrzlprmft
    Commented Aug 31, 2015 at 11:39
  • 2
    It puts the worst of the questions in a much smaller queue that actually gets cleared. Those with privileges can also favorite the questions to delete vote later.
    – bjb568
    Commented Aug 31, 2015 at 13:26
  • 2
    An excellent proposal. I find myself continuously annoyed by posts appearing in multiple queues simultaneously for no net value. And that VLQ has always been a completely useless form of dowvote replacement that noone knows how to use. A more constructive approach seems in order. Commented Sep 4, 2015 at 9:37
  • 1
    @bjb568: On SO, VLQ-flagging goes to Triage, which can't actually close questions by itself, except in rare cases where there are already two close votes and all three Unsalvageable reviewers have 3k. Everywhere else that I'm aware of, all queues are routinely cleared, with the exception of SU/SF. (I have no idea what's going on with those guys. Maybe they just don't have enough reviewers?) So it seems that the only cases that might be a useful workflow are the two secondary members of the Trilogy, maybe. Commented Sep 5, 2015 at 2:49
  • 1
    @NathanTuggy FWIW, English and Programmers often have a surplus of close reviews. Looking around. AskUbuntu is way up there. Wordpress has a bit of a surplus too. Drupal and android have some too. This isn't a trilogy only thing - a good number of sites have trouble handling close vote reviews in a timely manner.
    – user213963
    Commented Dec 8, 2015 at 19:52
  • @MichaelT: Hmm. At first, the number of active reviewers on those sites looked ridiculously low, and I was ready to write that off as a problem of engagement. But it appears that most of them have roughly 10 times as many reviews per 3k user as SO does (after counting the doubled cap). So this does seem to be a problem. Commented Dec 9, 2015 at 20:34
  • @NathanTuggy and thus the one of the core problems with the review queue and minimum number of (close) votes to act on something. There are several very different sets of sites that need to be looked at in their own right with the corresponding cultures of when to close and how active the core group is (you will likely find that on most sites, the majority reviews are overwhelmingly done by a very small group while the 3k pool is a bit more active than SO's). I believe that trying to find one solution for all sites will cause problems on many.
    – user213963
    Commented Dec 10, 2015 at 17:00

1 Answer 1

2

I'm not persuaded that this is an improvement, but explaining why requires an answer of some length. This proposal conflates two or three independent proposals:

  • Removing the VLQ flag.

  • Introducing the "needs improvement" flag and a "help and improvement" queue.

  • Separating out queues based upon reviewing criteria.

Each of those has its own pros and cons. Bundling them all together makes it hard to comment on your proposal, but I'll try.

"Needs improvement" flag

Let me give you some common-sense advice that eliminates the need for a "needs improvement" flag:

If you see an answer that needs severe editing: edit it.

Don't be lazy. Flagging it for someone else to edit is not helpful. There's nothing preventing you from editing it yourself. Anything the reviewer can do, you can do.

Philosophically, flags are for exceptional situations that the flagger can't handle on their own and that needs help from someone with more privileges. "Needs severe editing" doesn't meet any of those criteria. It's not exceptional -- I see all too many answers that this could be applied to. It's not a situation that the flagger can't handle on their own: anything a reviewer could do, the flagger could do.

Pragmatically, "needs severe editing" is not a situation that is useful to flag. The hard part is not identifying such answers; I run into plenty of them every day. The hard part is finding enough time to edit everything that could benefit from editing. Flagging as "needs improvement" isn't helpful. Flagging effectively says "This answer could be improved, but I can't be bothered to do it; let's create work for someone else".

Also, I'm concerned that flagging "needs improvement" is a one-bit signal, but it doesn't indicate what specifically needs to be improved or how to do it. It's not actionable. It's just a flag saying "this sucks" but without explaining exactly how it could be better.

You could also run into cultural issues on some sites. Some sites discourage heavy edits from third parties. I have a mental list of sites where I've tried making edits to improve answers and discovered that such edits are unwelcome -- and I'm not even talking about severe edits at the level you seem to be contemplating.

Finally, I expect that on larger sites the "help and improvement" queue will be one that might grow unceasingly and never get emptied. There's a boatload of answers that "need improvement", but I'm not persuaded there's enough capacity among reviewer-editors to fix all of them or even a majority of them.

Removing the VLQ flag

The premise seems to be that answers we're currently flagging as VLQ could be either flagged as NAA or could be saved if they just went into a "help & improve" queue. I'm not persuaded by this premise. Or, to state it more constructively: I think it'd help if you presented data to show how this would help.

When answers are flagged with VLQ, usually that's because the answer is so bad that there is no reasonable way to improve it (short of editing it to replace it entirely with something entirely new that's so good it could have been posted as a new answer). Typically, the right answer for those VLQ answers is to delete them.

When we have a terrible, very-low-quality answer, it seems like the right thing to do is to delete it. If you can see how to craft a good new answer from scratch, starting from nothing, then post it as a new answer.

And philosophically, if an answerer puts so little effort into their answer that it deserves a VLQ flag, then I have zero motivation to rescue their answer for them. I'm fine with helping and improving answers where the answerer has put in 80% of the work and has some good stuff, but needs help for the remaining 20%. But if the answer has put in 0% of the work, or 5%, and reviewers are supposed to put in 100% of the work, or 95%, then I don't think that's a reasonable thing to ask reviewers to do. In my experience, usually when you've got a VLQ answer, turning that into a good answer is no easier than writing a good answer from scratch.

That's my anecdotal impression. I could be persuaded by data showing that my impression is wrong. Do you have data? Perhaps you could do a review of answers that have been flagged VLQ, and analyze them to see if my sense is correct?

Misconceptions about VLQ vs NAA

The question seems to imply there's no difference between VLQ and NAA flags. That's not quite right. There is a difference. NAA is for answers that don't even try to answer the question. If you've got an answer that is trying to answer the question but is total garbage, you need to use VLQ, not NAA. VLQ shouldn't be used to flag answers that are merely wrong, but if it's got severe problems that can't be rescued through editing, then a VLQ flag can potentially be appropriate. For instance, I've seen answers where they are only one sentence long, the answerer thinks they're trying to answer the question, but the reasoning is gibberish or reflects a fundamental irreparable confusion. For some such answers, a VLQ flag is appropriate, as a way to get enough 10K users to take a look and vote to delete.

So, when you say that anything currently flagged with VLQ could instead just be flagged with NAA, that's simply not correct. In my experience, many of those NAA flags will be rejected.

10
  • 3
    Flagging effectively says "This answer could be improved, but I can't be bothered to do it; let's create work for someone else". – I acknowledge that this flag should probably be used very rarely and maybe it’s the lesser evil not to have it at all, but I occasionally encounter posts (mostly on sites outside my expertise) which are clearly actual answers and clearly in need of improvement but for improving which I lack the expertise. That being said, this flag is nothing I have strong feelings about. The autoflags are more important anyway.
    – Wrzlprmft
    Commented Sep 4, 2015 at 18:17
  • 1
    The premise seems to be that answers we're currently flagging as VLQ could be either flagged as NAA or could be saved if they just went into a "help & improve" queue. – It depends on the site and who is flagging. The first problem is that there is much confusion as to what should actually be flagged VLQ. On some sites, VLQ is used for answers that need improvement (see here) and some users use it as such (see, e.g., my examples at the bottom of this answer).
    – Wrzlprmft
    Commented Sep 4, 2015 at 19:20
  • 1
    Now, I totally agree that going by its wording (“unlikely to be salvagable”), VLQ-flagged question do not belong in a HI queue. But those questions to which the flag does apply by wording could in my opinon be very well flagged NAA (“does not attempt to answer the question”), at least I have yet to see a counterexample. But that’s not my main point anyway and could be addressed by slightly rewording the NAA flag. The main point is that questions flagged NAA and VLQ are treated almost, if not fully identically, in particular they go to the very same queue.
    – Wrzlprmft
    Commented Sep 4, 2015 at 19:29
  • 3
    All that being said, the main problem here are autoflagged answers, i.e., answers with a bad quality heuristics. Most of these answers which go only to the LQ queue (i.e., if they are not a first post or late answer), are not utter garbage but actually in need of help and improvement. Therefore placing them in a queue which is about their deletion is bad – moreover so, as many of them are deleted only because they do not look OK by any common definition of those words.
    – Wrzlprmft
    Commented Sep 4, 2015 at 19:46
  • Anecdotally, I have found little if any difference in flag declination rate between VLQ and NAA, even on the same types of post. I have had very few flags declined (proportionally, at least), but of those, no small fraction on multiple sites has been from terrible semi-answers that were in no real sense NAA but were utterly unsalvageable, usually because they missed the point of the question entirely... which I flagged VLQ. So your last few paragraphs are a little misleading there. It's not the flag type; it's the crude heuristics too many reviewers use in that queue. Commented Sep 4, 2015 at 21:13
  • @Wrzlprmft, thank you for your constructive and thoughtful comments! "I occasionally encounter posts which are clearly actual answers and clearly in need of improvement but for improving which I lack the expertise" - The problem is, if improving the answer requires specialized expertise in a particular area, then flagging it and putting it in a queue is not a good strategy. It will show the answer to randomly chosen reviewers, who are no more likely to have the special expertise needed than you. (They might even be less likely: you were interested enough to read the question, after all.)
    – D.W.
    Commented Sep 5, 2015 at 1:51
  • @Wrzlprmft, "those questions to which the [VLQ] flag does apply by wording could in my opinion be very well flagged NAA" - There's no question that there is a lot of confusion around the VLQ flag, and its purpose often seems unclear. I agree that VLQ flagging could be improved. But this is a complex issue. I don't think it's accurate that you could just flag all the VLQ-answers as NAA. I've tried that, and I'll tell you what happens: the NAA flags get declined. (Understandably so, as often the VLQ-answers are at least trying to answer, or think they are -- so they're not NAA.)
    – D.W.
    Commented Sep 5, 2015 at 1:53
  • Anyway, I think the VLQ vs NAA issue is complex, too complex to resolve in a comment thread -- it's complex enough to merit its own Meta thread. But before starting a new post on it, I would highly recommend looking for data: say, 5-10 randomly chosen answers that have been flagged as VLQ, and analyzing for which of them a NAA flag would be appropriate and accepted -- and maybe do this for 2 or 3 SE sites.
    – D.W.
    Commented Sep 5, 2015 at 1:55
  • "the main problem here are autoflagged answers" - That's helpful, thank you for identifying that focus! Perhaps it is worthwhile to pull that one out separately, and try to identify a new proposal that is more narrowly crafted for auto-flagged answers, without pulling in the complexities associated with messing with VLQ flags, NAA flags, and "needs improvement" flags?
    – D.W.
    Commented Sep 5, 2015 at 1:56
  • 3
    @D.W.: Okay, let’s assume there is some fuzzy or clear line between NAAs and VLQ answers. Then have one aptly worded flag that comprises both of them, because they are treated differently only in marginal aspects, if at all: Both kinds of answers are sent to the LQ queue and surfaced to moderators under certain conditions; both kinds of answers should be deleted, if flagged correctly. It’s almost the same as the difference between too chatty and not constructive comment flags, which are treated the same in every respect.
    – Wrzlprmft
    Commented Sep 5, 2015 at 7:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .