6

We see these kind of statistics all the time:

Flag statistics

Most of the flags are not an answer flags and many of those are actually invalid flags since many users flag incorrect or low-quality answers as not an answer.

We have proposals to improve that situation. Are those proposals considered by the team or is this actually not really a problem?

  • 1
    In the past I've seen the statement that NAA flags are generally so fast for mods to handle that there wasn't a compelling need to change the workflow. (That was coming from a mod related to a proposal that NAA flags somehow feed a /review queue.) Not sure if that still stands though; it was a while ago. – Servy Mar 4 '14 at 22:04
  • 3
    "very low quality" is ambiguous and is rarely correctly used. Spam is spam. Invalid flags are an inner measure to notify someone flagged incorrectly. However, a lot of people use SO as a forum and post "I'm having this problem too" or "did you fix it?" or ask for clarification in answers - mainly new users. – Benjamin Gruenbaum Mar 4 '14 at 22:11
  • 2
    @BenjaminGruenbaum: When I go through the not-an-answer flag queue I see it used wrong all the time. – juergen d Mar 4 '14 at 22:12
  • 1
    @BenjaminGruenbaum Yes. The question is simply whether the mods are capable of handling all of the NAA flags (and in a reasonable timeframe). Also, looking at the queue (I just went through most of a page) the percent of valid flags was really low. And I mean really low. Below half. Mostly it's people using NAA for a bad/incorrect answer. I don't spend much time in 10k tools, but I don't remember it being that low in the past. – Servy Mar 4 '14 at 22:14
  • @Servy: Some folks see a bad answer to a question, and they hear Gilbert Godfried in their head screaming "That's not an answer!" Ergo, their flag choice. – user102937 Mar 4 '14 at 22:20
7

90-95% of NAA flags are reasonably accurate. Compared to VLQ at 60-70%, that's really quite good. Because of this, they're not normally that much of a burden on the moderator team, at least in contrast to VLQ and especially "other" flags.

Inline sidebar: Not An Answer flags during the past 30 days on Stack Overflow

NAA   Helpful On Now-Deleted Post Helpful AND on Now-Deleted Post
----- ------- ------------------- -------------------------------
57203 52733   52860               52281          

Either way you slice it, that's over 90% accurate.

That's not to say this couldn't be improved, however. A big problem right now is simply that they're exposed to 10K users, but 10K users can't vote to delete them! They can dispute them or pile on more NAA flags, which can help the moderators process them more quickly in some instances but can also just inflate the flag count (this was a particularly bad problem during Winterbash and the recent moderator election).

We could do something similar to what we did with VLQ flags (direct them into a queue where a large number of low-rep users are allowed to delete them) - but this is kind of a waste when only 1 out of 10 won't be deleted.

A better option might simply be to put them in front of 20K users and give them the ability to delete OR decline, with only controversial and unhandled going before the mod-team.

  • How accurate are link-only NAA flags? I feel you made yourself fairly clear in your previous answer regarding answers solely comprising potentially-but-not-definitely useful links, but the newly proposed NAA flag replacements do not have the scope for link-only answers. Do you agree with Robert Harvey that they should be flagged with a custom reason rather than NAA? Based on my own flags, it does not seem like link-only is an abuse of NAA. – chappjc Mar 4 '14 at 23:31
  • 2
    "Link-only" isn't a reason to flag. "Doesn't attempt to answer the question" is though, and when folks flag irrelevant / tangential links as NAA, those generally are marked helpful (and deleted). Folks get into trouble when they see a link in a short answer and then think, "what should I flag this as?" - if you stick to flagging things that are actually problems, you don't have that particular crisis of the soul. – Shog9 Mar 4 '14 at 23:34
  • So, a "See here." answer is OK as long as the link has a relevant nugget on the linked page? I tend to flag answers that send the reader on a wild goose chase, even when you can sort-of glean the answer from the link. – chappjc Mar 4 '14 at 23:37
  • Depends entirely on the context. What is the question asking for? – Shog9 Mar 4 '14 at 23:45
  • 3
    "90-95% of NAA flags are reasonably accurate" -- Really? I'd put that stat at closer to 60 to 75 percent. It used to be 90 to 95 percent, but nowadays I'm estimating that I decline one of every five "not an answer" flags that are cast (when I'm in "strict, letter of the law so I don't break the Review Audits" mode). See Also meta.stackexchange.com/q/224167 – user102937 Mar 5 '14 at 3:37
  • Approx. 90% of NAA flags as of February were marked Helpful / resulted in deleted answers, down a bit from last year, @Robert. – Shog9 Mar 5 '14 at 3:43
  • Is that slash an "and", or an "either/or?" Because I'd buy the either/or; and if that's the metric for choosing audit posts, then maybe there isn't a problem. – user102937 Mar 5 '14 at 3:46
  • See edit, @Robert – Shog9 Mar 5 '14 at 3:50
  • @Shog9: So you are saying you would agree to 9 of 10 NAA flags when you go through the flag queue - Interesting. I need to stop marking those as invalid. – juergen d Mar 5 '14 at 10:24
  • Just mark 'em invalid if they're invalid, @juergend. 10% is still around 190 flags a day... – Shog9 Mar 5 '14 at 16:59
  • @Shog9: Reviewing lately I thought about your post. And I am guessing 90% helpful NAA flags come from 90% people agreeing on the wrong flags. When I go through the queue every second NAA flag is invalid IMO but 200 every day seems doable. – juergen d Mar 9 '14 at 22:57
  • The design of the 10K flag queue definitely makes this more irritating than it needs to be, @juergend. Still, I think you're remembering the invalid ones more readily than the valid ones - or perhaps they just hang around longer in the queue? – Shog9 Mar 9 '14 at 23:28
0

The problem is people flagging answers that are better dealt with by down-votes.

Getting rid of the "not an answer" reason probably won't solve the problem, as the people doing the flagging will only select a different reason.

The best solution is that the Moderators continue to reject invalid flags(which they do), perhaps with a mechanism to make the rejected flag more visible to the flagger, so that they know not to do that.

  • Thanks for the answer. But I did not ask how this should be handled. There are proposals for that. I was asking if people think we should change something or not. – juergen d Mar 4 '14 at 22:23
  • @juergend: I'm not really sure what you are asking here, other than "is the team ever going to do something about this?" We don't know that. – user102937 Mar 4 '14 at 22:36
  • 1
    @RobertHarvey I read the question more as, "Is the current state of things entirely manageable, or is there a problem that we should start considering solutions for." The idea being the whole premature optimization is the root of all evil; if it's not broken, don't try to fix it. – Servy Mar 5 '14 at 0:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .