Drop "noisy" comment flags from the moderator queue unless an answer has 5 or more comments

Question

The problem

There are a huge number of noisy comments, and they generate a huge number of flags. But, a large number of flags are raised on comments where they won't do very much good: if there's only a few comments on a post, removing one of them doesn't make an appreciable dent in the noise.

Here's a fun chart:

The x axis is the total number of comments on a post (deleted or visible). The yellow line is the fraction of moderator-actioned obsolete and too chatty flags raised on posts with this number of comments. The blue line is the same, but cumulative, i.e., the fraction of such flags raised on posts with this number of comments or less. Data is for Stack Overflow during the past 356 days.

As you can see, over 50% of "noisy" comment flags are raised on posts with 5 comments or less. On average, that's well over 100 flags every day. The raw numbers are smaller on most other sites, but the percentages are pretty much the same (with the exception of a handful of sites where pretty much any comment turns into a long discussion/argument - you know who you are).

Now, don't get too fixated on the numbers here; the real cost is probably a lot higher, since some folks use "not constructive" for literally everything and I'm not counting those. My point is simply that a majority of these flags are chewing up moderator time in situations where they're not actually doing any good.

The solution

First, we need better data on this. Re-working the wording of comment flags (and dropping redundant options) should help there.

But then... I suspect we can easily get away with just ignoring "noisy" comment flags (those that don't indicate rudeness or some other serious problem) on posts that have fewer than 5 comments in total (deleted or visible).

To that end, we should hide these flags from the moderator flag queue when there are no "rude" or "other" flags on the same post, and when the post has never had more than 4 comments on it. Focus moderators' attention on getting rid of serious problems and removing noise when it actually obscures signal. The flags can hang around until more comments are posted or until other flags are raised... Or they can hang around forever, doing no more harm than the comments themselves.

See also: Drop "not constructive", combine "noisy", reword "rude" and "other" comment flags

Answer 1

Comments aren't any more useful just because they're few in number.

Here's a few examples of short comment threads which I would flag on sight as a non-mod and delete as a mod, and on which flags are very easy to handle:

• (on a post by UserA)

  UserB: You've made a typo in the second paragraph: [...] should be [...]
  UserA: @UserB Thanks, fixed!

• (on an answer to a question by UserA)

  UserA: Thanks, nice answer! +1

• (on a highly-upvoted post, or any post really)

  UserA: Why the downvote?

Handling "obsolete" or "not constructive" or "too chatty" flags on these takes a matter of seconds, so they're not even expensive in mod time. Why should those comments have to stick around just because they're the only comments on a post?

People might use the 'less serious' flags even for serious issues.

If a comment is "rude or abusive", it's also likely to be "not constructive". Faced with a choice of different flags to use, people won't necessarily go for the "rude or abusive" one - maybe because it feels too harsh, maybe because they like to use the "not constructive" flag whenever possible, maybe because they tossed a coin and it came up tails. I don't want to see offensive comments left in place just because people used the "not constructive" flag option and no moderator ever saw them.

Flags shouldn't be left pending forever.

Partly this is a philosophical objection: the whole point of flagging is to get moderators to look at things which need their attention. If certain flags never actually have a chance of being shown to mods and getting handled, why let people raise them at all?

On a more practical note, when people check their flag history and find old comment flags still pending, they might either take it as a sign that mods are being lazy or inactive, or go and complain on meta that their obviously-valid flags haven't been handled. Such meta complaints might actually end up taking more moderator time than just handling the damn comment flags would have done.

Answer 2

I don't like the idea of ignoring flags. It sends the wrong message to users that are trying to help clean stuff up.

I propose a different solution: Automate comment removal. I've been automatically flagging comments for removal since July 2014. Of the 238,966 comments flagged on Stack Overflow in 2016, 35,610 are mine. Nearly 15% of all comment flags, which is actually down from 2015. I am a big part of Stack Overflow's comment flag volume. But, I've been accurate. In 2016, I had 12 declined comment flags.

My system isn't as complicated as other recently shown automatic tools. I have a simple Naive Bayes classifier. I only flag things that the classifier is very sure is "too chatty" or "obsolete". I don't do anything with "not constructive" for may of the same reasons you've outlined.

Last night I adjusted what I flag to only flag comments that have multiple siblings. Previously, I wasn't considering the number of comments on the post, but based on this proposal and further discussion, it makes sense to focus on the noisier posts. It's not like Stack Overflow is hurting for comments either.

Filtering the comments doesn't need to be at the time of creation. As you mentioned above, a "thank you" comment isn't hurting any one. But, come through a day or two later and clean up these old comments automatically. Right now, I'm looking at comments that are 48 hours old or older. There are plenty, every day, that can be cleaned.

What would it take for Stack Exchange to implement this type of comment clean up on your side?

Answer 3

Why not remove spam flags until there are 5 or more, while you're at it?

Comment flags do indicate a problem. Something must be done. Hiding comment flags means that the problem remains on the site and every visitor suffers from it. That's no good.

We might then work around that deficiency by creating a chatroom where people post links to comment to flags, in order to reach the threshold where action will be taken. It used to be necessary to do this to get the requisite 5 votes to get a post closes; then the close review queues automated this on most sites other than SO. And now we'd have to do this for comments?

Or people might get into the habit of flagging obsolete comments as “needs moderator attention”, just so that moderators do notice. Which I'm sure will lead moderators to decline the flags, leading to more whining on meta. And the whiners would have a point: if an “obsolete” flag is basically ignored, then the only way for a non-moderator to request the deletion of a comment is to do whatever it takes to summon a moderator!

This change only means uglier sites and more work for people other than moderators. As much as I feel I'm wasting my time handling obsolete comment flags as a moderator, I don't want the alternative of making everybody else waste their time on useless flags or convoluted flagging processes.

Answer 4

I don't think this is a massive time sink for mods.

Anecdotally, a good chunk of these flags are valid - either small edits that are needed, actual noise and take up minimal moderator time to decide on a correct course of action.

I like the related idea but quick cleanup of un-necessary flags feels more efficient than leaving them there until it blooms into a thread of comments, many of which are un-necessary.

The streamlined flag system does seem like it would improve triagability but for most sites, this feels like a non issue. The 2-20 seconds I spend looking at a comment dosen't add up all that much, least in my experience.

If we must offload these - from the (and I use a dirty word here) cultural standpoint - why not make a queue for these? These flags are anonymous anyway and this feels like the sort of thing the community can handle. That said, at SU volumes of comment flags, except on a quiet day, the benefit of this seems minimal.

Alternately, have it opt in per site. SO makes a great guinea pig test bed, sure but letting mods pick whether they have this would likely keep most folk happy.

Answer 5

What you're proposing is not a solution. You're simply hiding the existing problem (too much work for moderators) and creating an additional problem that we had in the past: flags stuck on pending for months/years. Do you want to go back to people asking why their flags are stuck for so long?

Answer 6

Make "noisy" comment flags lowest priority in the queue instead of hiding them

I agree that there is little harm in keeping a comment that says "thanks" or the like, especially if there are so few other comments on the post that none are being hidden. That said, I'm philosophically opposed to hiding any kind of flag.

Instead of hiding a "noisy" comment on a post with fewer than 5 comments, the system can heavily de-prioritize it in the moderator flag queue since it's not urgent. The moderator flag queue is already prioritized by type of flag (among other things), as explained on the flag queue page:

In addition to prioritizing the queue based on flag type (e.g. rude flag, not an answer flag, comment flag), it can be further prioritized based on characteristics of the flag within that type: a comment flag on a post with fewer than 5 comments can be de-prioritized compared to a comment flag on a post with 5 or more (since there aren't enough comments in the former case to cause any comments to be hidden).

In my experience as a moderator on Science Fiction and Fantasy these kinds of flags don't take up too much time and our mod flag queue is often empty anyway -- constraints on the moderators' time isn't an issue for us. These kinds of flags are a problem on larger sites like Stack Overflow, but they won't distract the SO moderators from more pressing issues if they're de-prioritized. If the SO mods never get to them because their queue is always filled with more important issues then it's no big deal and it's the same result as hiding them. But on the sites where the moderators can keep the queue empty these flags are still dealt with.

To further de-prioritize such comment flags, they could even be put into a secondary mod queue so that a moderator who doesn't have time to look at them doesn't even have to see them. Or they could be set to not increment the mod flag count in the top bar. (Personally, I don't think these are necessary, but they're other possibilities.)

Finally, if the moderators are really so pressed for time that they can't properly handle all the flags in the queue, why not elect more moderators?

Answer 7

Why not let non-moderator users at a certain rep threshold (20k, for example) delete/vote-to-delete comments that contain the word(s) "thank you" or some variation ("thanks" "thansk" "tnx" "10x" "hanks" "thaks", etc), if the comment has been flagged as "Not needed (anymore)" (or a separate 'noisy' option)?

You can add the list of them to the 10k moderators tool page and have it visible only to 20k users.

These comment flags would only make it to a moderator queue if they were not cleared within 7 days after the last action (ex: a comment is flagged for this, someone votes to delete it 6 days later, and a second person casts the second, final delete vote on it 6 days after that; the flag would never be seen by a moderator).

Delete comments are still visible to moderators, so in the event of abuse or a meta post, mods could easily go to a post where comments had been deleted and see who has been deleting what comments.

And in keeping with current practice, comments deleted on the same UTC day that the delete vote was cast would mean that delete vote gets refunded to the casters.

Answer 8

I don't know how this plays out for SO, but on smaller sites it works a lot better if somebody flags a comment stream early at the first sign of it going off track. I'd much rather a flag on the first comment that starts to veer into debate rather than get paged in when there are 25 and three users are all fired up at each other.

By the same token, the resulting impact of a flag handled early is much greater. People mimic what they see, and if we allow comments that would otherwise be deleted to proliferate up to 5x per post, our site is going to be a marshland ... not as deep as a lake but even harder to navigate in a boat.

Comment flags should not be handled any differently based on how many comments are on a post.