14

One of the moderators on Stack Overflow in Portuguese was complaining about the number of "closed without comment" flags they had to deal with, considering the community has plenty of users who could provide guidance to newcomers and not leave all the work to the mod team.

That flag only exists on beta sites, as a way to ensure new users get the help they need when they post a question that's not well received by the community. That's a noble goal, but I'm not sure this flag is the right way to achieve this.

Some sites can stay in beta for a long time and have pretty mature rules. The community grows weary of re-writing what's already explained by the text on the "closed as" box under the post. You can't expect top users to keep detailing why they voted to close a post every time they do it.

Also, while the flag might help a moderator improve a post, or explain the rules to a newbie, some posts just can't be saved. For instance, there's nothing an experienced user, or moderator, can do to save or improve questions that are clearly off-topic for the site. They should get closed and there's no point in wasting anyone's time on them. The default close text does a fine job explaining what happened to the poster.

I did some research on a handful of old beta sites. Of all posts flagged "closed without comment", the majority (>40%) of them were closed as off-topic. That's a lot of unnecessary work over the years.

Why a queue?

  • Anyone can do this; There's no point in leaving it all to the mods
    Right now the moderators are solely responsible for fixing, or not fixing, these questions, when there are dozens of other users who could do it just as well. This task doesn't require any specially knowledge and could easily be handled by the community.

  • It gives people who close without commenting an opportunity to learn too
    The current model skips over the fact that a bad post is not the only one to blame for that flag. There's also the users who voted to close a question and did not care to provide any information, guidance or assistance to the poster. A learning opportunity is wasted when a moderator just goes in and silently fix that mistake.

  • Users would get assistance much faster than they do now
    The queue would make sure that everyone's constantly reminded of the importance of commenting on posts you vote to close (when there's a need to do so) and that, in case a post does get closed without an explanation, that mistake gets fixed much quicker than before.

Data!

After some comments, I realized I had to go deeper! Apologies for taking so long, so let's begin... Out of all SE sites, Core Review has the most "Closed without comment" flags, at 1588 flags so far. Little over double the number of flags on OnStartups (775). So let's start crunching numbers:

  • How useful is the flag?
    Let's take a look at how the mods review these flags, to see if they're being useful.
    How many of them were declined by the moderators?1

    Flags | Declined | Pct. Declined
    1646  | 597      | 36.27
    

    It sure looks like these flags are useful after all but as @rolfl pointed out, it takes less clicks to dismiss flags as "helpful" and mods might be inclined to do that, given that it's a Community flag. This means this might not be the most accurate way of measuring just how useful this flag really is.
    So, maybe, as a way to measure how effective the flag is, we could see how many posts received comments after they were flagged:

    Flags | Commented | Pct. Commented
    1646  | 637       | 38.70
    

    It didn't change much. That's almost 2/3rd of all flagged posts that got no help at all.

  • Could they have been saved? So, out of those that did receive some care due to being flagged, did that care work? Were they re-opened?

    Commented | Reopened | Pct. Reopened
    637       | 117      | 18.37
    

    Looks low, like the extra guidance didn't matter. So let's see how many uncommented posts get reopened:

    Uncommented | Reopened | Pct. Reopened
    1009        | 32       | 3.17
    

    That's a considerable difference. It's the kind of thing I was looking for. The comments do make a difference and, right now, the moderators are the only ones responsible for adding them. I don't think that should not be the case. But, as I said, some questions are simply beyond saving (the obvious case being questions that are clearly off-topic) and, as such, there's no point in leaving a comment. Let's remove all off-topic questions from the last set of data and see how that goes:

    (Only off-topic)
    Uncommented | Reopened | Pct. Reopened
    57          | 5        | 8.77
    

    That's certainly an improvement, but still not even close to the percentage of posts that get reopened when you add a comment.

Sorry for taking so long to gather this and post here, but other things (professional and personal) got in the way. But now, after looking at the numbers, I'm even more convinced there's good reason to remove this automatic flag and replace it with a queue, where the work of helping new users understand and get used to our sites is shared among existing users.


1. This doesn't match the numbers I gave on a comment down below. That query had an error in it, and was returning the number of flags disputed.

  • Do you have statistics on the decline rate for these flags? – Gilles Sep 18 '14 at 19:11
  • It's not high. The Workplace has the highest one, at 13.66% of them declined. SOpt declines 8.09% of them. – Gabe Sep 18 '14 at 20:10
  • 1
    Really? I'm very surprised that any site I mod has less than 13.66% of declined “closed without comment” flags. I know some of my fellow mods always mark Community flags as helpful, but still, I should account for more declines than that. Have I been slacking on flag handling that much? Can you post the stats? – Gilles Sep 18 '14 at 20:18
  • @Gilles Code Review is the site with the most "closed without comment" flags (1561 so far). I'm taking a look at the stats there and I'll amend the post soon – Gabe Sep 18 '14 at 20:20
  • Oh, interesting, as a CR mod, that's why I am intetrested in this discussion.... we have that many? I know that I just click the helful-option on the flag simply because it is 1-less click than decline. Using that as an indicator of whether the flag is helpful or not is.... meaningless, @Gilles – rolfl Sep 18 '14 at 20:29
  • Gabe, that is great that you have numbers, but what is missing is the qualitative component of the process. Since these numbers involve data from Code Review, I can assure you that, for the auto flag, occasionally I do add a comment, and, in those cases the reason for that is because I think a comment may make a difference (because the close reason itself was not obvious enough, or because I thought with some directed modification it would help). In other words, I would comment on those questions that would most likely be reopened anyway. – rolfl Oct 7 '14 at 19:23
  • @rolfl That makes perfect sense but I can't seem to think of a relliable, consistent way to measure that. And in the end, the point isn't that comments would save over 9000 posts, but that we can allow more people to take the same action you, as a mod, are taking now (judging whether or not a post can be saved through comments). – Gabe Oct 14 '14 at 15:46
  • Hey @Gabe, I got my head stuck in the details, and missed the big picture. Yes, offloading that process of judging whether to comment, or not, to a trusted queue, is a good idea. Two issues/thoughts though: 1. is the queue for beta sites only (seems like the same reasoning can be applied to graduated sites) 2. what about review badges if it is beta-site only.... does the queue disappear on graduation (along with the chance to earn a badge...). – rolfl Oct 14 '14 at 16:57
  • 1
    @rofl I think it makes perfect sense as a queue for graduated sites too. The reason it's a beta-only flag is that graduated communities are mature enough to know better, and the volume of questions is too big for make it worth the effort of the mods. This makes sense, for a flag. But, again, making it a queue solves the workload issue and makes it useful and beneficial again. – Gabe Oct 14 '14 at 23:15
10

No, The auto-flag for closed-without-comment is intended to be there for the early stages of beta sites, where the idea is that the comments can be used to define, and refine what the scope topics are about.

In other words, if your site is new, and you are not quite sure about things, providing feedback to the asker is important.

On 'graduated' sites, the auto-flag is turned off. It is assumed that the close-reasons are self-explanatory.

If your mods are getting inundated with closed-without-comment flags, then it probably means that you should be giving feedback to the community managers, that the 'beta settings' are becoming more of a problem than a help, and that maybe it is time to graduate... ;-)

References: Why doesn't Community flag questions closed without a comment on graduated sites?

  • 1
    It's not that they're getting inundated, it's that most of the time it's pointless work that could've been done by other people, just as well, much faster. It makes no sense to keep it in the hands of only a few people. – Gabe Sep 18 '14 at 19:18
  • 3
    "One of the moderators on Stack Overflow in Portuguese was complaining about the number of "closed without comment" flags they had to deal with" ... I misunderstood, @Gabe ;-). If it's pointless, then why have them at all? My point is that the flag is only designed for defining scope of the site, if it is pointless, then turn them off, not queue them for others. – rolfl Sep 18 '14 at 19:21
  • Yeah, I gave the wrong impression there. The biggest complaint on SOpt is that, for a mod, any occurence of this flag is annoying. Either there's nothing to be done, or it was something the community could've handled themselves without moderator action. You can't just remove the flag because it is useful, but throwing them at moderators isn't efficient, or effective. – Gabe Sep 18 '14 at 19:59
  • 1
    So, @Gabe now you have a predicament, at the time when the flag is most useful, there's no-one with the reputation to manage the queue.... which brings it back to the moderators, with the trust of at least the SE Staff. – rolfl Sep 18 '14 at 20:06
  • I don't necessarily agree that the flag is the most useful during the site's infancy. That might be the case because mds get tired of dealing with them, but if it's turned into a queue, and the workload is more evenly distributed, more people can provide more guidance as the site keeps growing and gets more new users. – Gabe Sep 18 '14 at 20:14
  • Oh, as an aside, do you expect the queue to contribute to things like steward/reviewer badge? – rolfl Sep 18 '14 at 20:23
  • I haven't thought about it, but I don't think it would be a problem. I might be incredibly wrong, though – Gabe Sep 18 '14 at 20:25
  • I absolutely agree. For mature sites, the "closed without comment" flag is worse than useless. I have declined every single one of these flags on Code Review as "unhelpful". Turning it into a queue just foists the stupid busy work onto more people. – 200_success Sep 18 '14 at 21:02
9

I can't give statistics because moderators have no access to any history or statistics for automatic flags. But in my subjective experience, I end up declining almost all the “closed without comment” flags: either someone has commented by then (often, it was a moderator who closed then commented rather than commenting then closing), or the explanation is obvious (migration, duplicate). When I take action at all, I think downvoting an obviously wildly off-topic question is more common than commenting on a question that is indeed unsuitable for a reason that isn't obvious to new users.

So I propose a simple solution: remove that flag altogether.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .