Should we focus the moderator flag queue?

Question

Basically, after reading through (again) the Theory of Moderation post, it got me thinking.

A lot of flags that reach moderators (as well as 10k users, but not enough users actually do something about it).

Let's have some data:

Do all these flags belong?

Now, a good (great) deal of those can be (or should be) handled by the community.

Moderators are Human Exception Handlers™, they don't need to deal with things the community deals with daily.

So here's my proposal:

• Close question flags coming from users <3k should go to the review queue instead of the flag queue. Perhaps those should get a slight boost so that they appear sooner. (If the question gets closed, it's approved. If too many "leave open" votes are given to it, it's declined).
• Low quality flags obviously go to the low quality queue. There's a little a moderator can do that a normal user can't do in this state anyway (Either edit and improve, or downvote and delete).
• Not an answer flags. I propose a new ability for high privilege users (20k or more), to allow them to convert an answer into a comment, much like the current mod's ability. It'll take 3(?) votes, and the post will be converted to a comment on the post most voted (if no agreement is made, either take the comment to the question, or possibly have 5 voters?).

The rest of the flags (i.e.: Too many comments, duplicate answers, disputes and vandalism), as well as free text flags should remain on the moderator's queue, because they are truly exceptional.

An exception to the above though, if the same post is flagged multiple times with any of the above flags, it should reach the mod's queue, as it seems more "urgent" intervention is needed.

That, in addition to upgrading users with more close/delete daily votes. I know for sure that in some (_{coughphpcoughjquerycough)} users consistently max out their daily votes before noon.

Duplicates

Another issue is the case of duplicate content. (As in, the same user/group post the same question several times). It should be differentiated from questions "which may have an answer on this other question". A moderator isn't an expert in every single field in their site (take Stack Overflow for example), he cannot accurately determine whether the answer given in the flag is correct or not.

TL;DR

Mods shouldn't be required to deal with noise. Moderators are there to save us when the community has maxed out its potential intervention in a post. Why waste their time with flags that could very easily be handled by normal users?

Answer 1

Not an answer flags are probably the easiest, quickest flags to handle - they tend to be short, easily recognizable, and there are two buttons ("convert to comment" and "delete") that'll dispatch them with a single click (with one annoying exception). Since they don't actually chew up a lot of moderator time, and (as Mad notes) most comments-as-answers don't actually need to be comments either, creating a queue for these is probably a waste of time.

Close and Low Quality flags are already in /review - getting them out of the mod queue was one of the big goals for the redesign. Unfortunately, that got kinda hung-up half-way because of the mess that is the flagging system, with the end result being that when the community handles a flag by saying "don't close" or "don't delete", the flag isn't dismissed and ends up in the mod queue anyway. That's in the process of being fixed, along with a bunch of other weirdness surrounding flags.

I disagree that close flags should be given priority over close votes - that's effectively saying that any user with 15 reputation has a better chance of identifying things that need to be closed urgently than users with 3K+. However, I wouldn't mind seeing close flags split out into a separate review queue, accessible only to 10K+ users, as effectively a mentoring program.

Answer 2

This certainly makes sense for Stack Overflow, and maybe also for the rest of the trilogy and some larger SE 2.0 sites. But I don't think the flag queue should be filtered at all on all the other SE 2.0 sites with a low flag volume. The very high-volume sites should be treated differently, the way SO moderation and moderation of a small beta site work is just too different for one set of rules to work for both.

I'd also add a time factor, if the flags aren't handled after x hours, they should still get to the moderator queue.

There are a few small things still missing to allow the community to deal with all the common flags effectively, the most important one in my opinion are the notification and visibility rules for deleted posts. Moderator comments always get to the users inbox, comments from regular users only if they are stock comments from review.

I don't think the convert to comment feature should be available to non-moderators. My impression is that a large number of users tend to do the nicer thing if they have a choice, which leads to crap questions getting migrated instead of just closed. The vast majority of "not an answer" posts should be deleted, and not be converted to a comment. If this tool is made available, I'd at least require a unanimous vote.

Answer 3

I like the idea in general, and I think it's always been the goal, at least for as long as I've been seated as a moderator. Offloading the majority of the work to many instead of few hands has been a bit of a battle. Getting people to flag stuff in the first place was interesting too, remember flag weight? I think we're close to coming full circle, and it's been quite a ride.

Some points about your proposal, followed by a bit of my own brain dump:

NAA/Comment Conversion

The convert to comment thing works best on very old posts where some useful bit of tangentially related information was left as an answer and up-voted without really answering the question. Normal users would typically not even see this as an option, because the comment author would not have enough rep to comment in the first place. I don't think we should be too free with the ability to circumvent system restrictions, even with three votes.

NAA flags are being handled better by the community, but still left in our queue. I'd suggest that this is a feature as it lets us review the accuracy of reviewers and reverse the (infrequent) mistake. It would be nice to make them visible from another sort type, however, and not increase the global flag count. This brings up another good question, how can the system better help moderators to teach users that are interested in helping?

Duplicate

I agree that moderators should only get involved in duplicates when we're looking at question or answer repetition, we have system raised flags for identical content and consecutive closed questions. We have a special message for question repetition, so we'd have to ensure that cases of it is brought to our attention even if the content posted is a little different. Multiple questions by new users marked as duplicates is the event to catch here.

Now for some stuff you didn't cover:

Vandalism

If enough 10k users counter flag a community 'possible vandalism' flag, just bury it. That means three people that are sensitive to good content being deleted in the name of rage have reviewed the user's activity and edit history and determined that there's no more to see.

Duplicate Content

I personally feel that we should be doing more to keep truly (as in byte for byte) duplicate content out of the system to begin with, perhaps by storing hashes of the last (n) questions or answers that a user has posted and slowing them down considerably if they attempt to post the same thing multiple times. It's an area of opportunity for the just-in-time help that we really need to be presenting. It's a waste of the user's time to allow them to post answers that just waste the time of others to delete, in almost 100% of all cases.

Very Low Quality

We need to work on renaming this one. It's just too broad, even with the help text underneath it. Too many people use it to indicate that they simply don't like something, for some unknown reason. Until we come up with a way to make it, well, not like that, enough counter flags by 10k users should invalidate the flag without us having to intervene. Does the post warrant immediate deletion without any effort to try and edit it into shape? It's not that hard to judge.

Final thoughts

There's a re-tooling of the last re-tooling of the re-tooled moderator tools in progress, at least as far as I know. The /review system is also still very much a work in progress. However, I think these are all good points that could be fundamentally addressed, even if the eventual implementation is vastly different, or even based in a whole new kind of system.

A drawback of becoming a moderator on a high volume site is not having time to deal with flags and participate in other areas. Unlike mods on new sites, SO mods have very little time to do work on community building activities. I can't remember the last time that we had some kind of contest that moderators came up with and presented to Stack Exchange for approval. Never mind participating by answering questions. When elected to a high volume site, you should have an expectation that you'll be spending the majority of your time in the proverbial salt mines, but I'd really love to see that lessened a bit.