We've been getting a lot of complaints about moderator flag decisions being incorrect or inconsistent lately.
- Flag declined, but question closed for the same reason
- Why does flag marking as helpful/declined not always correlate with moderator action?
- Why was this not-a-real-question declined?
- Mark flags as valid if question is subsequently closed for the exact reason (eg not a real question) as the original flag
- Flag moderation "disputed" but question closed anyway
- Bind flags to revisions to prevent unjustified declination resulting in flag weight loss
- Moderators should see the post as I flagged it, not the edited version
- Flag declined after a day
- Inconsistent decisions on flag validity on clear cases
- Declined flags which are later approved should be removed from profile (10k)
- Is there anything broken with the way flags are being handled?
Some of the complaints about specific posts turn out to be unfounded, but the general problem still exists. Sometimes moderators make mistakes.
In part because I'd like the chance to fix my own mistakes, and in part because I'd like the complaining about flag weight to stop (or at least be reduced), I'd like to propose that moderators be able to reverse flag decisions, or at least nullify them in borderline cases. Here's how I picture this working:
- A user flags a question for moderator attention as "Not a Real Question."
- A moderator disagrees and dismisses the flag as unhelpful.
- The community later closes the question as "Not a Real Question."
- The user now has the option to dispute the flag decision from their flag history console, and three things can happen:
- The flag can be reversed by a moderator, which means the user gets back the 10 points of flag weight that they lost plus whatever they would gain from a helpful flag at their current flag weight. This would be used in cases where the moderator originally just made a mistake like hitting the wrong button.
- The flag can be nullified by a moderator, which means the user only gets back the 10 points of flag weight they lost. This would be used (at the discretion of the moderator reviewing the disputed flag) in borderline cases where the moderator may not agree 100% with the flag, but the user shouldn't be penalized for it.
- The flag dismissal stands as "unhelpful." In order to prevent every single flag that gets dismissed as unhelpful from being disputed, the user will lose an additional 10 points of flag weight if their dispute is not either reversed or nullified.
If this gets implemented, any future complaints here on Meta about flagging decisions should be met with immediate closure and comments instructing the user to dispute the flag.
I'm not 100% convinced of the utility of having both reversal and nullification, so maybe an implementation only needs to include one or the other. (See my update below.) As always, I'm open to ideas and suggestions for improvement.
TL;DR: Moderators should be able to reverse flag decisions, but users should have to risk something so every single decision doesn't get disputed.
Matthew Read wrote in his answer below:
Our friendly neighborhood waffles has determined that approximately every 1 out of 6 declined flags, or ~17%, is for a post that gets closed. That's really high. This auto-nullify method would eliminate the trashing of flag weight for users who were probably doing the right thing to flag these to-be-closed questions.
1 out of 6 is really high. A lot higher than I would have guessed, but I was only thinking of those cases where moderators mistakenly dismiss a flag as invalid and the community (or another moderator) later takes the exact action suggested. In a lot of those 1 out of 6 cases the flag is marked invalid purposely to send a signal to the flagger that the post really didn't need to be flagged at all (like when it should have just been downvoted, edited, or a vote to close would suffice). In those cases I'd really prefer if the flag had never happened, so I'm even more in favor of having both the ability to nullify flags and the ability to reverse them.
If we can't get this as a moderator ability to review and correct mistakes, I agree with Matt and waffles that just automatically nullifying all of the flags that are disputable would alleviate most (if not all) of the problem.