TLDR version: If I wanted to, I could spoil everyone's flag weight by marking all flags I see as invalid. Nobody would earn flag weight, and I wouldn't lose anything either. This is IMNSHO broken.
I have seen Jeff's take on disputed flags, but I disagree with him:
This pits flaggers "against" each other, which is not at all in the spirit of the thing. In order for one flagger to "win" (gain flag weight), the other has to "lose" (lose flag weight). In my view everyone who flags, with extremely rare exceptions, is making the site better and I almost never reject a flag.
Why does anyone have to lose flag weight? This doesn't make sense in at least two scenarios:
UserA correctly flags an answer as "not an answer." UserB flags the flag as invalid. Moderator comes in and deletes the answer. No flag weight is awarded to UserA, even though they were right.
UserA incorrectly flags answer as "not an answer." UserB (correctly) flags the flag as invalid. Moderator decides UserA's flag is incorrect. UserB doesn't earn flag weight, nor does UserA lose flag weight.
It's not uncommon to have flags at two different points in the post lifecycle that are both correct. That is, an answer will be (correctly) flagged as "not an answer", then the OP will go back and edit in detail to make it a better answer. Now someone (correctly) flags it as "previous flags invalid". Both flags are correct!
In that case, just allow both flags to be deemed helpful!
We've considered a few alternatives (make mods dismiss each flag individually, add a third "indeterminate" outcome button) but they all seem worse than this simple solution.
IMO this is not a good solution. It makes flagging flags as invalid not only useless, but harmful. If the original flag was correct, this prevents the original flagger from earning FW. If it wasn't, it will prevent them from losing FW - both of which were the original intended consequences.
I think flag weight should be awarded to the user(s) the moderators agree with, without penalizing the user(s) who put in the wrong flag, if Jeff thinks this "pits flaggers against each other."