I just ran across this answer (10k only) in the flags queue. I was slightly confused when I first saw it, because the "not an answer" reason was greyed out, which means the first flag had been dismissed. But the answer was not deleted. It was very clearly not an answer, so I was curious, "who cleared this flag and didn't delete it?"
So I went to take a peak at the timeline... Cleared by Community? Interesting... Oh, the not an answer flag pushed it into the Low Quality Posts queue. And... well, what do you know? A single Edit action completed the review. This not only cleared the not an answer flag as helpful, but it didn't generate any other sort of auto-flag on the post. The post clearly still needed attention. It's still not an answer.
I understand that an Edit action is meant to "fix the post" and I don't mind the fact that it completes review. However, that ideology only really applies to the "very low quality" flag reason. Now "not an answer" flags get pushed there also, and I imagine it would be pretty rare that a not an answer flag could actually be cleared out successfully just by an edit by a random user in a review queue. It's much, much more likely that the user just didn't pay attention to its non-answer-ness and edited to improve formatting, grammar, or whatever else. So I propose one of the two things should happen when an Edit action completes the review for a not an answer flag:
Generate a "disputed low quality review" flag like any other post would get when something "contradicts" the flag in that queue. Use the subtext "Edited but flagged as not an answer" to describe the situation.
Don't automatically dismiss the flag when it is a not an answer flag that caused the review item to be created. Just complete the review like normally, and leave the flag alone so a moderator can still look at it.
