If I flag an answer as "not an answer" and Moderator X (whomever that may be) deems that flag invalid, the flag shows in my flagging summary as "Invalid".
If I later look at my flagging summary and see that the answer concerned has been deleted, by Moderator Y (Note: For all I know, X and Y could be one and the same, that's irrelevant) presumably as the result of flags from further users without the original answer having been changed, it would seem reasonable for my original flag to have its "Invalid" status cleared.
I'd suggest that a flag which is "cleared" wouldn have a neutral overall affect on a users overall weighting, i.e. it's excluded from calculating flag weight, rather than the -10 it garners as an invalid flag or the value between 10 and somewhere near 0 it gets as a valid flag.