It appears that comment downvoting has been discussed in the past as a feature request.
In that case, the problem was dealing with comments that were factually incorrect, off-topic, or misleading:
if you strongly believe that a comment is misleading, incorrect or off topic
Jon Skeet responds here indicating it would be better to respond with a comment clarifying or correcting the erroneous comment.
I think this makes sense for comments that are of a factual nature. For comments that are just snarky or rude, I think a downvote is preferable to commenting. It lets everyone know that the snark and rudeness is unwelcome at a single glance and could allow the comment to get buried under the noise. It subtly encourages polite behavior without moderator intervention.
Responding to snarky/rude/trolly comments is only going to potentially feed the troll. Such a person is likely going to feel they need to respond in kind and a pointless thread of ugly snarky arguing will ensue. Better to downvote their comments and just let everyone ignore the bad stuff. Let the good/nice stuff float up.
Flagging is one option available, but that requires moderator attention. It seems we want to be able to get rid of snarky/rude stuff quickly without needing to rely on a moderator.