I've recently been in a discussion on SO where I had downvoted a question because it is a common question (similar to any simple question that causes annoyance) and I downvoted it as such and then left a comment stating "-1: This issue is discussed in-depth all over the place." which I agree may have been a bit harsh but still to the point. I then got jumped on by one of the older members who said "@M.Babcock, -1 to your comment since it was completely unhelpful."
The comment was left as a justification of the downvote not as a direct criticism of the question. More often than not on SO a downvote goes without a distinct reason why and someone ends up asking for a reason, so I've found the best way around this wait-and-see is to include a reason if one isn't already provided (in which case I upvote the reasoning comment).
My question is this, are we supposed to leave comments justifying downvotes or not (regardless of how correct they may or may not be)? If not, then please help me understand the best way to mitigate the "who downvoted?" or the "+1 for the unidentified weeny who downvoted without justifying" situation.
For the record I find myself curious of other people's downvotes more then I downvote myself.