Especially when they should know better:
PS: And someone wants to reopen it!
I don't think it's wrong. I've done it myself a few times, usually only when a high-Rep user (10k or above) answers a question that is pretty clearly out of bounds.
Note that when you hover over the downvote arrow it says "This answer is not useful", and an answer from a high-ranking site user on a nuke-worthy question is just that: not useful. In my opinion, it sends the wrong message to other users on the site, because it is a tacit approval of the inappropriate question from an experienced user. In short, it sets a bad example, and usually I'll leave a comment when I downvote saying as such.
However, it should be noted that it can be easy to accidentally answer a duplicate without realizing. I probably wouldn't downvote for that, unless it is such an obvious many-times-over duplicate that they should have been able to find one with a trivial search. Not all duplicates are easy to find. I usually reserve this kind of "you should know better" downvoting for really off-topic or totally subjective poll questions.
I typically don't down-vote the answers because many times the one answering the question does not realize it is a duplicate or it is arguably an edge case type question. They get more slack because they are less responsible for searching than the Asker.
If it is a super awful question, I might down vote. I don't think it is wrong either way.
If the OP has a fair amount of rep then I tend to downvote - they should really know better. Having lots of questions that are the same is not awesome so the question is not awesome - it is very bad.
If they are a new user, I will probably just vote to close and add comment hinting that they maybe should have spent a few minutes searching etc.
However, if the question is a question that has been repeated many times over and it's pretty obvious the OP didn't bother searching at all then it's an immediate downvote from me.
Just my 2¢