It depends on the answers in question.
If the new answer adds something more to it, then it's fine. This something more could simply be a better explanation of the answer with greater detail. It could be adding additional possibilities to the answer. It could be covering points that the original answer did not provide.
This is especially true of "fastest gun" answers, which are short and quickly accepted. It may be functionally correct, but it is Spartan, sufficient, and perhaps overly brief. Someone can come along later with a real, detailed answer, and there's no reason for this answer to not be rewarded.
The only time I would say that downvoting a "duplicate" is fine is if it truly is duplicated. Either it is a copy-and-paste job (in which case it should probably be downvoted and flagged) or it covers the same material as the other. And even then, I wouldn't suggest downvoting it as simply not upvoting it. The other answer is accepted and likely upvoted. So the duplicate likely won't catch up. And you're not going to counter the reputation gain from any upvotes, so you're not really helping.