The "good" reason for downvoting someone's post is because it is genuinely bad. Now, there is a certainly an element of subjectivity involved, because what feels "genuinely bad" to one person may not be so to a second person, and may even sound "pretty good" to a third person, who will downvote, not vote, and upvote respectively. This part is all "honor system," and assumes that everyone is voting their genuine convictions, as they might for a political candidate.
A bad reason for downvoting is for an answer that is OK, but not as good as some other. Then the proper course is to "not vote" for one, and upvote the other.
The worst reason for downvoting is "personal" downvoting, where one downvotes the person, rather than the post.
You don't need to remove your post just because you think someone else's is better, but if you do, you'll earn the "disciplined" badge if it has three or more (net) upvotes.