Why do you cast downvotes on answers?
What are the criteria to down-vote an answer?
Should an answer be down-voted only if it is not correct, or there are other criteria too?
This is how I see it:
You should absolutely take a neutral point and never downvote an answer if you personally dislike the author or if the answer addresses some annoying issue that irritates you but still (unfortunately) happens to be true. As they say, don't kill the messenger.