All that is required (term used loosely, your vote is your own and what you do with it is completely up to you) for downvotes is for the user to find the answer not helpful or not useful.
In fact, the answer was not wrong, merely incomplete or maybe not the best.
By the simple criteria expressed above, and depending upon just how incomplete, or how far from the best, that is just cause for a downvote.
I am simply frustrated because I am trying to help folks out over on SO, and when I share a solution that works for me, I only mean it to help.
And we appreciate it. However, if the "works for me" answer is not as good or as helpful as another, we tend to vote up the best answer and conversely vote down the "works for me" (but still not a good idea or good practice) others.
If I may, let's look at a example I see in your history, this this recent jQuery answer. I preface this by saying I know jack and squat about jQuery. But it seems you posted a "works for me" answer, but it is about a method officially deprecated and discouraged for use. So despite it working, people tend to downvote so as to promote other better practices. This is a good thing. People post working suggestions all of the time, but those working suggestions are riddled with worst practices, bugs or security holes, or are simply unsupported, and better alternatives exist. So those working suggestions get voted down, because they are not as helpful as they need to be for this site.
As for contesting, simply do what you did in the aforementioned answer. Post a comment, ask for clarfication. Short of receiving that, look at the other answers and see what you might learn. Often, the reason for the downvote is simply that better alternatives exist, and those alternatives are plainly visible already in one or more other answers.