The number of votes on your question is your answer; however, I will elaborate.
In actuality there are two main reasons why you're getting downvoted. The first is easy: you reward effort rather than good answers. Stack Overflow is about answers, not effort. You're free to upvote your way, but that way is not the general Stack Overflow way.
The second is the deeper and more important reason, which is that just because you take the time to answer a question doesn't mean that question is good. You can answer bad questions. Maybe you personally don't, but other people do, myself included.
As a matter of fact, I've downvoted more questions I've answered than upvoted. For example: this very question.
Needless to say, while you might answer only good questions, others do not necessarily do the same. A comment asking people who have answered to upvote is not a welcome comment, especially if those who answered considered your question bad.