I didn't expect such a level of ingratitude.
Nothing tells you that question asker downvoted you. It could've been any of the competing answerers, and the reason could've been any of the following: tactical, ideological or whichever reason people use when they downvote.
My own reason for downvoting is when the answer is obviously wrong or misleading and may actually make damage to anyone who think that it is correct.
My solution for the problem you described is to answer the best I can or am willing at the moment anyway.
And if I receive offensive/insulting/leeching comments from the OP after that I either delete the answer or simply ignore it and tend not to answer that person's questions anymore, or both.
When it comes to downvotes, it all comes down to the fact that they are anonymous, so you can't really do anything about it. Anyone (almost) can downvote, and some of the people are cool and know stuff, while some are narrow, overly competitive or simply [expletive]s. That's the human nature, no way changing it.