People are free to vote as they see fit. They can downvote an answer / question if they think it is not helpful. If you honestly think your answer is perfect, then you may want to request clarification in the comments below your answer:
It would be nice for the downvoter to explain why he/she thinks this is an unhelpful and/or incorrect response so I can, perhaps, improve it.
But make sure you do that politely. If you demand an explanation, that might result in more downvotes. If nobody replies, just wait. If your answer is actually helpful, it will eventually gain more upvotes.
Secondary problem with this, is that users can not "up-vote" or "down-vote" an answer unless they have at least 200 rep
That's not true. Vote Up privilege requires at least 15 reputation. Vote DownVote Down, on the other hand, requires at least 125 reputation.
I think it would be a GREAT idea if some accountability were set on down-voters such as forcing them to enter in a reason as to why they voted the question or answer down
As Pekka notes in the comments section, this was one of the first feature requests here on Meta: Encouraging people to explain downvotes
And to have a reporting system in place for invalid down-votes as this ruins the reliability of the site overall.
Even if the downvotes are "invalid" as you say, you shouldn't take them personally. Voting is done (at least supposed to be) on the basis of the quality of the post, and not on the person who posted it.
