If I recall the reason for voting on answers is so that answers that are "more right" float to the top while answers of lesser value sink to the bottom.
That makes perfect sense to me. I don't have to scroll and scroll to find an answer that might be close, but not marked as the answer. Positives float up and up, and votes indicate credit.
I propose you do-away with negative votes beyond -1, on both questions and answers... If an answer finds itself in negatives it's already marked as wrong... Wrong enough to not be considered right. The goal of having it at the bottom have been realized.
Beyond the negative 1 mark however; it's more of a trolling thing..
I wonder about the people who feel a wrong answerer should be punished. Why were they reading the negative posts? What value is there in a score of -5? Does the respondent get stupider? Is a - 10 more wrong than a -2?
If there's a value to the community or the questioner, then I don't see it?
So my suggestion... Don't let answers receive negative votes if it's already < 0..
The same could be said for questions.. Sometimes stupid questions come out of our mouth.. we shouldn't be punished beyond giving us less than "nothing". Let us live in shame, but don't keep smacking when we're down.