source | link

I think part of the problem is that the effect of voting is +10/-2, but the number next to the question/answer is +1/-1. Say an answer gets 2 upvotes and 5 downvotes, they've gained 10 rep, but it their question/answer is at -3, so it's unlikely anyone else will vote it down unless it's trolling. Don't really know how you'd tackle that short of subtracting .2 for each downvote and then rounding.