I find the downvoting system to be very permeable to abuse. Despite the loss of rep when downvoting, the cumulative effect is more felt on the person being downvoted. And this is perfectly acceptable. However, in a non perfect world like this one I've been witnessing on numerous occasions a "monkey see, monkey do" effect.
A question or answer rep remains pretty much unchanged for a while but once it is downvoted it is often followed by a series of successive downvotes. As I interpret it, users are copying the last action. The system is being twisted.
I'm not sure how prevalent this is. Only the devs could probably answer that question after doing some some data mining. But I've seen this happening on numerous occasions. Often on questions or answers that while certainly meriting a downvote, are in no way a case for a flurry of negative rep. It's particularly damaging to newcomers, who posting on the wrong forum suddenly see their perfectly valid question (albeit, done in the wrong place) being voted down to... say -6. For many of them, this becomes an unfriendly place.
I suggest the system remains unchanged. Rep is calculated the same, downvotes and upvotes too. A flurry of downvotes can still happen. However, let the webpage display at most -3, even for those cases where the true rep is actually less.
I suspect this will help curb the enthusiasm of the downvoters.
This is tied to the Peer Pressure badge and still helps give the poster a clear indication of the general non appreciation of their post. Limiting it to -2, or -1 would be probably too much since there is probably an element of "punishment" (take the word liberally please) that would be lost. But -3 seems good enough.
A really bad post should have lots of downvotes:
A really bad post should be closed, period. And anything below -3 should be considered for closing instead of yet more downvotes. Nothing is gained by this approach. Not even those who downvote since they too lose rep. What happens is, I suspect, most users that have enough rep to vote for closing down a post still prefer to downvote.
Why not just limit the downvotes to -3?
This would be my approach if I was in charge. But I'm not. In any case, limiting the downvotes to -3 may not fit well into the current system which does not remove enough rep. A bad post worth 3 downvotes would remove 6 rep for the poster. If it had 1 single upvote, it would need 4 downvotes to get to -3 and the poster would even gain 2 rep(!).