The current functionality of [practically] unlimited downvoting encourages abusive users to engage in conduct that is shady and coward. Although downvotes also decrease voter's reputation score, the truth is that it ultimately allows for unwarranted censorship of anything that does not fit that voter's ego or [capricious] liking. Two suggestions that seem straight-forward to implement may solve this issue.
No user with privilege to vote down should be allowed to issue more than 'n' downvotes during a period (for instance, a month or a quarter). The same limit should apply across a site, regardless of users' reputation score in that site. Implementing this suggestion (by adding two columns to user's "header" table: one for downvote_count, one for renewal_date) would take care of serial downvoters.
Limit the number of times user X can downvote user Y during one same period. Implementation of this suggestion will involve creating a new database table with the four following fields: (1) voter (primary key); (2) downvoted user; (3) date/timestamp; and (4) question/answer where downvoting took place.
Some (luckily few) users are obsessed about downvoting specific contributors. This happens to me in Law SE, where one self-portrayed attorney shows a recurrent impulse to downvote my answers even though his comments (when it pleases him to make any) turn out to be inaccurate, redundant, or absurd. By now I have accumulated a few screenshots reflecting that decreases of my reputation match [date-wise] equivalent increases of that user's count of downvotes.
Clearly, the purpose of SE voting is to reflect the effort or value of a contributor's posts. It is not aimed at suppressing any and all disagreements with someone else's valid, informed points. Downvoting for the sake of egotistical censorship is an abuse of SE's reputation mechanism.