If one kept track of last few up and down voters to show others it could stop people from upvoting themselves or disgruntled persons from downvoting. e.g., downvote cause I don't like you, go to your other posts and downvote them too(because you would see their name show up in your posts as downvoters).
What is meta?
- Any Stack Exchange user can ask a question
- The community provides support, votes on ideas, and reports bugs
- Your voice helps shape the way Stack Exchange operates
Why would you want to stop people from voting? Voting is how the Stack Exchange platform ranks content to the top of the page or pushes it down to the bottom. If votes are not anonymous, then this discourages people from weighing in and honestly helping the system operate as it's designed.
As long as you're doing what you're supposed to do, writing good questions and answers and following the guidelines in the FAQ, you're reputation score will continue to increase.
Also, if you're dealing with digruntled users, I suggest possibly reflecting on your own behavior. If you're leaving constructive, helpful comments, people are less likely to lash out at you:
I'm not suggesting you have an attitude problem or that you're doing something to warrant being on the end of someone else's abuse, but it's worth some self-reflection. Did you recently make an insulting comment somewhere, perhaps? Most of the people who I see on Stack Overflow who are polite don't seem to have a problem with being the victim of serial downvoting. :) It does happen occasionally, but like all other risks, it can be mitigated without fundamentally changing the voting system.