I recently saw a question on which the accepted answer had 13 votes, and the next answer had 24 votes. It was a relatively old question, so I figured I'd do the next answerer a favor. I downvoted the accepted answer and upvoted the next answer, and voilà, they got the Populist badge.
Now I'm feeling kind bad for downvoting the top answer, as it was a perfectly good answer. I don't think what I did was "right", so I'm wondering what the community thinks of a case like this.
EDIT: I went back to the question, edited it and removed the downvote. At least this way, I can undo the damage.
Now that it's back to 13/25, will the second answerer lose the badge? The initial downvote/upvote thing happened a few days ago.