Downvoting requires you to have a certain amount of rep which is completely reasonable but downvoting after you mistakenly upvoted a question or an answer should not require you to have some rep. I don't see any point. If you upvote an answer or a question, you should be able to downvote it too, maybe not after ages but within a reasonable period of time. Maybe after you upvoted, the question/answer was edited and sounded wrong to you. So, you decided to change your vote. It should be a ok.
To Err is Programmers.