Why am I not allowed to change my vote after some period of time? There are times when I go back to things I voted on, and decide they didn't deserve that specific vote. (I get wiser as I grow older)
The justification of Jeff to prevent the "tactical downvoting" is not good enough.
It could be prevented by different approach, as I suggested in http://meta.stackexchange.com/a/135243/163139
The time limit to undo up-vote could be completely removed.
When user tries to undo downvote, it should be allowed after time limit(currently 5 min), but do not refund reputation point.
These changes will allow users to freely undo their up and down-votes but still will prevent unhonest "gamers" to restore points by undoing down-votes.
Basically it's to stop people gaming the system.
One of the ways people did this was by down-voting rival answers and then removing that vote when their answer gains popularity. There are others.
See the following blog entries: