The "vote too old" window must be increased to let people fix genuine "first impression" mistakes.
I appreciate that you're trying to solve the Tactical Downvoting problem, but this way sucks, and Dan Dyer has proposed a much better solution in his comment on the above link.
Example 1: I read the top answer. I think it's good. I upvote it. I read the second top answer. It provides not only a better answer but also shows that the top answer was strictly speaking wrong. I upvote the second answer and try to remove my vote for the top answer. OOPS! Too late now. I've permanently upvoted the wrong answer. What's worse is that because of the way it's written, and because it's currently on top this will keep happening forever, and the second top answer (the correct one) may never "win".
Example 2: I read an answer. I think it's wrong, so I downvote. Shortly afterwards I start having second thoughts and do more research, and indeed it turns out that the answer was correct. But it's now too late to change the vote.
I strongly urge you to find alternative solutions to all problems you're trying to solve by making votes permanent. I've already mentioned someone's great suggestion for Tactical Voting. Are there any others?
Edit: there are several examples (more) describing the problem of downvoting something permanently. Even that isn't always seen as terribly damaging; downvoting and then undoing the vote must be so minor that I honestly don't see it deserving a "fix" like the "vote too old".
If we need a fix for downvote-then-undo tactical downvoting (which many have denied, see links above) at all, then I'd rather have all my downvotes have a permanent effect on my rep, but be undoable (like Andomar suggested)!