Could have sworn there used to be a heads-up when you got down to one vote, but it sure doesn't work that way now. Very frustrating, to sit there considering a question, finally decide to vote, only to find out that I've been out of votes for a while.
Ideally, this would work the same way as up/down and comment votes do: when I get below a threshold (5 sounds good to me), let me know how many votes remain each time I cast one...
George Stocker suggested a change that would also work to solve this: show the number of remaining votes on the close dialog itself. I like this idea, as it avoids creating more pop-ups while still providing the necessary information.