Why don't I reclaim my downvote penalty when the post I downvoted is deleted?
This is just a small niggling annoyance, but it happened to me today that I downvoted a clearly incorrect answer on SO (leaving an appropriate comment to explain my downvote). As expected, I lost one point for the downvote.
However, later I came back to the question and realized that the answer I had downvoted had been deleted. My lost point was still gone, however.
Now, I don't mean to quibble too much about a single point, but it seems to me that deletion of an answer is a tacit admission that the downvoters were correct. In this case, wouldn't it make more sense to restore points lost due to downvoting, in the name of not penalizing successful quality control?
Also, a related question--what happens to the two points that my downvote should have subtracted from the answerer? Has he still lost them, even though his answer no longer exists? At the very least, there should be parity here.