Reputation freed by a deletion, both down and up, is not immediately recalculated, as the reputation shown on the page is a cache. There are recalculations taking place from time to time, especially when the reputation system is tweaked.
(Other operations that work the same way are f.e. migrations to other SE sites.)
If a user wants to do so, he or she can ask for a recalc flagging a post for moderator attention.
When the recalc takes place, all reputation issues you stated are resolved for the person recalculated: if I deleted an answer or question that was downvoted, I get refunded. If the person that downvoted recalculates, he or she gets the reputation back spent for the downvote.
Badges are not taken back once awarded. So if the recalc did not take place before the badge conditions were met, the badge is awarded. For certain badges if the system finds out that you are not eligible any longer, the next time you would get the badge it is looked at as having already been awarded.
There is always the possibility to see your "real" reputation by going to the reputation widget. For this, add /reputation to the base url of the site you are interested in, for example stackoverflow.com/reputation. It will give you a real time calc of your reputation with some interesting additional information on daily limits reached.