Deleted users, like deleted posts, revert all reputation changes that they influenced. All of their votes are removed, thus any uprating or downrating no longer are in effect. And consequently, like what would happen if the bounty question were deleted, the bounty is reverted from having been awarded in the first place.
Bounty isn't some sort of cash exchange - it essentially represents one user earning the value of the votes another user earned. When a user is deleted, votes on their posts no longer provide any reputation value, thus the bounty can no longer have a source.
I'm inclined that in keeping with how the reputation system works (as a representation of earned votes, and keeping in mind that deleted users have their votes reverted), the former is not something that should be implemented.
I'm personally of mind that the latter is a bug that should be fixed, though. Acceptance is another facet that is visually retained post-deletion, but grants no reputation. I can see the logic of mirroring this behavior with bounties, but because of the explicit numeric representation of bounties, and the lack of any other mechanical influence they have (acceptance pins answers), I don't see any good benefits, mechanical or otherwise, to retaining the bounty marker from deleted users.