I see that not only myself, but other people have accidentally added bounties to the wrong question. Can we add the ability to cancel a bounty within the first few minutes of creating it? (Or perhaps 30 seconds if you feel 1 minute is too long)
If you just started the bounty, you can flag the question where you placed it with a custom flag. Describe why the bounty has been placed in error, and moderators can remove it and return the reputation.
This is pretty easy for us to handle, as long as the bounty hasn't yet been given to someone. However, we are hesitant to remove bounties after they've been around for a while, particularly if people have started writing answers on the bountied question.