Sometimes you start a bounty and, since the problem is difficult and people try to reach the bounty, there is higher probability than usual that you get very low quality answers. Sometimes these answers even get some upvotes, thanks to a lot of people viewing it. When the problem is so difficult that no one solves it, the investor does not award the bounty and implicit bounty award takes place. It would be extremely humiliating for the investor to see that one of the bad answers is getting the bounty. Therefore I propose to add the following rule:
Do not implicitly award bounty to an answer that has been downvoted by the investor.
I agree this will not probably happen often, but I have actually seen bad answers taking two or more votes. And I think it doesn't make any sense to award an answer which is bad according to the investor, so I think this case is worth handling.
EDIT: I wouldn't mind because someone will gain rep. What bothers me is that answer which is apparently wrong (possibly have been too quick in the urge to win the bounty) will be marked as bounty winner and thus confuse the visitor, giving the feeling that this is a good solution and the question is answered and OP satisfied.. This would be sick! In case of the accept button you have it under control but this can become really weird especially for 200+ bounty.