A question with a bounty has already been opened for at least 48 hours. In a busy community like StackOverflow, this is more than enough time to reasonably determine if the question is a good fit for the community.
On a smaller site, like Project Management SE, moderators see every question that comes in; thus, we have a 48 hour window to figure out if we need to unilaterally intervene when the community cannot.
Once the bounty starts, as Shog9 mentions, there is a contract that exists between the asker and answerers that offers X amount of reputation in exchange for research and effort invested in providing the best possible answer. It makes sense during this time period to allow the question to remain open, at least for the duration of the bounty period.
This is a temporary state. Once the bounty period ends and the answerers have an opportunity to receive their bounty award, the community can then take a hard look at the question and determine it's future.
It's been established that any rep earned for answers with a least 3 votes will be preserved if a question is ultimately removed. Thus, if the same rule applies to bounties, then this shouldn't be a problem in terms of violating this bounty contract.
UPDATE:
Once a bounty is posted, it is locked until the end of the 7 day period. The contract has started, and we shouldn't violate that contract.
However, one possible solution could be to prevent bounties being offered on any questions that contain a single flag or close vote. A moderator would have to clear these flags before the bounty period could begin. This would effectively prevent an answerer from losing time spent researching the solution.