Even though I was originally in support of this feature, we have to decline this for a few reasons.
Downvoting isn't a good tool to combine with bounties.
I had originally suggested coupling the two, as a way of the bounty creator showing to the system that they really think the answer is not useful rather than just being able to simply click a button and have it disappear. However, these two systems wouldn't work well together.
Mostly because it would introduce a few unexpected user experience issues to the whole system that would make thing uncomfortable. The current bounty award system is fairly straight-forward, and anyone can figure out exactly why a bounty was or wasn't awarded by looking at some information. Downvoting is anonymous, though. Users can see there is a downvote on the post, but not who cast it. By linking downvoting to an explicit "no award" feature for the bounty system, we effectively remove the anonymity of downvoting for whoever creates the bounty on the question because users would be able to deduce "well I was eligible, I didn't get it, so they must have downvoted me."
As pointed out in the comments, requiring users to downvote in order to avoid awarding a bounty would also introduce the implicit requirement of needing at least 125 reputation after you created the bounty and lost however much of it.
We've had something like this before.
Long ago, users were able to award the bounty to their own answer. It was effectively a "do not award it to anyone" feature because they didn't get the reputation back, but it also didn't give it to anyone else. It was not well-liked and we don't want to return to the endless drama that it can cause.
Remember, a bounty is really a payment for advertising.
Whether you award it, the system awards half of it, or it just goes to waste - you paid for the attention that your question received, not the explicit ability to decide its ultimate fate. When half the bounty gets awarded, it doesn't get attributed to you. It gets attributed to Community, because the community's voting on the answers is what decided it would be awarded. So we're not putting any words in your mouth here or indicating that you endorse the answer in any way, and there's little other reason not to allow the system to just take control if you've failed to award it yourself.