As already stated, Bounties do not guarantee correct or high-quality answers. Bounties simply highlight your question and therefore can lead to more exposure and better answers.
Can I retract/refund my bounty?
The first place to look for should be the bounty help page for you.
It very clearly mentions that:
All bounties are paid for up front and non-refundable under any circumstances.
That should make it clear that you cannot take back the bounty because it was paid right when you started the bounty and will not be refundable no matter what are the circumstances.
What if my bounty does not result in good answers?
The help page also makes it clear that Bounty does not guarantee good answers, it only provides you a place in featured tab:
Part of what you’re “paying for” with the bounty is to get
additional attention for your question, over and beyond what a
normal question gets. A bounty does not guarantee a response,
however, and reputation refunds are not available if no answers are
received as a result of the bounty.
Okay, what if I don't award it at all?
Since it appears that you did not manually award the bounty, the help page answers for that scenario as well:
If you do not award your bounty within 7 days (plus the grace period), the highest voted answer created after the bounty started
with a minimum score of 2 will be awarded half the bounty amount. If
two or more eligible answers have the same score (their scores are
tied), the oldest answer is chosen. If there's no answer meeting those
criteria, no bounty is awarded to anyone.
If you do not award it yourself, it falls on the community to judge the answers for the bounty. The answer which was awarded the bounty has a score of three and therefore fulfills the criteria for the automatic award.
So the simple answer is, you cannot remove the awarded bounty from answers which you deem unworthy. Community has judged the said answer and found it helpful. You simply do not get a say in it if you do not award it yourself or accept an answer yourself (In which case the answer you accepted would have been awarded the bounty).
Is it unfair?
You paid for 7 days of a slot in featured tab, you got the slot and exposure. The answerers answered to win the bounty and community found two of them helpful so the one which received more votes got the bounty since you did not care to award it yourself within the grace period. What's unfair there? Only thing unfair would be if people got the option to retract the bounty because that would then serve as a loophole. Start a bounty, get exposure, get more response and then retract the bounty. Can you understand how it is unfair and messy?
Sense of entitlement
Also, it is evident from some of the comments there that you were acting in a very unnecessary, aggressive and entitled manner there. Always keep in mind that everyone is a volunteer here, no one is getting paid to help you. Offering a bounty does not mean community owes it to you to provide good, correct or even bad answers. If they are bad answers, that's what the downvotes are for but apparently community disagrees with you and thinks they are good/helpful answers. There is a sense of entitlement here which is wrong and misplaced.