I've added bounty to the wrong question (not mine).
Is there any way to cancel the bounty if I made it only minutes ago?
Not by yourself. However, you can flag the question for Moderator attention and mention that you want to cancel the bounty (and provide the reason too), Moderators can take a look and choose to refund the bounty.
Note that in most cases moderators are reluctant to revoke bounties that've been active for more than a short time: it cheapens the meaning of them if they can disappear without warning. Also, moderators cannot revoke bounties that've already been awarded.