Only users with 10k reputation or above can see deleted answers if they are viewing the question you answered, and only mods can see them listed on your profile.
This is for a very good reason - so that if valuable content is removed, for example if a user decides to remove all their own posts in a fit of anger, we can flag to undelete it.
I wouldn't worry too much about the fact some users can see your post. All of us have a few bad answers we've removed after re-reading. Actually, removing your answer on review is a good sign that you're committed to giving quality answers and there's even a badge for it.