There's an employee viewed filter and public viewed filter.
but I don't want my page to be visible to the whole wide world.
only strictly limited to the people I send out the link to.
Perhaps I'm missing something here, but your profile is either available via a URL and is therefore "on the Internet" and "public" or it isn't. In other words, barring some kind of password protection preventing access to it, if there is a link, then it is public and available to the whole wide world, assuming they know the URL.
Note: The Careers system itself doesn't provide any public (non-paid subscription) mechanisms to "discover" profiles other than through search engines that have crawled the site.
So, if what you are really looking to do is to obfuscate discovery of your profile, then I would suggest two things: