In any case, we protected a question that had received a couple spammy answers in quick succession, and the notice reads:
This question is protected to prevent "thanks!", "me too!", or spam answers by new users. To answer it, you must have earned at least 1 reputation on this site.
From the reputation tables in this FAQ post, this seems to be the way it works for private betas in general. Is it possible that at some point in the past, guest posts were allowed on private betas? I couldn't find another private beta site to check, but on this site guest posts are not allowed, so protecting questions doesn't actually impose any restriction on who can answer.
Here's an excerpt from the help center entry for the privilege:
What are protected questions?
A protected question prevents answers being added by anonymous and very new users.
Based on that description, this is a bug.