How many people being able to answer a question does it take for this question to be valid?
None. The suitability of a question is not evaluated based upon the number of people who can answer that particular question. There are other far more important factors we consider, all listed in the FAQ.
The benefit of using the factors we do is two-fold. First, they are decidedly more objective in judging the quality of the question based solely on its own merit, rather than the speculative merit of potential answers. Second, they aren't going to change as the site's user (and therefore knowledge) base grows.
The consequence of having a site intended for beginner- and expert-level questions alike means that there will probably be a non-trivial number of particularly difficult or tricky questions that can only be answered by a limited audience of users. That's perfectly okay. We do want all questions to eventually get answers, and we want to encourage the posting of those answers in whatever way possible, but we certainly don't want to create a rule forbidding or discouraging people from posting high-quality useful questions just because there might not be enough people [yet] who can answer them.
That said, certain questions that only a limited number of people can answer might still fall under the domain of "too localized", not just because of the limited number of people who can answer them, but because of the nature of the question itself. For example, if you ask a really tricky question about the behavior of the C# compiler, good money says that not very many people are going to be able to [correctly] answer your question—you'd better hope Eric Lippert happens across it. On the other hand, asking when C# 6.0 is going to be released is not a good fit for our site for entirely different reasons.