In order to get a specialist badge, one must accumulate at least 400 upvotes for a single tag. Jeff set the bar extremely high because answers to questions tagged subjective tend to get upvoted a lot, and he didn't want to give out a bunch of subjective tags the first day. He excluded Community Wiki questions from the counts for similar reasons.
Four hundred upvotes is a lot. If you only answer questions with a particular tag, never ask a question, and none of your answers are accepted, you will have accumulated a reputation of 4001 -- easily within the top 1000 Stack Overflow users at this point.
Realistically, you probably won't have a specialist badge if you haven't done enough work to put yourself in the top 100. It seems the only way to get a specialist badge is to specialize in Stack Overflow.
I think there should be another path for people who have answered only a few hundred questions or specialize in areas that don't get much traffic. As the title suggests, I propose that the specialist badge also be awarded for at least n accepted answers for a particular tag.
I think the number of accepted answers is a better measure than the number of upvotes. The latter can be skewed by the popularity of the question(s). The former shows that you've helped people solve problems -- repeatedly -- even if the problems only garnered the attention of a few.