I believe it is a bit of both.
The site itself is different from traditional forums, and therefore instantly gets attention. The reputation points and badges also has a lot to do with it's appeal.
This attracted users who at the end of the day are brilliant, or very proficient in a particular area of expertise, or even users that are mediocre but still know the answers to common questions because they had to find those answers before. This means that even in the smaller areas you will find at least one person that can answer a question.
In my experience SO has been great from the beginning, however I did follow it since the very first podcast came out, so also understand the original goals behind the sites. I do believe that it's the combination of users and design that make it work. It does however need both. Even with the best design in the world, without it's users it will be just another site. Ask anyone trying to launch a SE site at the moment.