Would we prefer to have a site that strictly focuses on one topic but gets little use or a site that has a more relaxed focus and gets a larger audience?
I have to reject your assertion.
It is exactly when a site is small that QUALITY is more important than ever. When the site is small, every question asked appears on that front page for a long time to come — So EVERY question plays a HUGE role in the design of the site.
It's that front page that will drive what types of questions people will emulate when they see what the site is about. The "wrong" type of questions will only encourage more of that type of question to be asked.
So saying that we should relax the controls that define a smaller site will only send it in the exact opposite direction it needs to go. The "quality content" that defines a Stack Exchange site comes from the hard work of the community behind the scenes, making sure the site provides something beyond the mundane, uninspired questions asked 100 times one every other phpBB forum out there. That doesn't just happen spontaneously.
If you believe nothing else, believe this:
Lots of questions is not Job One!
Encouraging (or even allowing) uninspired, off-topic questions is not a sustainable way to build a site. They may pump up the question count in the short term, but users will quickly tire of the banality of it all and leave in droves. The absolute lowest priority in all those analytics you see on a site is the question count. How well a site force-fed a bunch of low-quality questions is not going to cut it. Pity that people still think that is the end goal.