Stack Exchange's built-in search returns results really really quickly. Are the pages spidered and indexed? Is there some schema optimization in use? I'm just curious.
What is meta?
- Any Stack Exchange user can ask a question
- The community provides support, votes on ideas, and reports bugs
- Your voice helps shape the way Stack Exchange operates
They're using the new mind-reading features in .NET 4.0, so the server already knows what you're going to search for (and starts looking) before you even hit the submit button.
But seriously. :)
I believe they're using SQL Server full-text search (is this covered in one of the FAQs?) so yes, everything is "spidered and indexed"...
edit here's a blog post that mentions them in the same sentence... :)
Keep in mind that SO doesn't have to search eleventy-billion or so web pages like Google does. It's more like 350,000 questions. Many databases are much larger than that.
Mainly because it's not a very good search. It is easy to return bad search results fast.