17

Possible Duplicate:
How does Stack Overflow work so well with search engines?

I notice that every time I or some else asks a new question on StackOverflow it only takes a few minutes to a half hour (at max) to have the question page and subsequent answers indexed by Google.

Moreover, the newly indexed questions manage to be listed either at the top or near to the top for keywords that match the question's content - which is understandable because of a combination of StackOverflow's PR and typical content authority in question phrasing.

Where I'm dumbfounded is how StackOverflow manages to get itself indexed so rapidly and uniformly by Google? Is individual pinging enabled on StackOverflow each time a new question is entered? I don't believe a sitemap is utilized (accessing sitemap.xml results in a 404 [however that might be caused by not being a search engine user agent])?

Any insights about how StackOverflow manages to achieve such excellent SEO would be great. Even better, we could turn this into a generalized discussion about the various methods to achieve similar SEO and which tend to produce the best results (pinging on new content, metadata, blackhat - cloaking, etc.)?

I'm looking forward to hearing some interesting input on this!

Improve this question
0

3 Answers 3

Reset to default
5

This was addressed at Webmasters not too long ago

Sites with high PR and are frequently updated will have their sites crawled more quickly then normal. So it's not a technique, per sé, so much as it is a result of the site being popular and constantly changing.

Improve this answer
1
  • Even with all best practice and techniques suggested by Google, you don't get a page into the index that has been created 30 seconds before. Nothing to do with page rank, which has been abondoned anyway arround 2009. I have read somewhere that Stackoverflow and Google have a special contract that shifts SO content into a Googlebot fastlane.
    – Daniel W.
    Jan 13, 2020 at 17:31
2

It's indexed this fast simply because it's good. Stack Overflow has original, high quality content and tons of traffic. The site is also fast, a thing which Google really likes. Google is directly interested in crawling the site as fast and as often as possible since it's relevant to most programmer queries. Also, from what I know from the old podcast, sitemap.xml is only available for search engine crawlers.

Improve this answer

Not the answer you're looking for? Browse other questions tagged .