Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 157 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

I heard on an older podcast that (A) Google spider traffic is significant on SO, and (B) the sitemap only contains entries for the newest 50,000 pages. Would it be more efficient, in terms of cutting the Google spider traffic, to add every page to the sitemap?

  • It would allow Google to recognize the pages that have not been changed since they were indexed, without accessing those files individually.

  • It would take 40+ sitemap files, but the files with older pages could be updated only once a month or so (for new comments, etc.)

share|improve this question

You must log in to answer this question.

Browse other questions tagged .