What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

There have been many questions about the Stack Overflow sitemap, but this question is not like those. My goal is not to figure out how the sitemap works, I just want to look at it (don't worry, I'm not going to abuse it, like the issue here!). I looked at http://stackoverflow.com/robots.txt and found the reference to http://stackoverflow.com/sitemap.xml, but I get a 404 when I try to access the sitemap. How does this work? Can you explain where the sitemap is? I see a comment in robots.txt that that reference to sitemap.xml isn't technically valid - if the real reference is in another place, what is this place? (Also, this question may belong on a different site, please migrate it if so.) Thanks!

UPDATE: From the answers, I've concluded that the sitemap is actually at http://stackoverflow.com/sitemap.xml, but access is restricted to only googlebots. Thanks!

share|improve this question
    

3 Answers 3

up vote 5 down vote accepted

Unless you're Google, Bing, or another valid search engine, why do you need to see it?

http://en.wikipedia.org/wiki/Sitemaps

The Sitemaps protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. A Sitemap is an XML file that lists the URLs for a site. It allows webmasters to include additional information about each URL: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site. This allows search engines to crawl the site more intelligently. Sitemaps are a URL inclusion protocol and complement robots.txt, a URL exclusion protocol.

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156184

Sitemaps are a way to tell Google about pages on your site we might not otherwise discover. In its simplest terms, a XML Sitemap—usually called Sitemap, with a capital S—is a list of the pages on your website. Creating and submitting a Sitemap helps make sure that Google knows about all the pages on your site, including URLs that may not be discoverable by Google's normal crawling process.

share|improve this answer
    
I don't "need" to see it, I'm just curious as to how it: a.) functions - how does the whole extension to other sitemaps thing work (I believe this was mentioned in one of the podcasts), and b.) how you're referring to it so that search engines can see it? I'm building a new site and would like to have such sitemap things, so this becomes a programming question. Could you point me to any online resources or tutorials about this? –  Maxim Zaslavsky Sep 16 '09 at 23:44
15  
I don't consider Bing to be a "valid search engine" :P –  Super Long Names are Hilarious Sep 16 '09 at 23:52
6  
I'm still curious what "abuse" the sitemap resulted in, which caused it to be restricted. –  cletus Sep 17 '09 at 0:07
    
OMG! Teh Googles has a typo! –  Super Long Names are Hilarious Sep 17 '09 at 0:29
    
@cletus while(1){wget .../sitemap.xml} seems like an obvious abuse of a large, single file –  dbr Sep 14 '10 at 12:42

There is currently no public-facing site map. The site map is the 6 buttons at the top of the page, your name link to your profile, and the glorious little envelope. There aren't too many dark corners around here...or if there are, nobody's told me about them--which is par for the course.

share|improve this answer

If my memory serves me correctly, the SO sitemap isn't what most people think of when they think of a sitemap. It's not a couple nested unordered-lists showing links to your profile page, etc. If I remember correctly, it's a massive page containing hundreds of questions used for quick-indexing of SO-content, hence its restricted access.

share|improve this answer
    
Oh! I understand! –  Maxim Zaslavsky Sep 16 '09 at 23:44

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .