What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

I'm cross posting this: http://stackoverflow.com/questions/1972815/canonical-links-and-paging

I just did a google search for "press the Alt button" site:stackoverflow.com, unexpectedly it is missing the hit from page 5 of one of the most popular questions.

Is this canonical link feature playing nicely with paging?

share|improve this question
    
Does anything play nicely with paging? Do questions with more than 1 page of answers play nicely with... anything? –  Shogging through the snow Dec 29 '09 at 4:01
    
The amount of hoops a site that supports rich filtering and sorting needs to jump through to keep search engines happy is enough to make me a real sad panda. I just made my response permalinks route to a different area all together just so I could include it in the robots.txt file –  waffles Dec 29 '09 at 4:06

1 Answer 1

up vote 2 down vote accepted

It's unlikely Google will index pagination of the form

http://stackoverflow.com/questions/9033?sort=votes&page=5#sort-top

because of this in our robots.txt:

Disallow: /*?
# above line, refer to http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360

relevant section of page, quoted:

To block access to all URLs that include a question mark (?) (more specifically, any URL that begins with your domain name, followed by any string, followed by a question mark, followed by any string):

User-agent: Googlebot
Disallow: /*?
share|improve this answer
    
Fair enough, is this good? bad? on one hand the you have content that is not indexable that is valid. on the other, the fix is mega hairy. –  waffles Dec 29 '09 at 4:08
    
note: globbing is not really in the robots.txt spec proper. some robots will not support it. –  waffles Dec 29 '09 at 4:09
3  
"answers" that are not on page 1 (that is, within the top 30 responses) are rarely useful. So no, I don't really have a problem with answers 31-500 not being indexed by Google. –  Jeff Atwood Dec 29 '09 at 8:12

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .