When a question has many answers, the Stack Exchange software will split them onto several pages. As of mid-2009, the second and later pages have included a
rel=canonical link tag pointing to the first page.
One effect of this is that search engines, such as Google, treat links to any of the pages as if they pointed to the first page. It also means that, if users search for a phrase in the question (which is shown on all pages), they won't see duplicate results.
Unfortunately, this use of
rel=canonical links has another effect: it makes any answers that fall on the second or later pages impossible to search for.
For example, if you search for a distinctive phrase from this second-page answer, all you get are SO mirror sites! Meanwhile, searching for the accepted answer works fine. This may not seem like much of a loss, since answers on later pages tend to be of lower quality (although sometimes they might simply be new, and therefore not upvoted yet), but it does lose us occasional "long tail" hits from people actually searching for something found in those answers.
This (ab)use of
rel=canonical might have been a useful SEO trick back in 2009, but there are better alternatives nowadays. The two solutions recommended by Google are:
- provide a "view all" page and point the
rel=canonicaltag to it, or
rel=prevlinks instead of
rel=canonical, as described here.
In fact, in this recent blog post and video, Google's Maile Ohye straight up says that the way we're currently using
rel=canonical is considered "improper usage":
"While it’s fine to set rel=”canonical” from a component URL to a single view-all page, setting the canonical to the first page of a parameter-less sequence is considered improper usage. We make no promises to honor this implementation of rel=”canonical.”"
(See also the video starting at 11:42.)
Thus, I suggest that we get on with the times and fix our pagination, either by providing "view all" pages or by using the appropriate
prev link elements.
- Original proposal to (ab)use
rel=canonicalin this manner (to fix duplicate search results)
- Older meta post from late 2009, pointing out the problem but providing no alternative
- Webmasters.SE post that originally brought this to my attention
In his answer below, balpha lists some problems that he suggests implementing this request would cause. Let me quote them here and respond to them point by point:
- Not marking duplicate content as such.
This issue does not apply to the solution using a canonical "view all" page, which in any case is the preferred solution suggested by Google.
It is doubtful whether (after the recent algorithm updates described in the links above) it applies to the
prev solution either: having a common header on each page of a paginated content sequence is very common, and I would be surprised if Google had neglected to deal with it appropriately. In any case, if in doubt, the easy solution would be to just ask Google (e.g. on their Webmasters forum) what they recommend.
- Adding irrelevant search results to the 99.9% case of searching for the question.
This is a non-issue for either of the proposed solutions. With a canonical "view all" pages, that page would be the only one appearing in search results. As for
prev, Google's Maile Ohye writes:
"When you implement rel="next" and rel="prev" on component pages of a series, we'll then consolidate the indexing properties from the component pages and attempt to direct users to the most relevant page/URL. This is typically the first page."
The official documentation says the same thing, namely that using
prev on a sequence of pages will generally lead to Google "consolidating their linking properties and usually sending searchers to the first page".
- Giving validity to those endless, almost forum-style question threads that we're on a mission to rid the Internet of.
I can't argue with this point, since it's primarily philosophical rather than technical in nature. I can only ask whether it's really worth making those threads effectively unsearchable.
- Spending time to work on (and risking Google punishing us for) solving a non-issue.
That's two issues; let me address them separately:
Yes, implementing either of these solutions would require some work for not much gain. However, the time and effort required should also be minimal (probably comparable to the time it took me to write this post, or for balpha to answer it): on one hand, we already provide
prevlinks in the body of the page, just not in the head where Google wants them; on the other hand, while implementing a good pagination system is a non-trivial coding exercise, providing a non-paginated alternative view should be a five minute job after that. (OK, make that fifteen minutes — it needs to be tested and deployed too.)
Google is not going to punish us for following their explicit recommendations. There is, however, a risk that Google may some day decide to start punishing us for doing what we currently do, which is (and always has been) against their guidelines for using
rel=canonical, and which is uncomfortably close to a known black hat SEO technique.
Admittedly, this is unlikely to happen soon or without warnings, if only because we're far from the only site currently (ab)using
rel=canonicallike this. (It's included in mistake #3 on this video of five common SEO mistakes.) Still, it's worth keeping in mind.
I'd also like to point out that Stack Overflow, while being the biggest SE site, may not be completely representative of other sites using this software. For example, on CodeGolf.SE beta, the very nature and purpose of the site often leads to large numbers of answers. One might well argue that this makes the Stack Exchange software poorly suited for this site (or vice versa), but — given that the site does exist and seems to be thriving — surely that's no reason to cripple the software any more than necessary.