18

I was recently looking for an old post on gaming.stackexchange.com, and one of the things I remembered was that the game "SingStar" was mentioned. Thus, I typed "singstar site:gaming.stackexchange.com" into google, expecting to only see a dozen posts or so. Instead, I was greeted with hundreds of PAGES of results:

enter image description here

Only one of the shown results is an actual post.

Having the tag-wiki page indexed makes sense, and maybe the tag-FAQs. However, the tag search-pages are just noise, especially the ones which combine Tag1 + Tag2, for every possible combination with a question. I think these should be marked with a noindex.

Improve this question

1 Answer 1

Reset to default
15

This isn't a bad idea at all, starting with the next build we'll be returning a 404 to crawlers looking at multiple tag /questions/tagged pages so we're not spamming the result list with duplicates. So (related to tags) you'll start seeing 2 top results, the wiki and the list of questions for that tag (assuming we're a popular result, of course).

Now keep in mind that this won't be an instant fix, it'll take a while for google to re-crawl these and remove them from their index, but it will happen.

Improve this answer
3
  • Has this been done yet? I'm still seeing these results on google.
    – BlueRaja
    Jul 8, 2011 at 15:57
  • @BlueRaja - it has, but it'll take some time for it to drop them from the index...since it's a header and not a robots.txt block (which is far more immediate)
    – Nick Craver
    Jul 8, 2011 at 16:52
  • Out of curiosity, why did you end up choosing this method of blocking rather than the more usual noindex? Doesn't returning lots of (false) 404's hurt your SEO?
    – BlueRaja
    Aug 19, 2016 at 15:42

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .