15

I'm subscribed to Google Alerts for my own name - for no particular reason that I can remember.

Today I received an alert that pointed to https://meta.dev.superuser.com/ which has a different IP address to http://meta.superuser.com:

> nslookup meta.dev.superuser.com
Server:  UnKnown
Address:  192.168.42.129

Non-authoritative answer:
Name:    meta.dev.superuser.com
Addresses:  64:ff9b::9765:4146
          64:ff9b::9765:c146
          64:ff9b::9765:8146
          64:ff9b::9765:146
          151.101.193.70
          151.101.129.70
          151.101.1.70
          151.101.65.70


> nslookup meta.superuser.com
Server:  UnKnown
Address:  192.168.42.129

Non-authoritative answer:
Name:    meta.superuser.com
Addresses:  64:ff9b::9765:8145
          64:ff9b::9765:c145
          64:ff9b::9765:4145
          64:ff9b::9765:145
          151.101.193.69
          151.101.1.69
          151.101.129.69
          151.101.65.69

It looks like development version of the site as when I try to login it responds with a Stack Overflow login page:

Looks like the Stack Overflow dev site, prompts for a staff login

Why is it redirecting to the wrong site?

More importantly why is it (a development site) being indexed by Google?

Update: Trying to access https://meta.dev.superuser.com/ leads to a certificate error. Skipping past that leads to a site that is the exact same as https://meta.superuser.com/, but with the same styling as Signal Processing (main site):

Meta Super User but with Signal Processing design

For a while, it was looking the same as a meta site for a graduated site without a design.

5

Hmm, there was an invalid setting for that dev site which allowed anyone in, including Google.

It's been set to private again. Will investigate how it was set improperly. Expect yelling and gnashing of teeth.

Sorry for the inconvenience!

  • 2
    So it took five months to find invalid setting. I have a feeling this is something that could have been fixed in hours, not months. So just wonder how to better draw your attention in similar cases in the future? For all we know, such things can cause major leak of private data. (e.g. if dev site contains private data as it's accessible only by staff.) – ShaWiz Sep 28 '17 at 6:14
1

It's indexed because the robots.txt on the domain meta.dev.superuser.com (https://meta.dev.superuser.com/robots.txt) allow googlebot to crawl that domain, but its restrictive.

Disallowed agent are listed below and I don't see the google-bot (Googlebot), and please open the robots.txt for all restricted path for all bots

User-agent: Mediapartners-Google Disallow: /

User-agent: Yahoo Pipes 1.0 Disallow: /

User-agent: 008 Disallow: /

User-agent: voltron Disallow: /

User-agent: Googlebot-Image Disallow: //ivc/ Disallow: /users/flair/ Disallow: /jobs/n/* Disallow: /jobs/a/* Disallow: /jobs/companies/n/* Disallow: /jobs/companies/a/*

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .