449

Since day one of Stack Overflow, all content posted on Stack Exchange sites by their users (i.e. you wonderful people) has been provided to the whole universe under the CC BY-SA license. For my fellow non-lawyers, that license basically means:

  • Anyone can use any Stack Exchange posts at any time without having to ask for permission
  • Making money off of the copied content is permitted
  • You don't even have to copy stuff from here verbatim; you can just use it as a starting point and make whatever edits you want
  • There are just two rules you have to follow:
    • You have to provide attribution. Simple links to the original post and author info are just fine.
    • You have to link the license and allow other people to use your content, as long as they follow these very same rules. How meta!

(If you ever forget any of that, and want to refresh your memory, the license info is linked to in the footer of every page.)

There are, in fact, a lot of people who republish varying amounts of our content for assorted reasons. Unfortunately, there are some Stack Content Republishers Attributing Poorly and/or Excelling at Ranking (SCRAPERs, for short).

In this context, "attributing poorly" means any use that doesn't follow our attribution rules or make any other reasonable attempt at give credit. This can get pretty egregious; I've seen SCRAPERs that not only don't link back to SE originals, but also use fake author info and post dates to make it harder to find originals. By "excelling at ranking," I'm referring to copycat sites that end up higher in Google results than the original SE sites do for the same content. That's not necessarily wrong, but in some cases, it indicates inappropriate SEO hackery.

So, the question is: what can you do if you spot a SCRAPER?

Return to FAQ index

1

1 Answer 1

324

What is a "scraper" and why is that bad?

Historically, SCRAPER here on Stack Exchange meant "Stack Content Republishers Attributing Poorly and/or Excelling at Ranking." More generally, a scraper is another website which copies content from our sites either by scraping directly from our pages, accessing the information through our API, or some other means. In principle, there's actually nothing wrong with doing this. Our content is licensed under Creative Commons BY-SA 4.0 and is freely redistributable, so long as they follow the attribution requirements and link back to us as the source. Some sites, however, do not follow these rules or bring up other concerns in the process.

Should I report these sites to the Stack Exchange administration?

You should only report the site to us if the site is a proxy. Whatever the purpose of the site actually is, malicious or not, proxies represent a serious security threat to our site. Sometimes they even start showing up in Google results and users click on them not realizing that they aren't actually on Stack Overflow. Users get confused, or try to log in and accidentally send sensitive information to a third-party service.

When should I not report these sites?

  • You landed on the malware page. We've already detected and thwarted quite a few sites, and when attempting to access them they will redirect you to this page on our network to notify you that the site is not the official site. You only need to contact us from this page if you think you landed there by mistake because you think the site you were attempting to access is blocked erroneously.

  • The site doesn't follow attribution requirements, uses a different license, or claims ownership of the content. Unfortunately, at this time, there is no action we can take as we do not own the content ourselves, and reports regarding such content are no longer accepted or actioned. You may, however, be able to take action yourself if you are the owner of the content.

What actions can I take myself?

There are some actions you can take yourself:

6
  • 4
    @BernhardBarker Neither really. A proxy is essentially a site that acts as a middleman. You send a request to them, they send the request to us. When they get the response, they send it back to you. It gives the appearance of our site being exactly duplicated on their own domain.
    – animuson StaffMod
    Commented Apr 25, 2022 at 15:28
  • 1
    If there's a list somewhere of such proxy/fake sites, we could at least add it to our search-result-filtering browser extensions Commented May 14, 2022 at 11:57
  • 1
    How would it be possible for me to tell whether or not a site is a proxy, without waiting to spot a change on the real site and trying to spot that same change on the fake site?
    – Niko O
    Commented Oct 7, 2022 at 9:46
  • 1
    Doubtful it still makes sense to report them, since SE won't do anything with the reports in the vast majority of cases.
    – Mast
    Commented Oct 9, 2022 at 14:34
  • @NikoO you can look at the domain name. Proxy site will have a different name
    – Стас
    Commented Jun 28, 2023 at 21:52
  • 1
    @Стас That... doesn't really answer my question. Like... of course the domain of a website that isn't e.g. stackoverflow.com will be different from stackoverflow.com. I'm asking how to discern a proxy (i.e. forwarding requests to the real site on the fly) from a non-proxy (i.e. displaying a copy of data that was scraped from the real site over time).
    – Niko O
    Commented Jun 30, 2023 at 4:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .