What's being done to protect users' copyright?

Question

I have posted questions about how to deal with scraping sites here: Mass infringement of Stack Exchange copyright.

At the time I was helpfully redirected to this question here:

• A site (or scraper) is copying content from Stack Exchange. What do I do?

The answers there give a process for how to report scraping sites. I've used this. It's slow and works after several months. This question here isn't a question about how to currently report sites that steal content without attribution. It's about how to deal with the deluge of scraping that's happening. It's also an enquiry to find out what processes SE has to protect SE users' copyright outside of the site-reporting mechanism.

Recently, I've seen more and more sites scraping content off SE EL&U and SE ELL. There are now so many that I can't keep up with them. I think we need a public ongoing page accessible on Meta where people can list websites that are nicking content without the full proper attribution, and where TPTB (the powers that be) can get back to us once the situation has been resolved.

I'm sure this problem is affecting other SE sites too.

I am now more interested in what SE is doing to ensure that it isn't neglecting its users. I'm a bit surprised that users seem to have to find this stuff and report it. I kind of feel that it's SE's responsibility to check on a regular basis who is scraping its members' (and its own) material. Shouldn't this be a routine practice on their part? If not, the agreement that they put forward to users is kind of useless. It amounts to - anyone can scrape your content without attributing it to you under the terms set out here on SE - so long as you don't report them to us. This seems unacceptable to me.

Some scraping sites seem to be worse than others. Some link to SE, but not to the page they scraped the content from. Some list the username from here on SE but don't link to the page or mention SE or the specific SE site. There is definitely a range of violations going on. But it seems to me that SE has a duty of care towards its users - at least a moral one - to try to ensure that the terms set out on the site actually apply in a meaningful way.

What is the policy for checking up on who's nicking what from users here and from SE in general?

I have been advised that the answer will probably be that users need to take their own measures to protect their copyright. Is this correct?

Answer 1

Glad you asked. I've actually been personally working on a new tracking system that we're going to be using to keep track of all the reports we have coming in. Previously, we used a big ugly spreadsheet which was stored on Google Sheets and anyone who has ever handled scraper reports would say is a nightmare to even look at. My aim was to fix that. We only just recently started inserting records into this new system (only reports from about the past 3 months are recorded in it), and it's still not completely finished, but it's getting there. It looks a little something like this:

To be completely honest, we don't actually do a whole lot in about half the cases that get reported to us. The site usually disappears or otherwise stops working for some other reason without us ever having to contact them. The cases where we do have to take action take a while because of our process. When we contact sites using content inappropriately, we send them at least two emails before we jump to taking other actions (such as attempting to contact their hosting provider). This process was even more dreadfully slow due to the way we were organizing this information - which made it hard to see which ones had been contacted previously or need contacted again, or what our next steps might be.

As far as finding them, we don't actively look for scrapers. We sometimes run across them while we're searching for things on our own, and we'll record them ourselves if we find them. But there is no active campaign or anyone here that is explicitly searching for sites which don't follow our license terms. Our users are a much more reliable source for finding these sites, because they're the ones with the questions, the ones searching on Google for all these random things, and the ones running across these sites on a more regular basis.

To get to some of the deeper issues in your question, let's identify one thing: we cannot protect your copyright for you. We don't own any copyrights for your content. We control the license that you agreed to use when you posted it here. The only authority we have over other sites using it is the terms we outlined in that license, which pretty much boils down to providing proper attribution.

Scrapers in general are not disallowed, and if they follow all the attribution requirements outlined, there is absolutely nothing we can do about them hosting copies of the content on their site. As outlined in the scraper FAQ post, these are the things we care about when users report the sites:

• The site is a proxy.
• The site outranks us on Google.
• The site doesn't follow attribution requirements.
• The site uses a different license or claims ownership of the content.

These are all things we can actually take action on, and want to know about.

As far as a publicly accessible list: we had considered making this new system public, but opted to keep it private since we don't want users messing around with the records or entering things on their own. Allowing this just creates more work for those who are processing these, which is exactly what we're trying to cut down on by making this new system in the first place. Keeping users out of these areas would require a login system, which we simply weren't interested in building for a tool like this. Hence, it was kept private and only accessible to Stack Exchange staff.