10

Announcing a change to the data-dump process says that the data dump download will be gated behind the following checkbox:

"I understand that this file is being provided to me for my own use and for projects that do not include training a large language model (LLM), and that should I distribute this file for the purpose of LLM training, Stack Overflow reserves the right to decline to allow me access to future downloads of this data dump."

(bold mine)

There's a certain ambiguity here, does this "and" really mean "and", or "inclusive or"?

I see two possible interpretations:

  1. Allow non-personal use that doesn't involve LLMs, and personal use regardless of LLMs.
  2. Allow only non-personal use, and only as long as it doesn't involve LLMs.

I asked this in an answer to the question above, but also want to create a full question for it.

Also, I don't understand how this ambiguous wording could've possibly been approved by SE lawyers.

2 Answers 2

12

First off, I am not a lawyer. I do not pretend to be a lawyer. If I am one of two people in the room and only one of us is a lawyer, it is probably the other guy.

That said, I'm pretty sure this is all legal. Technically, at least. Which is the best kind of legal.

CC BY-SA allows us to use data however we want, as long as the license terms are respected. It allows us to distribute it to whoever we want, as long as the license terms are respected.

I'm sure the lawyers (of which, I reiterate, I am not one) allowed this wording because SE is not restricting our use of the data. They are simply upholding their own right to distribute it to whoever they want. In this case, they don't want to distribute it to people who are gonna use it for LLMs. And they reserve the right to not give it to us if they think that's what we're gonna use it for.

It's important to remember, CC BY-SA does not force users to distribute anything, ever. It simply requires that if they choose to distribute it, it must be under the same or compatible license terms. If Stack Exchange wanted to just… not distribute the data file ever again, to anyone, legally there's no reason not to.

It would be a terrible move that violates one of the core principles of Stack Exchange, one that would almost definitely cause a network-wide riot, but a legal one nonetheless.

That said, it doesn't really matter what the "AND" means here since they can legally just refuse to distribute the file to you or anyone else for any arbitrary reason whatsoever if they really felt like it; if you're dealing in good faith you're probably fine but if you try to "Well, technically…" your way through literal loopholes they can just block you whether you deserve it or not. It doesn't actually prevent you from using the data file that you've already received for whatever purpose you want.

8
  • 9
    TL;DR You can use the data dump however (commercially, non-commercially, for LLMs, any way really) you want with the new terms as long as you comply with all CC BY SA requirements, if Stack Overflow does not like your usage though they might not give you the future versions of the data dump. Commented Jul 27 at 12:13
  • @AbdulAzizBarkat what stops one from creating a new account and obtaining the dumps if such restrictions were applied to an account? Genuinely asking. Commented Jul 29 at 3:33
  • 1
    @M-- I am not a lawyer but from what I understand from a legal standpoint nothing really ("Stack Overflow reserves the right to decline to allow me access to future downloads of this data dump." mentions they are not giving up their right to select who to distribute to but nowhere are you agreeing to stop after they declined once). Obviously distributing or not distributing the data dump is up to Stack Overflow (the company) and if they have some technical measures to prevent people from using multiple accounts to bypass this, they are within their right to do so. Commented Jul 29 at 5:07
  • 1
    IMO this isn't going to stop people from redistributing the data dump anyway. Either the company is being stupid or they are being malicious in the sense that some time after moving over the data dump they'll say something like "We don't see much downloads for the data dump and hence have decided to remove it". Commented Jul 29 at 5:13
  • @M-- probably the same kind of technical ways they have to guard against you creating voting rings full of sock puppets, or to prevent suspended users from just signing up again. Commented Jul 29 at 19:24
  • 1
    @testing-for-ya that works against an individuals (to an extent). All of this feels like the beginning of dooms day. I have to agree with Abdul, the day they would turn off the data dumps altogether is not that far in the future. As AMtwo mentioned, if it wasn't for the backlash, they (read CP) would have already done so. Commented Jul 29 at 19:30
  • @M-- I have no arguments there. I'm just addressing the question of how they would do it if they wanted to (try to) prevent access by specific users. Commented Jul 29 at 20:02
  • 1
    @AbdulAzizBarkat Eh, I think it might be something other than that. I don't think it makes much sense for them to "song and dance" at this point around trying to keep the dumps around if there wasn't a concerted effort by at least some prominent folks at Stack to truly keep it around. Working a tech position at a big "business-y" company myself, I suspect this move is mostly about delivering on upper management concerns, while still doing what they feel is best to keep the dump around... despite our collective protests about it being a terrible means of doing so.
    – zcoop98
    Commented Jul 31 at 17:42
6

I'd note that the dumps have been used to bootstrap other sites in the past, and practically there's nothing stopping commercial use in the current licence - the CC-by-SA as opposed to one of the CC licences that does not allow commercial use.

Practically there may even be some benefits there since the commercial project using the data would need to attribute the original source.

Also, to be blunt, unless it's a direct threat to revenue (possibly via the current favoured product), its unlikely the company would take action.

I'd also point out depending on your needs, there's other places one can get SE post data - old data dumps are out there and there's various mirrors aside from the IA copies. You could also get the data dumps from elsewhere - since SE can't really police downstream sources. There's nothing in the letter, or spirit of the changes in data dumps that prevents this.

I am also pretty sure many companies scrape pages which renders the use of the dumps moot anyway

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .