Skip to main content
3 of 13
The phrasing here was causing confusion - after discussing with Philippe, this was the intended way of explaining this situation..
  • 108.6k
  • 46
  • 292
  • 414

Much has been written lately of the company’s decision to pause the distribution of the anonymized data dump that has historically been posted.

Our intention was never to stop posting the data dump permanently, only to pause it while we begin to collect more information on how it was being used and by whom - especially in light of the rise of LLMs and questions around how genAI models are handling attribution. However, it’s clear that many individual users (academics, researchers, etc) have an immediate need to access updated versions of the dumps. So we are re-enabling the automatic data dumps (and uploading the one that’s about a week overdue). We believe that this can happen by end of the day Friday. We will continue to work toward the creation of certain guardrails (for large AI/LLM companies) for both the dumps and the API, but again - we have no intention of restricting/charging community members or other responsible users of the dumps or the API from accessing them.

As part of this project, API users should be on the lookout for a very brief survey that will be coming out (announced here and on that asks about the features that you most use/would like to see in the API or data dumps moving forward so that we can plan for those, as well as collect general input.

In the meantime, the data dumps will be re-enabled by end of day Friday. We will communicate here when that has been completed or if there are any delays. We will also post here prior to making any future changes to the dumps or distribution of the dumps.

Philippe StaffMod
  • 20.9k
  • 15
  • 63
  • 86