(Note: the quoted sentence was edited after I posted my answer. I am leaving this answer unedited and undeleted, so that it is available for folks reading the revision history.)
From the original post:
Many words have been written around the company's commitment (or perceived lack thereof) to the ongoing existence of the data dumps, the API, and the Stack Exchange Data Explorer (SEDE).
I find the inclusion of the "or perceived lack thereof" parenthetical to be incorrect, and unnecessary. At best, it is an unnecessary "dig" at members of the community who questioned the missing data dump earlier this year.
As I mentioned in my answer on that question, I was the individual who pressed the metaphorical & literal button to disable the data dump upload job. As such, I take that dig as directed (at least partially) at me specifically.
Because I find that statement as both factually incorrect and targeting me specifically, I do feel the need to address that comment.
Leading up to the disabling of the Stack Overflow Data Dump, I had a number of conversations with Prashanth, specifically around the Data Dump, the API, and SEDE. I have specific firsthand knowledge of the decision to disable the Data Dump, as well as some of the research that went into that decision.
I believe that the implication made by Philippe, that the Data Dump was never going to be disabled long term, is both untrue, and damaging to my reputation as a data professional, as a dutiful employee of Stack Overflow, and as a member of this online community.
Among the questions I fielded from Prashanth included:
- Can we pull down the existing data dump from the Archive?
- Can we limit who can access the Archive?
- How do we limit what data people can download from SEDE?
None of those are direct quotations, mind you. They are simply based on my memory of the calls we had discussing the topic.
March 28, 2023
On the morning of March 28, Prashanth contacted me via Slack DM to ask me to disable the data dump, and mark it as not to be re-enabled without his specific approval.
During that conversation Prashanth said (this is a direct quote, with my emphasis adding italics): "how long does it take to re-establish the link [to Archive.org] if we want to open it back up?"
As part of my response, I said (also a direct quote): "But if we don't upload on schedule, we're likely to have someone notice and ask about it on Meta. So we need to be prepared to respond--or better proactively explain it on a Meta post of our own. The Community Team probably just needs to be in the loop to not be caught off guard with the customer service side of it."
I was assured that the good folks on the Community Team would be informed and given the chance to communicate, after which point I disabled the data dump upload job.
...IF WE WANT TO OPEN IT BACK UP
I don't think it's necessary for me to say any more. Folks who questioned the company's commitment to the Data Dump were not only justified to ask questions, but they were correct.
The company was NOT fully committed to continuing the Data Dumps, nor were they committed to involving the community in that decision, nor was the promise to me fulfilled to work with the CM team to ensure things were properly communicated in advance.
I am thankful that the company has reversed the plan to disable the Data Dump, and that they are now committed to ensuring they continue going forward. But it is not true to imply that commitment has been unwavering, or that people were incorrect to doubt them.