Dear SE, please stop disparaging the moderators in the press

Question

The VP of Community for SE gave the following statement to the press:

Stack Overflow ran an analysis and the ChatGPT detection tools that moderators were previously using have an alarmingly high rate of false positives. Usage of these tools correlated to a dramatic upswing in suspensions of users with little or no prior content contributions; people with original questions and answers were summarily suspended from participating on the platform. These unnecessary suspensions and their outsize impact on new users run counter to our mission and have a negative impact on our community.

This statement is not correct. Specifically, it conflates the use of ChatGPT tools with suspensions, implying that moderators use those tools to decide about suspensions. This is not correct, while these tools were potentially part of such a decision, the decision itself is a manual analysis.

The statement also claims that the suspensions were unnecessary even though SE told us moderators explicitly that they don't know that. What SE said before was that they cannot be sure those suspensions were correct, which is a very different situation than what SE describes.

This statement misrepresents the facts and thereby disparages the moderators that have acted on AI-authored content. It would be nice if SE could be more accurate with facts when speaking to the press, and avoid disparaging the moderators on these sites.

Correction: The article previously attributed the statement to the CEO, now the attribution is changed to the VP of Community.

Answer 1

Moderators worked with Stack Exchange to develop the GPT policy

I fully expect that page to disappear soon so I'll paste it below.

A few important things to take away from this policy:

• "While this is the position of Stack Overflow staff, it’s meant to support the prior work done by moderators (namely, the temporary policy issued to ban contributions by ChatGPT)."
• "Currently, contributions generated by GPT most often do not meet these standards and therefore are not contributing to a trustworthy environment."
• "In its current state, GPT risks breaking readers’ trust that our site provides answers written by subject-matter experts."
• "Moderators are empowered (at their discretion) to issue immediate suspensions of up to 30 days to users who are copying and pasting GPT content onto the site, with or without prior notice or warning."

Moderators didn't just dream up a policy here. We worked with the community team to get it added to the help center. We worked with the company to codify suspensions for utilizing GPT. We worked with them to validate the few suspensions that were appealed and (importantly) brought back to us for discussion. It is my understanding that no GPT suspensions have been overturned.

In short, the moderators had the rug pulled out from under them with this recent change. No prior communication indicated we needed to adjust how we were moderating. No prior concerns were raised about any way we were detecting and removing GPT content.

This Help Center article provides insight and rationale on our policy regarding the usage of GPT and ChatGPT on Stack Overflow. While this is the position of Stack Overflow staff, it’s meant to support the prior work done by moderators (namely, the temporary policy issued to ban contributions by ChatGPT).

Stack Overflow is a community built upon trust. The community trusts that users are submitting answers that reflect what they actually know to be accurate and that they and their peers have the knowledge and skill set to verify and validate those answers. The system relies on users to verify and validate contributions by other users with the tools we offer, including responsible use of upvotes and downvotes. Currently, contributions generated by GPT most often do not meet these standards and therefore are not contributing to a trustworthy environment. This trust is broken when users copy and paste information into answers without validating that the answer provided by GPT is correct, ensuring that the sources used in the answer are properly cited (a service GPT does not provide), and verifying that the answer provided by GPT clearly and concisely answers the question asked.

The objective nature of the content on Stack Overflow means that if any part of an answer is wrong, then the answer is objectively wrong. In order for Stack Overflow to maintain a strong standard as a reliable source for correct and verified information, such answers must be edited or replaced. However, because GPT is good enough to convince users of the site that the answer holds merit, signals the community typically use to determine the legitimacy of their peers’ contributions frequently fail to detect severe issues with GPT-generated answers. As a result, information that is objectively wrong makes its way onto the site. In its current state, GPT risks breaking readers’ trust that our site provides answers written by subject-matter experts.

Moderators are empowered (at their discretion) to issue immediate suspensions of up to 30 days to users who are copying and pasting GPT content onto the site, with or without prior notice or warning.

Answer 2

Also from that article:

In a statement sent to Dev Class, Stack Overflow’s Beaudette told us:

“A small number of moderators (11%) across the Stack Overflow network have stopped engaging in several activities, including moderating content. The primary reason for this action is dissatisfaction with our position on detection tools regarding AI-generated content.

This is simply false. It has been made abundantly clear to staff that the issue is not a desire to rely on detection tools, unless they are including moderators' judgement and expertise in "detection tools". Some public descriptions, published since the article, of how this is false can be found here and here. They explain the issues very capably, and so I will not retread them here. This is not a new explanation; it has been moderators' position from the start that we know the detectors are inaccurate and require other evidence to support any moderation action. What we want is to be able to apply our judgement in the manner we do when handling any other moderation issue on the site, as well as to be able to discuss potential problems with our handling of issues instead of having blanket policies rolled out with no warning or previous discussion.

---

^{Also, it's now just shy of 17% of moderators across the network, but according to Zoe the 11% number was accurate at the time.
(yes, I know that's still potentially misleading as to the percentage of moderator activity on strike; no, I don't want to discuss that in the comments here.)}

Answer 3

Frankly, I find this quite frustrating. Stack Exchange already promised us a blanket no comment policy in 2019, then revoked that and replaced it with a more elaborate "no comment on individuals" policy in 2020 (see the accepted answer to the linked question). While the new policy was well-received at the time, I think this incident clearly demonstrates that the company does not understand why the original "we won't shoot off the other foot" policy (as Shog9 described it) was necessary in the first place, and is instead content to play semantic games with the more elaborate policy of 2020.

Talking about individuals is just one kind of PR malpractice. It is a particularly egregious one, so it is important to prohibit, but it is just as important to ensure that your statements are true, or at least true to the best of your knowledge. Here, we have a straightforward case of SE, Inc. saying one thing to the moderators in private (which I have not read), another thing to the rest of us on Meta, and yet a third thing to the press. Even though I can't see the private message, there are blatant and obvious differences between the second and the third statement. For example, the on-meta statement says that "[t]his standard would exclude most suspensions issued to date," which is nowhere to be found in the statement to the press. Obviously, I don't expect them to quote the entire policy at the press (although linking them to it would probably be a Good Idea), but failing to mention that you're undermining pretty much all prior enforcement of the existing policy is a huge omission.

In short: They can't even keep their public statements consistent with each other, and this is why commenting to the press was a bad idea.

Answer 4

NOTE: This is addressed to SE

Stack Overflow ran an analysis and the ChatGPT detection tools that moderators were previously using have an alarmingly high rate of false positives. Usage of these tools correlated to a dramatic upswing in suspensions of users with little or no prior content contributions; people with original questions and answers were summarily suspended from participating on the platform. These unnecessary suspensions and their outsize impact on new users run counter to our mission and have a negative impact on our community.

Please do not violate your own misinformation policy. Let's read through it. You manage to violate nearly ever section. Nice job.

Is likely to significantly contribute to the risk of physical or psychological harm to a person or a group of people

Many mods have their real name as their username and them being part of this group repeatedly disparaged in the press by a billion dollar company makes life more than a bit harder and causes harm.

Promotes widely disproven claims...

Your claim that the moderator suspended users unfairly because of GPT detection tools is false. You are unable to even give a single example. If, out of something like 30 million users, you can't find a single one that was unfairly suspended because of a GPT detection tool, then that really shows that even you know your claim is false.

Promotes the specific views of a party, government, or ideology by using false claims regarding others

You are currently promoting your own views by using false claims regarding moderators. Literally a fill in the blank.

The end

This means that with that paragraph alone, you have managed to violate 3 out of 7 of the clauses in your own policy. I can't find a single example that does worse. Stack Exchange, you will have no credibility if you don't even try to follow your own policy. So, yes, please stop disparaging the moderators in the press. Thank you.

Answer 5

They don't care about power users or mods or even take us seriously.

This is illustrated by the fact that they don't reply properly to any concerns or queries raised whenever a situation like this occurs.

They care about bringing in new users or finding new ways to monetize what they currently have, no matter the cost.

It's completely fine to malign power users and mods.

It's completely fine to outright lie to the press.

But we dare you to be "unwelcoming" to new user users.

They've forgotten that it's the community that makes these sites thrive, not the empty suits. But it seems that with time - and money no doubt - they've forgotten this.

This isn't the first time they've thrown us or our opinions under the bus, and it won't be the last.

• Stack Overflow is doing me ongoing harm; it's time to fix it!
• SE Inc. Director of Public Q&A retweets bad-faith, misrepresentative tweet
• Firing mods and forced relicensing: is Stack Exchange still interested in cooperating with the community?
• Revisiting the "Hot Network Questions" feature, what are our shared goals for having it?
• We are graduating the updated button styling for vote arrows

I'll be surprised if they even reply to this post.

Walk away. It's not worth it.

Let's see SO invest in proper moderator teams instead of relying on the good graces of volunteers.

---

Going forward, we will be working with the community to overhaul how we gather input and feedback from our moderators and members of the community to make sure that your voices are heard and involved in the process, not just informed after decisions have been made.

An apology to our community, and next steps

We want the relationship between the company, the community and its moderators to be based on open, transparent communication that will be made in good faith. I believe the deterioration of communication and trust has been a problem for quite some time. I believe that re-establishing transparency and open, two-way communication will be a key ingredient in rebuilding the relationship between the community, moderators, employees and the company.

To all of the moderators who have resigned or suspended your activities over the past few months: your presence and impact is missed. We value all of your work to keep your sites clean and communities healthy. We understand the many reasons why you felt that it was necessary to step down and that it was a painful decision. We are working on many of the issues that influenced your decisions to leave, and we aim to back these intentions up with actions, accountability, and consistent open communication. If you feel that your issues continue to go unaddressed, I invite you to post about them on Meta in a respectful way. And if you choose to apply for moderator reinstatement, we look forward to hearing about this as well and to seeing you back on your sites.

The company’s commitment to rebuilding the relationship with you, our community

https://meta.stackoverflow.com/questions/342903/well-always-endeavor-to-do-whats-right-well-try-to-do-it-better-next-time

We care about the concerns of the community [...]

https://meta.stackexchange.com/a/349473/218388

I can't be bothered finding more examples. Feel free to add them in. Maybe this could be added to The Many Memes of Meta.

---

Who knows, if I'm still around I'll catch you all at the next inevitable fiasco and we'll rehash it all, once again.