Statement from SO: June 5, 2023 Moderator Action

Question

Update: GPT on the platform: Data, actions, and outcomes

---

As you may be aware, a number of moderators (on Stack Overflow/across the network) have decided to stop engaging in several activities that they had taken on, including moderating content - in fact, almost all moderation tasks. The primary reason for this action is dissatisfaction with our position on detection tools regarding AI-generated content, and discontent with how that was rolled out.

We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives, which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions. People with original questions and answers were summarily suspended from participating on the platform. We stand by our decision to require that moderators stop using that tool. We will note that it appears to be very rare, however, for mods to use ONLY the ChatGPT detection tool, and frequently their own analyses were in use as well. We will continue to look for other, more reasonable tools and are committed to rapid testing of those tools and any suggested heuristic indicators.

The moderators who are engaged in this action served this community collectively for many years on the platform. Personally, I consider a number of them friends, and anytime friendship is tested like this, it’s difficult. I would like to say to them clearly that I hope they know how much I, and the whole staff and community, appreciate their collective decades of service to this community, and I hope that we are able to come to a path forward. I regret that actions have progressed to this point. The Community Management team is evaluating the current situation and we’re working hard to stabilize things in the short term. We’ll be working collaboratively with the community on the long-term solution.

I’ll be honest, the next few days and weeks might be a bit bumpy. Both sides share a deep and unchanged commitment to quality on this platform. Additionally, it’s important that we guarantee that anyone who has meaningful contributions to make has the opportunity to do so. As we have updates, I and the team will be sharing them here.

Answer 1

People with original questions and answers were summarily suspended from participating on the platform.

We've been asking for data here for a week. As far as I know SE has not provided a single case of an unjustified suspension. Rather the argument has been that it is impossible to detect AI-generated content at all, and therefore it is impossible to produce data on unjustified suspensions here. Which is a very frustrating point to argue against.

This statement here contradicts what SE has told us, it is a statement of fact that alleges that suspensions were unjustified. I expect from SE to back this statement up with data or retract it.

Answer 2

We ran an analysis

Alas. This is the moment where I miss the times where y'all used to publish Data Science times! (see e.g. November 2018 and December 2018) which not only showed some professional and responsible analysis of the data that you had, but did so in a way which was transparent to the community.

Ever since the announcement of the new policy imposed upon moderators, preventing them from using their skills to handle the surge of AI generated content, there was still no disclosure of the research that would justify it.

ChatGPT detection tools have an alarmingly high rate of false positives,

Moderators know this, and they do not make decisions based on such tools alone. In fact, some moderators claim that they don't rely on automatic detectors at all. This makes the argument a non sequitur.

which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions

And rather than interpreting this as "hey look, a lot of people are creating accounts and posting lots of answers with ChatGPT to exploit the system", it is appalling that your reaction to this was "hey look, a lot of people are suddenly interested in contributing to the site and moderators are suspending them for no good reason". In a scenario where a platform with gamification is already pretty loaded with answers to most questions, and an alternative platform (ChatGPT) is available to quickly obtain answers to the most frequent and simple ones, even Occam's razor would have guided me to the first interpretation. This is the part I would consider not very responsible. And without concrete evidence of undue suspensions taking place, it is also not transparent.

At this point, in the absence of factual data and research to back the claims made, this question would have been a bit more honest if it did not include the second paragraph. Because I understand that "the damage is done", and reverting the policy alone won't revert the damage, making that too a bit reckless. I shall wait to see what's coming next.

See also:

• Is it true that moderator overreach is turning away a large number of users?

Answer 3

This post is in violation of our Misleading information policy. Relevant sections quoted:

To ensure the integrity of our network, we do not allow any content that promotes false or misleading information or coordinated misleading information campaigns.

Broadly speaking, we do not allow and may remove misleading information that:

• Is likely to significantly harm democratic institutions or voting processes or to cause voter or census suppression
• Promotes disproven claims of election fraud or manipulation as factual or severely misrepresents the safety or validity of results from voting machines or other voting processes
• Promotes the specific views of a party, government, or ideology by using false claims regarding others
• Misrepresents, denies, or makes unsubstantiated claims about historical or newsworthy events or the people involved in them

Please correct this error.

• We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives, which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions.

  That is not what that word means. There is no correlation. Quoth Wikipedia:

  In statistics, correlation or dependence is any statistical relationship, whether causal or not, between two random variables or bivariate data.

  You have observed a fact, and a trend, and believe them to be related. What you have is a hypothesis, and many alternative hypotheses exist. You are playing off the popular conflation of "correlation" and "causation" to make this sound more impressive than it actually is.

• The primary reason for this action is dissatisfaction with our position on detection tools regarding AI-generated content, and discontent with how that was rolled out.

  No, that is the primary focus, goal or aim of the strike. It is not the primary reason; merely the proximate cause that broke the camel's back. You are downplaying the myriad reasons – systematic problems stretching back years – that led us to this point.

• People with original questions and answers were summarily suspended from participating on the platform.

  You have not provided evidence of this. You have been asked repeatedly, and last I heard (from you!), Stack Overflow had no evidence of this. Either substantiate your claims, or retract them.

• Your statements to the press, and leaked internal messages, make it clear that this is a deliberate, coordinated campaign of misleading information.

---

On an unrelated note:

• We will continue to look for other, more reasonable tools

  We've tried to explain to you that no such tools exist, and no such tools can exist in public (nor be created with the resources available to Stack Overflow). Any non-avalanche discriminator can be run backwards, and used to cause that discriminator to give arbitrary output; and empirically, most transformer-based tools can fool most other transformer-based tools.

  If you do not understand this by now, you as a company do not have the expertise to be working with AI.

  and any suggested heuristic indicators.

  I can confirm that this, at least, isn't made up. Discussions about heuristic indicators have occurred, where suggestions have been made, and the CMs have been looking through those discussions (and even participating).

• I regret that actions have progressed to this point.

  We know that you, personally do, Philippe. But the company overall does not appear to regret it sufficiently; or else it would have acted on that regret.

  Update 2023-06-12: I now have reason to doubt your sincerity. Friends don't allow – or, perhaps, write – this kind of internal communication about their friends, even if they're "just doing their jobs". I have to say, though: props. I really believed you. (And I'm still inclined to forgive, if we get an apology.)

  Update 2023-07-26: From a 2023-07-20 Moderator Strike update:

  Apparently, the statements sent out in Philippe's name did not undergo his approval and were put together by a PR team.

  My ire may be misplaced. (If that's true, I'll owe Philippe an apology.)

Answer 4

The primary reason for this action is dissatisfaction with our position on detection tools regarding AI-generated content, and discontent with how that was rolled out.

No, it isn't, and I think the community would appreciate if you didn't misrepresent the actual reasons for the strike, which have been clearly expressed. The primary reason for the strike is because Stack Exchange, Inc. has imposed a secret policy which de facto allows behaviour which the community overwhelmingly thinks should not be allowed.

That is, regardless of whether "detection tools", subject matter knowledge and experience, the user's post history, post frequency, or other evidence shows that a user has plagiarised from an AI tool, the users caught doing it can't be suspended.

You have nothing to gain from lying to us about our own motivations, so honestly this looks more like you are talking past us, and lying to some other audience about us.

Answer 5

You write:

The primary reason for this action is dissatisfaction with our position on detection tools regarding AI-generated content

I will remind you that the position we are dissatisfied with comprises not just the public position on detection tools, but very importantly the broader policy and interpretation, made clear only in private, that goes far beyond particular detection tools and effectively forbids treating the vast majority of AI-generated content as unwelcome on our sites, overruling community-determined policies on individual network sites.

I hope this was an inadvertent lack of clarity in your post here rather than an intentional misrepresentation of our position; understandably there are a lot of different pressures on your time and it's often possible to dissect written words more deeply than they were intended. Thank you for acknowledging that these tools are not the only basis for identifying AI generated content, I see this as an important step forward that was not clear in previous company statements here or to the press.

Answer 6

I am writing this not just a reply to this specific post, so please consider the following in the bigger scheme of things that happened in the last years. Also note that unless otherwise stated whenever I say "you" I am referring to the company as a whole and not to any specific individuals.

Premise

Let's just start with a simple premise. I no longer trust your words when you say that you didn't expect this, when you say you are sad about the outcome and so on.

I don't trust you. I can't trust you any longer after every single time you shove in our face the same theatrical act, with the same characters, the same plot, and the same twists.

Don't get what I am talking about?

Stack Exchange Scandals 101: a proven formula

Historically, most recent scandals have been following the same formula, with some minor changes every now and then.

• The staff announces some controversial changes out of nowhere. The announcement is usually made on Meta.SE and it is carefully crafted to be as invisible as it can imaginably be to the rest of the network. Optionally, the post is made because some meddling user caught the company red-handed. If an official post is made, it will be written in the most confusing way possible so that users won't even be able to agree on what was said ("did they want to prohibit moderation of AI generated content" VS "did they suggest to stop using detector tool until we can work out a better policy" is just a recent case).
• Corollary: whenever possible, the user base is expected to get only an incomplete picture of the issue. If applicable, mods will get some full guidance that contains the more controversial parts that they will be required not to share.
• The announcement is met with a huge backlash as multiple users start posting related questions on Meta. The company enters the "silence" phase — no answer is given, no comment posted, no "we heard you, please give us time to write back" is said. This phase usually lasts a few days as the company "sits-out" on the issue while letting the bigger fires die down.
• (Optional: while ignoring its own community, the company still finds time to talk with some external media news site and publish a misrepresentation of the reality that constitutes actual slander against a single or a whole category of users. Most of the time this seems to be done as a PR move to show that the company is doing their best on social issues.)
• After some time the company comes out with a "We are aware of the issue" post that on surface looks not completely unlike an apology. Under more careful scrutiny it is more often than not a non-apology in the form of "I am sorry if you were offended" or an indirect attempt to share the blame like "It is sad that we escalated so far". The post also carefully avoids giving out any info about what is planned to do to actually resolve the problem.
• At the same time the users start to plan their action: sometimes this can involve an actual moderation strike or users using their profile to spread the word.
• In a desperate attempt to reduce the visibility of the protest, SE revokes the right that was given to mods to advertise posts that they deemed important for the community to see. Removing links for a solidarity fundraising campaign from users' profiles, unfeaturing posts, anything goes in this step.
• The fire continues until the company concedes on some (minor) details. More often than not this is not a real win for the community, just an "it is better than nothing" scenario with most users preferring to accept whatever little Scoobie Snack the company dispensed instead of going into an endless fight. Sometime it may even go as far as look like a fake concession that was already planned to make the userbase more yielding in whatever negotiation is going on at the time.
• The company then starts a final memory-clearing campaign. The community will be rendered unable to fully know what actually happened, sometimes even going as far as to use law to be sure that no one can disclose the full picture by force-handing the actual victims into silence.

Obviously any new tire fire is different, but I expect them to more-or-less follow this same script. And with that behind us, let's move on to...

A Bastard's explanation (yep, that's me)

I know that what will follow will be seen as evil by some users. "Never assume ill intent" they will say.
Yet let me also tell you that there is an old saying that goes like "Once is ingenuity, twice is stupidity, three times is malice" or, if you prefer another version, Hotoke no kao mo sando - "Even with a Buddha nature (you can forgive) only three times".
With that said, feel free to downvote the following if you wish, but I will try to explain what all of this feels like and why I think history keeps repeating itself.

Assumption #1: the attempt to make problematic posts less visible is deliberate

First of all, let's start with a basic assumption: the company knows in advance what topics will cause backlash. It is quite evident from how they are presented. Why is there no secrecy involved when they announce some new feature they are proud of, or when the company makes some donation to charity? At the same time, things like unilateral changes to existing content licensing are posted without any attempt to actually make them visible and followed by a total silence while waiting for the bigger flames to go down.

Assumption #2: the actual motivations are never the ones being told

Let's go a little further. We know that the company knows that whatever they are going to post is problematic. But how can they be so sure that it will be problematic?
There are only two possible answers:

• The issue can be easily misunderstood, but if that is the case I would expect that instead of closing up into silence you would want to try to clear up confusion. Why keep your users angry if you really think that you can agree?
• The issue is clearly going against the will of most users, either by removing their agency to moderate content, by shoehorning unwanted content on them, or other similar things.

Since I don't think the company is stupid I will have to exclude the first irrational behavior. That leaves us with the second one.

Assumption #3: the company already knows in advance how the users will react and then try to act surprised when it goes that way

This is just a direct consequence of what we just said and is confirmed by the fact that the scripted non-apology post will usually contain some carefully written lines to look surprised, like "We didn't expect this" or even better "We didn't expect we (== you) would actually escalate to this".

You can see where this is going.

And a Tin Foil Hat one: the hidden agenda

By this point I have to admit that all so far gives a fairly bleak picture.
Thinking back at how Monica was picked out and slandered on a news site, thinking about how the company apologized for that only to do that again now, and looking at how users now even have to say that "actually they never promised to not slander user groups anymore, only individuals".

I can not reasonably avoid seeing some form of actual malice at work here: it is my blunt conclusion that none of these changes were actually made in the interest of improving the site content or the user experience.

What I see instead is a reiterated attempt at pleasing the stakeholders in the worst ways possible.
The Monica case was never resolved, and still today it looks like Monica was just a carefully picked scapegoat that the company decided to use to get some free advertisement in an attempt to present itself as caring for the rights of the LGBT+ community that not only backfired on them but also on the ones it was supposed to protect by further fueling rage and abuse.
Years later we now had to resort once again to a new moderation strike, and once again we see the company name-calling its mods to put the blame on them. Once again we see them post multiple inaccurate representations of mods' actions on the Internet to shame them and circulate the message that "The company is improving even if some (bad) mods are trying to fight back". And I admit that I went as far as to expect someone to be used as the scapegoat again to be the new Monica.

Sadly, I don't think that I am just hallucinating things as the feeling that these actions are part of a hidden agenda that not only is willing to go against the users for the sake of profit but is also quite open to publicly shame and even sacrifice them, seems to be quite a shared sentiment nowadays based on the comments I read on the site. Promises are usually made to stop the fighting but personally I find it rather unclear whether those are actual concession the company had to make, or just planned eye-candy, smoke and mirrors, used to make the userbase yield and accept the "better than nothing" resolution.

Also it is not unusual for the company to hide themselves behind grammatical technicalities worth of the "Evil Overlord" manual: after all, after the Monica scandal they just promised to never again slander an individual on the Internet, so slandering all the moderators is fine, right?

Conclusion

Over the course of the last years, the company has made their moves to show off how nice they are, how they care about defending others, how they work on social issues. Things like the Be Nice policy, revising how new users are guided in their first steps... all great things. On paper.

Yet, in the current scenario I can't see them as anything other than empty actions that didn't actually come from the heart but just from a need to "show off", to get media to talk about how Stack is doing the right thing, to win more users -> more ads views -> more money.
At the same time the userbase is portrayed as the enemy, whose toxicity causes nightmares, and who makes newcomers feel unwanted and leave.

I'll ask you a question...
Have you ever thought about the nightmares Monica probably had? Ever considered the users that went off the site after each of these scandals to never return? Ever listened to the mods asking you to not be toxic to them?

Once again, I ask you to stop and reconsider. You can work with the users and you can choose to make actual improvements together. The media awareness that you are "good" needs to be an effect of your good actions, not the target that you have to reach by any means by faking things — then happy users will suggest your site to their friends. Currently, the only thing I can suggest is to never get involved with an actual account here.

You probably still have time. But I fear that the candle is reaching its end.

Answer 7

As a moderator, I don't think I've ever used an automated GPT detector when investigating flags from users highlighting potential uses of AI to generate answers. And yet, my moderation practices are now all banned by the new SE polices. This makes no sense to me.

The false positive rate of automated GPT detectors is irrelevant if I don't use them. I don't understand why I'm prohibited from using practices that don't ever involve any use of any automated GPT detector, ever. A high false positive rate would be a reasonable basis for prohibiting mods from using automated GPT detectors; but it is not a reasonable basis for the current policy, which goes far beyond that. I shared this feedback with the company privately as soon as Stack Exchange informed moderators of this new policy, and I have not yet seen a response.

I remain dissatisfied both with the process of rolling out the new rules, and the substance of the new rules.

Answer 8

We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives, which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions. People with original questions and answers were summarily suspended from participating on the platform. We stand by our decision to require that moderators stop using that tool. We will note that it appears to be very rare, however, for mods to use ONLY the ChatGPT detection tool, and frequently their own analyses were in use as well. We will continue to look for other, more reasonable tools and are committed to rapid testing of those tools and any suggested heuristic indicators.

So, share the data. You obviously have it, and you obviously have it tagged, if you've gone through impact analysis on it. What are you waiting for?

At this point, it's the third time that I've heard in a week that there's an "unacceptably high false positive rate" without ever hearing:

• What the rate was before ChatGPT went mainstream
• What the rate is today
• How you know that these suspensions were made by using a tool
• Standard statistics values (r², etc.) to assess correlation

So far, there's been a lot of "moderators bad" and very little quantitative data for mods to even consider or try to see your side. You've provided unquantifiable, unqualified hearsay, which, if I'm not mistaken, is in breach of your own Code of Conduct.

Like, I don't want to be mean or anything, but the intended reaction to this repeated exposure to unsubstantiated statements that aren't even remotely verifiable has a name: Illusory truth effect

Answer 9

I came across the news of the strike on a different site, and I thought I could resist pointing out that the company is making exactly the same mistakes it made in 2019, but I can't. I'm probably wasting my breath, because I tried to make these points in 2019 when I was a mod. Maybe repeating them will help.

• Stop making official policy statements off the cuff in the Teacher's Lounge. That chat room should be for mods to get support from each other and the company, not for official company policy setting. Not every moderator participates in that chat room and chat messages are inherently a terrible channel for official announcements.

The company should make it clear that participating in chat is optional, and that mods who don't will still be included when discussing and distributing official company policy.

• Stop making unsupported assertions disparaging volunteer moderators in the press. Surely you can talk about what you believe is good for the future of the company without throwing the volunteers who made Stack Exchange the success it is today under the bus.

The data behind the assertions that new users were being unfairly suspended by moderators should be made public, or the assertion should be retracted and apologized for.

• Do not unfeature posts on site metas. It doesn't help contain or stabilize the situation; It just makes people angrier and more frustrated.

There should be a process that's followed prior to staff reversing the actions of site moderators and the first step should be a conversation with the moderators.

• Stop treating volunteer moderators like they work for the company. Site moderators are elected representatives of their communities, not extra labor that can be directed to achieve the company's goals. They know what their communities need to stay healthy, and rightly prioritize that.

The company should not be dictating policies intended to prevent moderators from using their judgment to do what is best for their communities. Either the mods can be trusted or they can't. If the company can't persuade the experts that a course of action is good for their community, maybe it's a bad idea.

A few parting thoughts The company does not act like they value the expertise and dedication of the volunteer moderators as a whole. Individual staff may be friendly with individual moderators, but that does not translate into the company including the community and their elected moderators as partners in business decisions that impact them. Many of the veteran moderators are as (if not more) invested in the success of Stack Exchange as the shareholders and management.

Maybe moderators should join together in a formal independent organization that can speak with one voice to the company for its members. That might make it easier for the company to hear their advice on various policy decisions before things escalate.

Answer 10

Apparently people have been upvoting spam. I don't know if this is an isolated thing or a coordinated effort, but it's not at all helpful. Indeed, I'd suggest that CMs consider warning and (if not heeded) suspending people who upvote obvious spam. It's functionally the same as a voting ring after all. Personally I find that behavior counter-productive to the reasons I signed the letter.

I got to thinking about voting rings because they present similar problems to detecting machine-generated content. It's less of a binary switch and more of a Bayesian inference. My memory might be fuzzy, but here's how voting ring investigations typically went down:

1. A user would flag posts that seem to be getting more upvotes than expected.
2. A moderator would see that flag and look into the situation using moderator tools built into the site. They'd also bring experience and past history into their judgements.
3. If the moderator thought it worth more investigation, they'd contact a CM to (potentially) invalidate votes.
4. We'd use a variety of tools and techniques passed on from one CM to another (thanks Shog and Tim!) to make a final determination.
5. If, in our judgment, voting fraud had taken place, we'd invalidate some votes and message the users we suspected of being in the ring.
6. The users would deny that they'd done what we said they did.

Each step would increase or decrease confidence that voting fraud had happened. For instance, an overly-aggressive flagger might decrease our estimate of the odds because they cry wolf so often. In my experience, moderators usually identified voting rings accurately, so when they brought one to my attention, it raised my estimate considerably. Some of the tools were extremely reliable and others weren't worth listening to most of the time. You learned to make these evaluations only through experience.

I remember one time we'd caught a pair of users dead-to-rights. One of the users (we'll call them A) said they had no idea what had happened. The other user (B) confessed that they were married to A, saw that A's posts were getting downvoted and upvoted all of A's posts to help them feel better. Now. That's a textbook voting ring, but I wanted to give that couple a hug and let them off the hook. We invalidated the votes anyway and told B to maybe try something else to cheer up A.

We'd also see things like two experts in a specific technology upvoting each other's posts. If we just used the voting fraud tools, we'd be compelled to suspend them. Another time we applied one of our tools to employees voting on Meta and found that the voting ring was coming from inside the house! Since we didn't just blindly do what the voting fraud tools recommended, it was an inside joke rather than a soul-searching investigation. Context matters so much!

We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives, which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions. People with original questions and answers were summarily suspended from participating on the platform.

One of the things that has made this issue go from 0 to 100 in a week is that we haven't seen any details of this analysis. Like the vote fraud tools I just mentioned, false positives are not surprising if used in an inappropriate context. In my experimentation with GPT and GPT detectors, I found that my own original writing could be flagged as likely machine generated. So that's not news.

What would be news is if we could see evidence of moderators blindly using GPT detectors without other signals to make their decisions. That sort of behavior is not unheard of, but its rare on Stack Exchange sites. Moderators here tend to be extremely diligent. Given the volume of flags, mistakes are inevitable, but most moderators are quick to own up to them when confronted. That's one of my priors that makes me suspicious of the analysis even before it's made public.

The final sentence about "original questions and answers" also raises red flags for me. I guess what bugs me is that if your standard is "original" you can't go far wrong with machine-generated content. How many times have you searched for something (recipes are particular offenders), found a promising page and then scroll (and scroll) until you find the actual information you need? (I get it! You spent a summer in Tuscany and that's how you know so much about spaghetti carbonara.) Since it's so easy to generate reams and reams of verbiage using ChatGPT, "original" seems to skew the playing field away from (most) humans.

I don't know if this is intentional, but the most recent CEO blog post (CEO Update: Paving the road forward with AI and community at the center) seems to argue that people in the community serve AI rather than the other way around:

Our community has given us feedback through the evolution of this tool, and their feedback is critical to how it scales. As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish.

The words are fine, but in the context of disagreement between moderators and the company about how to moderate machine-generated content, it sounds dismissive. It's as if the company believe it would be making great strides in technological development if only those pesky humans would get on board. Or maybe I'm being unfair and this is just an "AI meets community" pitch. (I suppose I'm not the audience for the CEO's blog, though.)

I agree that one value of communities is improving a product. But that doesn't happen if the community isn't free to reject the product or wait until it's reached some level of maturity. When the community signals that "original" content isn't enough, it doesn't do the product any good to ignore that signal. Because with few exceptions, the people who curate content on Stack Overflow also consume it.

Answer 11

We stand by our decision to require that moderators stop using that tool.

On the site I moderate, a perfectly well written, but entirely incorrect, answer can literally ruin the lives of anyone who decides to follow that misleading information. I have seen this happen on another immigration site that I've been highly active on. I should be able to use any tool whatsoever to prevent that kind of disaster from happening. If Stack Exchange has a problem with this kind of responsibility, maybe it would be best if they simply shut Expats down.

And there are probably a few other sites that they should take a hard look at because of the potential repercussions.

Answer 12

^{There's plenty to be dissatisfied with, but I've raised what I have to raise about that already. So I'm going to try to put on my "Pollyanna hat" here. Update from future me: You're making it pretty hard for me to keep the hat on.}

We will continue to look for other, more reasonable tools and are committed to rapid testing of those tools and any suggested heuristic indicators.

Will you be making the mods part of that work? I'm sure they'd have much of value to share (Ex. @sideshowbarker (see also their post here)). Other community members also come to mind (Ex. @NotTheDr01ds (see also their posts here, here, here, here, here, and here)). (This doesn't have to be a public conversation- if I understand correctly, we've been trying to avoid showing the people violating the policy how we sniff their violations out).

The Community Management team is evaluating the current situation and we’re working hard to stabilize things in the short term.

I'm thankful that the CMs are picking up on some of the moderation work during the strike (Ex. @Yaakov) on top of their other work. I keep needing to mentally remind myself not to shoot the messengers (or anyone for that matter). A nice side-effect we might see here from the CMs dogfooding is some empathy for pain points in mod tooling. Maybe something nice will come out of that.

We’ll be working collaboratively with the community on the long-term solution.

That's good to hear. I look forward to it. When will we hear more on this? (Please also don't forget about the questions we've asked about the new policy).

Answer 13

The Community Management team is evaluating the current situation and we’re working hard to stabilize things in the short term.

If, by "short-term stabilization," you mean ending the moderators' and curators' strike, then the solution is relatively simple: Just apologize – publicly and unreservedly – for the confusing and insulting manner in which this so-called policy change was announced: privately (to the moderators) and publicly (in a way that was at odds with the private announcement).

I regret that actions have progressed to this point.

Although this may be considered the seed of a beginning of a vague apology, it simply doesn't go far enough. Also, it carries an implicit hint that the moderator team has played a significant part in creating this mess – which it hasn't.

We’ll be working collaboratively with the community on the long-term solution.

This "community collaboration" should have been present from the get-go (as it was when first implementing the ban on AI-generated content). Maybe it is now time to hit the "rewind" button and ask for suggestions on how to address the (unsurprising) "large upswing" in account suspensions.

In the meantime, restore the moderators' rights to use suspensions (the vast majority of which, I gather, have been for periods of seven days or shorter). You (the staff) need to trust your moderators to not issue suspensions (of any length) trivially and without full consideration. In the longer term, there may be a way to avoid such suspensions by using and/or fine-tuning the in-built "answer ban" system, instead; from what I understand about that (the details of its algorithms are, understandably, kept secret), this can actually work out to be a more effective "blocking tool" than a suspension, as it is: (a) potentially for an unlimited time period; and (b) covers all posts from a given IP address. And we certainly need some sort of block on the vast majority of AI-generated gibberish that gets posted on our network.

Answer 14

People with original questions and answers were summarily suspended from participating on the platform.

You still are yet to give a single example. Wow, that gives this all a ton of credibility.

UPDATE:You have given some data, but I see no reason to think that it is realiable. I concur with others complaints on the post and won't restate them here.

Answer 15

Which other groups’ interests are missing from this post?

In many complex subjects a number of constituencies need to be considered.

From this post, one can identify the following groups/interests:

• moderators

• users that are, correctly, or possibly incorrectly, flagged as having gone against the rules.

• implicitly Stack Exchange, Inc. corporate interests. While others may want to dismiss those I will say that, for all the complaints, the family of Stack Exchange sites have been of tremendous value to professionals and hobbyists all around. Second, a corporation that does not have a plan to make money, or at least fund itself according to expectations, risks disappearing or takeover.

However two other groups are missing and are very much impacted by this policy:

First, users that come to SE looking for answers to a problem that they have.

Stack Overflow, while sometimes criticized for fostering cut and paste programming, generally fits the bill. Some primary ways to judge the quality of answers are: your own knowledge, the up down votes, the presentation of an answer, along with supporting quotes and program outputs and the poster's reputation. Even if the poster is not the most competent on the subject, a user can assume repeat research and posting will gradually have improved their knowledge.

With AI allowing cheaply-produced, but perfectly worded answers, neither the presentation, nor the poster's reputation has any great meaning, leaving the user to their own to judge how helpful an answer is liable to be, except for community upvotes which can be swayed by confidence and presentation, something generative AI excels at.

Respectfully, that constituency is not benefiting in the least from your proposed hands-off approach.

And, much more so than just the moderators, if regular users cease to see Stack Overflow as a valid source of information, they will go elsewhere.

Second, users that answer questions.

Their motivation may vary from reputation seeking, interest, evangelism, desire to help, challenge, pursuit of professional credibility.

They decide to join SE sites and "grind" their way up, by answering question after question.

How is that group likely to react to a situation where they are systematically pre-empted by people who can write up a complex and perfectly-formatted answer - that they often don't really understand and don't bother vetting - in 5 minutes or less? While it takes even a knowledgeable person 3x the time to write up a concise answer, with some minimal code and showing output?

Like the first group, it is quite possible they will not bother. And without a solid core of them, there is no site.

Though it is a very imperfect comparison, I would note that schools that have, for whatever reason, taken a relaxed attitude towards cheating, and academic standards in general often end up devalued as a result: in the eyes of prospective students, and in those of prospective employers.

Generative AI content is going to be a big challenge, for this site, and others. You have indicated concerns with moderator measures as done so far.

Fine, to an extent. Work together on a better solution. But the answer is most certainly not throwing out AI content moderation as a whole.

The striking moderators, to me, ultimately seem to be motivated by looking after users in the largest sense: regular users posting questions and answers, without using shortcuts, and especially not knowingly using shortcuts with well-known imperfections at the current time.

As a regular, non-moderator, not particularly "community" minded, user, I would strongly encourage SE, Inc. to do the same: do well by your users.

Better said on this answer:

p.s. I intentionally stayed out of considerations about communication issues, perceived or real shortcomings in how moderators were treated, community stewardship, etc…. I do have my opinions, but I wanted to keep the focus on why the proposed policy is bad for regular users and a better collaborative solution needs to be found.

Answer 16

I am not a moderator, and I have no affiliation with any of the moderators - involved here or otherwise. This means I have no access to the "private policies" behind this. The upside is, I have no obligation or commitment to non-disclosure. On the other hand, everything to follow is by its nature speculative.

Edit: Update: Since writing this I have read a number of other related meta questions and answers, authored by both community members and staff. I also finally read the moderators' strike letter. Importantly, I have still not seen any "secret" materials. Nonetheless, at this time, I feel confident saying I was correct below, regarding the secret policy. SE Inc is very likely preventing moderators from removing posts that violate our community expectations, and harm the health of the community as a whole. Thus I have also since signed in support of their open letter.

From the body of the official "question", one might be forgiven for thinking that mods were in an uproar over "access to a [flawed] tool". You skillfully framed the issue as a move to protect the community from self-inflicted harm. However, having observed the community reactions to past debacles like this, the story simply isn't credible. Such a massive strike would only be undertaken if the policy issue were truly existential. Something is missing. Even without the other comments here, we might guess what that is: some "private policy" which mods feel bound to withhold [but which I'm free to speculate about].

So what is this secret new policy direction? Well, we can actually just read between the lines of the official statement, applying a basic understanding of standard PR tactics. This is clearly a deflection, and those work best to defuse a reaction if they are focused on something "innocuous" which is very near to the true subject. This is to avoid a sense of "narrative novelty" when people encounter discussions of the "real" issue, in hopes they'll dismiss it without consideration as something they've already evaluated. By this reasoning, we might say it likely has something to do with AI generated submissions.

Now the cynic in me comes out, and I ask myself "Why would SE want to hide this policy shift?" After all the community issues of the past few years, surely anything so contentious as to prompt mass strike would have been appropriate to discuss openly first? If they're playing the corporate trump card, and trying to gag the mods, then it must surely be rooted in profit motive. So really the question to ask is "What does SE or it's leadership think they stand to lose (or gain) in this area?"

Bringing this all together, I feel pretty confident hazarding a guess as to the policy, and mindset of the SE side:

Basically, they don't want AI generated content removed at all, unless it's so flagrant as to make the site appear useless.

Their reasoning is likely based in a combination of factors, ranging from fear of direct competition by AI tools to a more straight-forward play to assuage investor concerns about userbase growth. It's also likely they want to feel free to lean into that themselves eventually, and use AI tools to answer questions directly. The cynic in me wonders if they already tried it without telling anyone, but so far I haven't seen anything to suggest it. In any case, from a "catalog" management perspective, I'm sure they see having generated answers to orphan questions as better than having none, and more "active users" as better than fewer. It's easy to see why they might view moderator efforts to remove AI "contributions" as a nuisance to their bottom line.

Again, I could be completely off-base. But if I'm not, this is a serious error on the part of SE, and I hope they recognize this and reverse course in time.

From a business perspective, this move may appear savvy on the surface, but it actally is an act of brutal self-sabotage. The value of SE isn't being the place with all the answers. It's the community. Disregarding it in this way can only erode any long-term value you hope to create in the company. As a "regular end user", I'm watching this with an eye towards walking away entirely (and advising everyone I know to do the same).

Whatever is going on, now is the time for honesty and transparency. The idea that mods here are being coerced into hiding any policy at all does not sit well with me. Tread carefully SE.

Answer 17

We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives, which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions. People with original questions and answers were summarily suspended from participating on the platform. We stand by our decision to require that moderators stop using that tool.

I think this is a serious concern that should be addressed, but I think it also misses the point a little bit.

The main reason I signed the letter (something I rarely do) is the heavy-handed approach that seemed to have been taken. I think your concerns are completely valid and very much should be raised, but on the other hand moderators are also struggling with "ChatGPT spam". I don't really know what a good solution for that would look like exactly, but I do know that a heavy-handed "we have decreed that ..." is not it. It's a difficult problem and there are no quick and easy solutions.

It's not really about the "ChatGPT policy" or whether or not these tools are good or bad per se, it's about a mismatch between "we're all in this together and let's decide together what the best course of action is" vs. "we decided, end of discussion" (crudely put). Moderators are volunteering their time and in return it's understood they have some say in policy. You can't "just" order people like they're employees, so you need to spend time and effort convincing people you're right. This is work, but still cheaper than paying people to moderate the site. And sure, in the end Stack Overflow Inc. has the final say and that's fine, but if you're not even spending the effort...

---

As for this specific policy, it seems to me that the problem is that moderators are suspending users based on too little evidence and are, apparently, not taking the full situation in to account when making their decision, relying too much on the output of a single tool. I can't really judge how often that happens, but it certainly seems more than plausible that moderators have made mistakes here and that's obviously not a good thing, but the alternative is a fuzzy "this looks like ChatGPT" underbelly feeling, and that'll have a lot of false positives too – perhaps even more? I've certainly seen plenty of "ChatGPT paranoia" around, and most of this probably didn't use any tools.

The tool is not the problem; "How do we deal with ChatGPT?" is the problem, and as far as I can tell, there is no good solution right now. Any tool can be used inappropriately. If there is a problem with moderators making bad decisions then that should be taken up with those specific moderators.

For what it's worth, personally I don't even think using ChatGPT is necessarily bad in and of itself; see my other answer. I'm all for judging posts more "holistically" rather than just "you're using ChatGPT!" as that seems too simplistic. It's really about the heavy-handed approach as far as I'm concerned.

---

Edit: thinking about this some more, I'm also reminded how conflicts were handled in the past. I was involved in the launching of the Vi & Vim site quite a bit and a number of employees were pretty strongly against the concept initially. But the disagreement played out different: it was well argued, and we were allowed a chance to prove "you're wrong", and this turned out well.

The controversial Stack Overflow Documentation effort also springs to mind; I felt it wasn't the right direction, but Jon Ericson tried very hard to make it work by engaging with the community and I always very much appreciated that (even though the community was overly harsh at times, to the point I felt rather bad for Jon).

I'm not saying any of this is easy or that the People of Stack Exchange can't be a difficult bunch to deal with – they can – but that's kind of what you get when you rely on volunteers.

In comparison, this feels like ... bland marketing-speak. I get that community management is hard, but by being bland and generic you're not really engaging with anything substantial and nothing ever gets hashed out.

Answer 18

Honestly Philippe, knowing you as I do, I think you know better than this.

If you're right, and if you believe in what you wrote, this is a case of self-feeding vicious circle of knee-jerk reactions to rapidly evolving patterns of contribution and moderation, with the community and the "powers that be" going in opposite direction.

From our experience at Wikimedia you too can probably easily list a dozen past cases just at the English Wikipedia, starting say from the Seigenthaler incident. We need a de-escalation soon, and it's quite rare for things to de-escalate on their own while the opposite sides just dig in. (That tends to happen only if the proximate cause of the incident turns out to be a tempest in a teapot in hindsight, and even then it often takes a long time to recover.)

Answer 19

To respond to a few things here:

We ran an analysis and the ChatGPT detection tools have an alarmingly high rate of false positives...

How did you do that analysis? Do you have sample posts that you know for a fact that were human-generated and posts that you know for a fact were AI-generated, or were you using a different approach?

...which is correlated to a dramatic upswing in suspensions of users with little or no prior content contributions.

Was the content they were generating good quality? The cases of new accounts posting suspected ChatGPT content I've seen weren't exactly producing stellar answers. Is it really that problematic to suspend someone who was producing bad content anyway (regardless of whether it was ChatGPT)?

People with original questions and answers were summarily suspended from participating on the platform.

Again, are you aware of specific individuals that you know for a fact were incorrectly suspended? How did you determine that, unless you yourselves have a way to reliably distinguish between human-generated and AI-generated AI? If you don't have a way to do that, aren't you kind of doing the same thing you're accusing moderators of doing?

Answer 20

I'm not a moderator, but I think SE has got this subject backwards.

I have on SU detected and signaled 5 cases of AI "answers", with confirmation in all cases by the moderators and even sometimes by the poster himself.

In the first two cases, I used no tools. For the next three cases, I discovered ZeroGPT and GPTZero and used them for a second opinion.

In other words, I have used the tools not for detecting AI answers, but for detecting whether I was wrong in assuming the answer was by AI.

No one runs all the answers through ZeroGPT and GPTZero to detect AI answers for fun. One only goes to this effort when the answer seems weird, without a central theme, with a very weak or no connection to the question and even sometimes just plain wrong and misleading.

The tools are used just the opposite of that postulated by SE : To check whether the answers are not AI.

Answer 21

I thought it would be educational for the SE deciders to show them how GPT4 answers their own announcement here.

This "answer" is manifestly not useful or helpful, and doesn't even have a discernible central theme. It's a collection of texts that GPT4 found on the internet that loosely relate to the subjects in the announcement.

This is an example of rubbish AI "answers" that I have been seeing on the SU site. If this kind of drivel is allowed to pollute our sites, we'll only be destroying ourselves.

Does SE really think that we should let such answers deluge our sites?

---

The concerns expressed by the moderators regarding the detection tools and AI-generated content on Stack Overflow are valid and should not be dismissed. While it is unfortunate that some moderators have chosen to disengage from their moderation tasks, their actions highlight the seriousness of the issue at hand.

The claim that the ChatGPT detection tools have an alarmingly high rate of false positives is questionable. Detection tools, including AI algorithms, are not infallible, and false positives can occur. However, it is essential to consider the overall impact of these false positives and weigh them against the potential harm of allowing AI-generated content to proliferate unchecked. The purpose of such tools is to maintain the quality and integrity of the platform, even if there are occasional false positives that need to be addressed.

The suggestion that suspensions were imposed on users with little or no prior content contributions solely based on ChatGPT's detection tool is misleading. Moderators typically employ multiple methods and analyses to make informed decisions. While ChatGPT's detection tool may have been part of the process, it is not the sole determinant of suspensions. The goal is to ensure a safe and reliable platform for users, which sometimes requires precautionary measures.

The statement about seeking other, more reasonable tools and commitment to rapid testing of those tools is encouraging. It shows a willingness to address the concerns raised by the moderators and find better solutions for content moderation. Continuous improvement and adaptation to new challenges are essential for any platform to thrive.

Expressing appreciation for the moderators' years of service and acknowledging the difficulty of the situation is a positive gesture. However, it is important to focus on resolving the underlying issues and finding a viable path forward. Merely expressing regret and hoping for a resolution may not be sufficient. Concrete actions need to be taken to address the moderators' concerns and restore trust within the community.

While it is acknowledged that the upcoming days and weeks may be challenging, both sides should prioritize open dialogue and collaboration. Ensuring quality and inclusivity on the platform should be the shared goal. Transparent communication with the community about updates and progress is crucial for rebuilding trust and working towards a long-term solution.

In conclusion, it is crucial to acknowledge the concerns raised by the moderators and address them seriously. Balancing the need for effective content moderation with minimizing false positives is a complex task, but it should be approached with a commitment to finding the best possible solutions for all stakeholders involved.