A recent statement from the SE company to the press suggests that company spokespeople (the CEO or whoever writes his statements) believe that moderators have been relying on ChatGPT detection tools to find and suspend people who post plagiarised AI-generated text:
Stack Overflow ran an analysis and the ChatGPT detection tools that moderators were previously using have an alarmingly high rate of false positives. [...] We stand by our decision to require that moderators stop using the tools previously used. We will continue to look for alternatives and are committed to rapid testing of those tools.
The purpose of this meta post is to make, very clearly and distinctly, a point that has been made before (in the middle of longer meta posts) but perhaps not noticed by SE upper management. Making it the title of a meta post will make it much harder to ignore or miss.
Moderators do not (and never did) rely blindly on automatic GPT detector tools.
The decision to delete content or suspend users is always taken seriously, and experienced SE moderators are not foolish enough to rely blindly on ... any automated tool, really. Just like we don't automatically suspend everyone who shows up in "suspicious votes" tools, we don't automatically delete every post that automatic GPT detector tools claim is AI-generated, nor suspend every such poster. We use a variety of heuristic methods to judge such cases, together with good old human judgement. Wouldn't it be ironic if we used an automated process to ban people who used an automated process to create their posts? We don't, and we never did.
Moderators have been secretive about the exact methods used to detect AI-generated posts. Yes, but that's the same with any type of site rule-breaking. We're also secretive about the exact methods used to detect voting fraud. If we announced publicly how we detect any kind of plausibly-deniable misbehaviour, people who wanted to misbehave would simply adjust their technique to continue misbehaving while avoiding our methods. That's why we haven't announced the exact process we use to figure out whether someone is posting GPT copy-pastes. But the process is not, and never has been, "run it through a GPT text detector and then immediately reach for the ban hammer". If the company thinks it is, that's a major misunderstanding which probably contributed to the disastrous recent announcement. Perhaps this PSA can help to clear up the misunderstanding? As a relatively short post making a single clear statement, it could easily be conveyed by CMs to upper management (hint, hint).
Don't believe me? Check it out: Stack Overflow moderator sideshowbarker has handled ~10,000 ChatGPT-flagged posts but never used any detection tools to do so.
