In How can we determine whether an answer used ChatGPT? (SO specific; it might be different for other sites), moderators initially didn't disclose how they detect AI generated content, probably in order not to make it too easy for those wanting to circumvent the AI content ban. Later they divulged that they do not only rely on tools and that indeed they mostly train themselves on ChatGPT content. But in the implementation phase they didn't have complete information about the ground truth, i.e. they wouldn't really know for sure if deleted content really was AI generated. Indeed the company seems to strongly doubt that, given their alleged ban on banning at the end of May the exact wording of which might be published soon.
If only there was a direct way to solve this controversy and get to know how accurately moderators can really detect AI content.
Oh wait. I think there is. Take some random questions from before ChatGPT, take a random answer from them (negatives) and add an equal amount of ChatGPT-generated answers to these questions (positives), then let volunteering moderators classify them.
The true false positive rate for bans would still depend on the true rate of AI generated posts (which is unknown) and on the exact way how moderators focus their attention on answers and some context would be missing, but it would end the discussion about the accuracy of detection tools and it might give the company an acceptable off-ramp (if only they knew how accurate moderators can detect AI content).
It's some work to set up and collect the data, but only a fraction of the work that was done during implementation of the ban and I think it will help being more safe about any statements like "we can accurately detect AI content". It would probably be worth it.
It's also reported that it was offered by moderators multiple times to the company, but such a test doesn't depend on the company.