14

I've flagged some ChatGPT-generated posts (in June) on Stack Overflow. Recently they've been reviewed, and most of my flags were helpful. Then the posts I flagged were deleted.

Are moderators now allowed to delete ChatGPT-generated posts, if they 100% are ChatGPT-generated?

It's not a duplicate of (Interim) Policy on AI-content detection reports, because the proposed duplicate is too broad.

Improve this question
19
  • 4
    Moderators were able to delete ChatGPT-generated posts before the rule change in late May. Are you asking whether they're able to again, or were you not aware that they were able to delete ChatGPT posts before May 29th-ish? Also, 100% ChatGPT-generated is sort of a high bar. As obvious as most of the posts are, there's always a small sliver of doubt (not that that should prevent taking appropriate action, but just sayin'). What I'm curious about is whether suspensions for ChatGPT spam are being issued. If they are, they seem way slower than prior to the strike.
    – ggorlen
    Commented Aug 30, 2023 at 5:40
  • 2
    @ggorlen I'm asking "whether they're able to again". I'm aware that they could delete ChatGPT posts earlier in this year and even ban users that post ChatGPT answers.
    – Petəíŕd the Wizard
    Commented Aug 30, 2023 at 7:09
  • I've seen a few of my GPT-flagged posts deleted since the strike ended, so I assume "yes", albeit at a lower and slower rate than before. Whether due to volume overload, less mod power or both, or just random luck and small sample, I don't know.
    – ggorlen
    Commented Aug 30, 2023 at 15:13
  • 4
    There are currently 6,160 flagged posts on SO. Of those, 4,534 are "in need of moderator intervention" flags, which tend to be the most labor intensive (anywhere from seconds to days per flagged post), with the possible exception of plagiarism flags, which tend to take a huge amount of time. It's going to take quite a while (think months) for the moderators to work through those flags.
    – Makyen
    Commented Aug 30, 2023 at 15:48
  • 3
    Does this answer your question? (Interim) Policy on AI-content detection reports (not directly, but it's implied by / a subset of the talk about "moderator actions")
    – starball
    Commented Aug 30, 2023 at 16:53
  • @starball why propose a duplicate that isn't exact?
    – Petəíŕd the Wizard
    Commented Aug 30, 2023 at 17:26
  • Do duplicate closures work differently on meta sites?. I'm not sure if I'm doing the right thing, but I tried to use my judgement. If I'm in the wrong, sorry. Also, a duplicate covering an issue in more general terms is in certain cases a good thing. Generalization can prevent a bunch of scattered Q&A about more localized instances of a single thing.
    – starball
    Commented Aug 30, 2023 at 17:28
  • Why downvotes? ​
    – Petəíŕd the Wizard
    Commented Aug 31, 2023 at 7:01
  • Meaning of down-votes on meta Vs main sites.. I'm guessing that this has already been covered and people want to avoid thinking about the AI future as it's terrifying.
    – W.O.
    Commented Aug 31, 2023 at 15:11
  • moderators have always been able to delete posts, chatgpt generated or not
    – user13267
    Commented Sep 1, 2023 at 8:23
  • @user13267 moderators able to delete any post. But are they allowed to? Of course moderator can delete, for example, this post. But this will count as a diamond-power abuse.
    – Petəíŕd the Wizard
    Commented Sep 1, 2023 at 8:25
  • I don't get why they wouldn't be allowed to, even if that post happened to be generated by chatgpt. Being generated by chatgpt shouldn't (doesn't) need to have any relationship to why it should be deletable or not. And as far as I know there is no system in place at the site that makes a post undeletable if it happened to be chatgpt generated
    – user13267
    Commented Sep 1, 2023 at 8:29
  • @user13267 you probably missed the whole drama with the strike, and disallowing to delete ChatGPT posts.
    – Petəíŕd the Wizard
    Commented Sep 1, 2023 at 8:36
  • no I didn't miss it you seem to have misunderstood it. Some random spam/low quality content that got posted, was always deletable (for being random spam/low quality content), regardless of whether anyone used chatgpt to generate it or not. If you thought it was not allowed to delete them, in case they turned out to have been generated using chatgpt, then you must have been mislead by some of the same dramamongers
    – user13267
    Commented Sep 1, 2023 at 9:17
  • 1
    @user13267 By "ChatGPT-generated" I mean exactly the posts that are not nonsense, not spam, and is a right answer, but generated by ChatGPT.
    – Petəíŕd the Wizard
    Commented Sep 1, 2023 at 9:32

3 Answers 3

Reset to default
21

One often finds that ChatGPT posts are... often nonsense, or of very low quality.

"100% ChatGPT generated" is difficult. We're allowed to delete for low quality, some degree of nonsensicality and if some other heuristics are matched take action on the account as a whole.

We're allowed to delete ChatGPT generated posts based on reasonable evidence as opposed to extraordinary certainty.

Improve this answer
16

Yes, moderators are allowed to take action on AI-generated or assisted posts within certain guidelines.

However, the backlog of flags from during the strike was significant, especially on SO, and many AI posts weren't even flagged during that time since we had no idea exactly what policies would come out of the negotiations (or when).

I have 266 pending flags on SO at this point, most (perhaps all) of them post-strike and AI, so I know things are a bit slow. In addition to the extensive backlog, I suspect that reviewing AI flags is now a bit more time consuming for moderators given the updated rules.

As always, users can help by explicitly listing all of the reasons in your flag why you believe a post may be AI/LLM/GPT. Flags which simply say, "ChatGPT" are likely to take mods much longer to handle.

Improve this answer
7
  • 2
    This is good to know, but it's the first I've heard that the flag text for a GPT answer matters. I've been just saying "Looks ChatGPT" or "ChatGPT spammer" if they've posted more than a few in a row. It's a bit time consuming to explain why in detail, given the volume of flags I'm raising, especially in quick succession for the same user. Are there guidelines on how to best raise a GPT flag in a way that's best for the mods, other than not to comment on the post?
    – ggorlen
    Commented Aug 30, 2023 at 15:08
  • 6
    @ggorlen That's a good point that it hasn't been "well advertised". The best discussion around that topic I can find is here. Laurel makes the point in an answer that, "These details are very important because they can considerably shorten the amount of time a moderator spends on [investigation]." Just keep in mind that while you and I need to spend additional time flagging the answers with details, if we don't do it, the Mods have to spend that additional time, and they are dealing with hundreds (currently thousands) of flags.
    – NotTheDr01ds
    Commented Aug 30, 2023 at 15:19
  • 6
    @ggorlen As to "what" to put it in the flag, any reasons why you believe it is AI. If it just "looks" AI to you, that's fair - Let the Mod know. If you see technical errors, things that you believe a real person wouldn't say (but an LLM might), etc. then add that to the flag. Even a note that there are multiple answers from that user that appear to be AI-generated can be helpful.
    – NotTheDr01ds
    Commented Aug 30, 2023 at 15:21
  • 5
    Personally, mine look something like: Suspected AI: Possibly the 12 most recent from this user (1) a reason, (2) another reason, (3) another reason -- I've become pretty adept at identifying the reasons why something is likely AI, so I often have multiple data points to cite. Multiple mods have told me that they appreciate the format of my flags. And yes, I number and bold each so that they are easy for the Mods to read as quickly as possible.
    – NotTheDr01ds
    Commented Aug 30, 2023 at 15:24
  • 1
    @ggorlen Another tip, if you are flagging a lot of them - Consider using a separate app (or sidebar) for writing up the flag, then copying/pasting into the flag. That allows you to keep the browser tab open on the answer itself while you type up the details. I personally use Vivaldi browser's Note sidebar for this, but there are many other options, of course.
    – NotTheDr01ds
    Commented Aug 30, 2023 at 15:36
  • 1
    "Even a note that there are multiple answers from that user that appear to be AI-generated can be helpful" So this is permitted? I've seen people (on math.se) say that you should only refer to the content of the particular answer, and mentioning patterns in the user's answers is a no-no. The people saying this weren't moderators, but they did include users of long experience and high reputation. And I never saw any official statements with either "it's okay" or "it's not okay" to clear things up.
    – JonathanZ
    Commented Aug 30, 2023 at 15:52
  • 3
    @JonathanZ I've asked about this as well, and the general Mod guidance has been that (a) this type of statement still focuses on the content, (b) flags are private, while certain chat rooms are public and do have rules about mentioning users, and (c) a single flag can sometimes suffice to cover multiple answers in this way, saving you and the Mod time. If in doubt, though, consult a Mod on the site where you are flagging. If you don't get a response, I'd say go ahead - At worst they'll then let you know via flag comment or other means. :-)
    – NotTheDr01ds
    Commented Aug 30, 2023 at 16:30
5

The new interim policy concerns what the Community Management team will accept for reports and appeals. For details, see (Interim) Policy on AI-content detection reports.

Moderators were never restrained from deleting posts that blatantly are very low quality (VLQ), other than voluntarily restrained by themselves as part of the moderation strike. When a moderator found a VLQ, regarding the deletion of the post, it doesn't matter if they were written with or without the help of generative AI tools. Due to the new policy, what might be changed are the heuristics used to determine if a post content will be treated as being created by generative AI tools to apply / request additional penalties to the post creator.

Without seeing the specific cases, we can't infer the deletion cause of each of them; we can only deduce what might have happened based on the personal experience of each of us and the literature reviewed.

I'm a member of Stack Overflow with the privilege to vote to delete, and a community moderator on Web Applications SE. In both communities, I have seen VLQ posts where GPT detectors reported a very high probability of being GPT generated that are grammatically correct but show AI hallucinations, as they include Google Apps Script / JavaScript methods, among others, that don't exist and point to documentation that doesn't exist either, the included hyperlinks return error 404, page not found. On Web Applications SE, as moderator, I will delete these posts. On Stack Overflow, if the post already has enough downvotes, I will vote to delete it.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .