22

As a non-native English speaker, I am often a little bit unsure about grammar, wording, or phrasing of my posts. Ignoring the fact that this might be hard to detect by others: if I ask an LLM like ChatGPT to improve my post in a linguistic-only fashion, without changing the meaning, does the result then count as "AI generated content"?

Of course, it should be clear that I would proofread the result before I post it, to do my best that the original meaning really wasn't changed.

To give you an example: I ran the upper text through ChatGPT and got the following result:

As a non-native English speaker, I frequently feel uncertain about the grammar, wording, or phrasing of my posts. Disregarding the fact that these uncertainties may be challenging for others to discern: If I seek assistance from an LLM like ChatGPT to enhance my post solely in terms of language, without altering the meaning, would the outcome still be considered "AI-generated content"?

Certainly, it should be understood that I would thoroughly proofread the result before posting to ensure that the original meaning remains unchanged to the best of my ability.

Since AI generated content is forbidden on some SE sites, or, where allowed, has to be attributed as such, I think we should have some consensus whether such improvements count as AI generated content or not. Even if we get consensus that attributition is required, the question remains whether a site which forbids AI generated content in general may or should allow "AI improved content" under the constraints I scetched above.

--

You may be interested in my final assessment to this Q&A below.

26
  • 5
  • 6
    Let me put it like that. There is AI tool that let you add things to existing picture, e.g. "add sunglasses". See my current avatar. See the original. The AI not only added sunglasses, it also changed other things, sometimes in subtle way that's hard to detect. Same goes with what you want to do: the AI will change the meaning. Not "maybe", not "possibly". It's 100% certain that the AI will hallucinate and change your text. You might be able to detect it, or not. But there's only way to fix it: don't do that. Jan 10 at 7:14
  • 7
    @ShadowWizardLoveZelda: "It's 100% certain that the AI will hallucinate and change your text. ", well, I agree it is 100% certain that the AI will change one's text, but it is not 100% certain it will "hallucinate" (=invent something which goes contrary to the original meaning). Sure, there is a certain risk that I overlook some change of the meaning - but there is also a certain risk with every post I write that my bad English will be misunderstood by a reader, which I can see from the reaction I get to a post. IMHO that would not be really different by some AI-improved post.
    – Doc Brown
    Jan 10 at 10:35
  • 3
    If it looks like a duck and quacks like a duck, it will be treated just the same as a duck would. Jan 10 at 10:44
  • 11
    Folks are generally happy to fix language issues. I'd rather people are responsive, and work with the rest of the community to fix their posts than on a somewhat unreliable, and honestly disliked tool. Jan 10 at 10:44
  • 10
    There is absolutely no reason why anyone would have to use AI to improve their posts here. Jan 10 at 11:04
  • 2
  • 1
    @ResistanceIsFutile: *"Do you seriously think that your English needs improvement," - sometimes I do.
    – Doc Brown
    Jan 10 at 12:28
  • 1
    It looks like you managed just fine without AI all these years. Jan 10 at 12:47
  • 1
    @DocBrown about hallucinate, I mean that it will randomly drop words or just use a different word based on what it "know". For example "My cat has for lags, one year, and one teil" can be "fixed" by the AI to be "Cat has four legs, two ears, and one tail". It's hard to detect because it's correct for most cats, but the person saying this meant to actually say "one ear", and the AI "fixed" it, not only corrected grammar. And I'm sure that's exactly how AI will work, hence my previous comment. So when you go and post such "corrected" question, people will wonder "OK, so what's wrong?". Jan 10 at 13:56
  • 9
    The core issue i have with this approach is that, while you're asking the LLM to do X, all it's really doing is what it always does: output text that seems to match the prompt. there's no guarantee it will actually adhere to what you're asking of it because that's not what it's designed to do. It's just an LLM.
    – Kevin B
    Jan 10 at 16:02
  • 5
    @DocBrown One thing to keep in mind is that using a stock model will make it seem like your text was created entirely by AI instead of just rephrased. You can create a LoRa of your own writing that can be layered on top of a base model that will help an LLM preserve your voice while still suggesting phrasing based on the base model. As with any tool, the more skill you have with it, the better the results. I believe genAI is a powerful tool that will positively impact many areas, but I would not recommend relying on a generic LLM to suggest changes to your writing for SE posts.
    – ColleenV
    Jan 10 at 16:55
  • 5
    @Resistance Re; "It looks like you managed just fine without AI all these years" – being able to adequately do something manually before a new tool exists is not a very compelling reason (in of itself) to avoid using that new tool if it ends up being genuinely helpful.
    – zcoop98
    Jan 10 at 17:38
  • 1
    I personally just can't relate. I like my posts to be in my voice- however imperceptible that may be sometimes.
    – starball
    Jan 10 at 20:53
  • 2
    @ResistanceIsFutile: IMHO these tools not as bad as you pretend, but I agree that one has to be careful how to use them.
    – Doc Brown
    Jan 11 at 12:27

8 Answers 8

26

TL;DR: Maybe, but it's probably forbidden. In principle, it's not impossible to use ChatGPT in an allowed way, but it's a lot of effort, and if you are able to, you probably don't benefit from it. I'd advise against.


We recommend (and in some cases, require) that people do not use these tools, because they don't work as advertised. People who use them end up with questions that don't ask what they want to ask, and answers that don't say what they should. It is theoretically possible to use these tools in a way that avoids these issues, but that does not make it likely.

Consider the example from your question.

  • ChatGPT's reduced the Flesch reading ease from 53.90 to 34.14 with pointless word substitution:
    • “often” → “frequently” (note: subtly-different meaning)
    • “Ignoring” → “Disregarding”
    • “this” → “these uncertainties” (note: wrong)
    • “might” → “may”
    • “hard” → “challenging” (note: subtly-different meaning)
    • “detect” → “discern”
    • “ask” → “seek assistance from”
    • “improve” → “enhance” (note: subtly-different meaning, but more correct if you interpret it as jargon; probably coincidental)
    • “in a linguistic-only fashion” → “solely in terms of language” (note: worse, though the meaning's preserved at least)
    • “does […] then count as” → “would […] still be considered” (note: removing “then” would be an improvement, imo)
    • “result” → “outcome”
    • “Of course” → “Certainly” (note: subtly-different meaning)
    • “should be clear” → “should be understood” (note: quite different meaning; different grammatical interpretation of clause)
    • “I post it” → “posting”
    • “do my best” → “ensure […] to the best of my ability”
  • It's changed the strength of your convictions:
    • “am often a little bit unsure” → “frequently feel uncertain”
    • “I would proofread” → “I would thoroughly proofread”
    • “really wasn't changed” → “remains unchanged”
  • It's not actually fixed the few issues.
    • “unsure about grammar, wording, or phrasing of my posts” is lacking “the” before “phrasing of my posts”. This makes it ambiguous whether you mean:

      • “unsure about the grammar, wording, or phrasing of my posts” (all three restricted to your posts)
      • “unsure about grammar, wording, or the phrasing of my posts” (the first two in general, the last restricted to your posts)
      • “unsure about grammar, wording, or phrasing [, especially of my posts]” (all three in general – bracketed passage optional)

      ChatGPT's just picked one: not an appropriate solution when the author's meaning is unclear.

    • “improve my post in a linguistic-only fashion” is okay, but could be something like “improve only the language of my post”. What are the “terms” in “enhance my post solely in terms of language”? Not a correction.

    • “Of course, it should be clear that” is slightly hard to parse, but the meaning is clear from the context. This could be replaced by the idiom “It goes without saying” (originally French). ChatGPT's suggestion has a different meaning.

This is not a faithful copy-edit. The original was fine, and the “improved” version means something different. Allowing ChatGPT to invent (what appears to be) meaning is exactly the sort of thing we're trying to prohibit with our “AI generated content” bans. I'd say that this is not allowed, but more than that: it's pointless.

Even if a post's English is completely broken, with some of the words mistranslated and the grammar all wrong, I can usually grasp the meaning, and then I can edit. This is not a major barrier to contribution: Retrocomputing's most prolific contributor occasionally writes a paragraph like that, and everyone's happy with the situation.

If a post has been machine translated by traditional methods, I can sometimes work out what it was supposed to say, by spotting odd constructions and approximately reversing the translation in my head; but it's harder. If a post has been translated by ChatGPT, all of those signals have been stripped and smoothed out, and I can't parse any meaning from the resulting nonsense. There's nothing for it but to add comments asking what the author meant, and hope they don't use ChatGPT on their responses.

Regardless of whether it's allowed, using ChatGPT to “improve” a passage is a really bad idea.

7
  • 6
    Thanks, that was helpful. It shows me that one has to be very careful with this approach. Most of the subtile changes in meaning would not bother me, but where you marked “these uncertainties” as wrong is indeed a detail I missed by my own sloppiness. So I guess it will be ok to let ChatGPT suggest an improved writing, and then cherrypick certain phrases one-by-one, by verifying each individual statement that the meaning hasn't changed against the authors intentions. This, however, may gives indeed a bad effort/benefit relationship.
    – Doc Brown
    Jan 10 at 15:48
  • 1
    @DocBrown We used to have software that checked against a database of manually-chosen suggestions. Microsoft Word 2013's was quite good (but I think they've switched to ChatGPT now). LanguageTool still has this functionality, though their online offering has some kind of AI (and I'm not sure whether it's the “we're calling it AI because that's trendy right now” kind, or the “we're just throwing it at ChatGPT” kind).
    – wizzwizz4
    Jan 10 at 16:01
  • 1
    @DocBrown, what you describe seems like what Grammarly provides (at least last time I checked): it suggests localized corrections with explanation why. It's free version doesn't provide full phrase replacement, but based on declared goals, I'd say you don't need it.
    – markalex
    Jan 10 at 17:25
  • 6
    @markalex In my experience, Grammarly provided really poor suggestions, and that was before they added AI (GPT) suggestions. That's before getting into all the privacy issues. I would recommend against using Grammarly.
    – wizzwizz4
    Jan 10 at 17:32
  • If using Grammarly only for publicly posted answers in the manner similar to chatGPT, without plugins and such, I'd say no issue with privacy. Regarding suggestions: IMO, what it suggests way better than what GPT generates. And also it has great advantage of being only a suggestion, that you can reject.
    – markalex
    Jan 10 at 17:36
  • 1
    @Levente: I think this is not an adequate comparision. I am talking about write a new post, then let an LLM give me some suggestions on how to improve it formally, and then decide deliberately which of the improvements I might accept or not. Just as I would work with a human proofreader.
    – Doc Brown
    Jan 10 at 19:53
  • @DocBrown you are right, I have implicated a context that was not really relevant. Sorry.
    – Levente
    Jan 10 at 20:05
15

On sites where AI generated posts are banned, there are contributors and moderators on the lookout for such posts. AI generated posts were not banned just for the sake of it, they were banned because they caused significant disruption, by providing large numbers of poor quality answers.

If your post looks almost identical to an AI generated post because you've rewritten it using an AI, then your post, and by extension you, run the risk of being given the same treatment that AI generated posts are subject to on that site.

Neither moderators, nor other contributors are going to spend the time to decide whether your answer is a pearl in the AI sand, they are just going to treat it the same as all the other AI generated content. There's simply too much sand to do otherwise.

In short your post will likely be deleted and you may be suspended.

10

I guess an important question is if you actually need to.

English isn't my first language and I have a learning disability - and folks are generally very good-natured about fixing the quirks in my posts. As long as your content is comprehensible, people are generally willing to put in the time to understand and help fix a post.

That mutual aid and community engagement is an essential part of how the SE model works. There's no shame in having your post edited by others. It's how things work here.

I've also previously suggested elsewhere (I can't remember where) to have at least the first post in your own voice if you're using tools to extensively rewrite your posts. If nothing else, it lets people know what you started off with.

3
7

I have used ChatGPT to weed out grammatical errors, stylistic faults, unusual turn of phrases, false friends, and non-idiomatic expressions in students' English essays. It is faster than me. When you have something like thirty essays to read over the weekend, a tool like ChatGPT is a godsend. None of the suggested improvements by ChatGPT have been inaccurate, absurd or hallucinated-based.

However, if the OP decided to mindlessly adopt all the suggested corrections, their writing style will inevitably change, the post will begin to read like an AI generated text. An insecure non-native speaker should only adopt a few suggested corrections and improvements, hopefully, the worst offenders. Besides, an answer on the majority of Stack Exchange websites doesn't have to be 100% grammatical, and written in flawless English. A good answer just needs to be supported, clear and its meaning unambiguous. By rewording the suggestions sapiently, the author's original "voice" should prevail, and at least know any downvotes incurred are due to the quality of the answer and not about its English.

Finally, it's worth noting that not every error identified by ChatGPT is about grammar, sentence structure or semantics, sometimes it's just a question of a formal form being preferred over an informal one.

2
  • 6
    This is one of the things an LLM tool is really good at… It stinks for generating facts, but it is really good at generating grammatical, idiomatic English. It’s also great for generating writing prompts and example sentences.
    – ColleenV
    Jan 10 at 12:04
  • 3
    I think this is probably the best answer to the question; using an LLM as a copy editing tool seems best done in this way, in my view. Take the output as a set of suggestions rather than hard truths, and pull out what you don't like. This is essentially how I used something like Grammarly in college (well before it had any AI bits); a sort of improvement suggestion engine, rather than as a source of irrefutable, unchangeable truth.
    – zcoop98
    Jan 10 at 18:02
6

Yes, I would personally count that as AI-generated content. At least to such an extent that, as I wrote in another answer a while back, the LLM should be attributed as having written the text. Whether or not any site wants to allow that, is up to them, that can't be decided for all of them here on MSE.

Also, do note that if you're not sufficiently proficient in the English language as a non-native speaker, these tools are not your savior. In the above-quoted answer, I also posted text that didn't need changing into an LLM and the quotes show that the thing makes changes that aren't needed. If you run a quote through an LLM with a prompt to edit or rephrase it, and then run the result through the same LLM with another prompt to edit it, it will be edited or rephrased. No matter if it's needed or not, LLMs like ChatGPT aren't made for spelling/grammar checking or only rephrasing or rewording those phrases that need it. They just do as they're told, and change the text, sometimes for the worse. This means that whatever you get back, will be completely AI-generated and no longer yours, even though the text you get back contains a lot of words similar to the original ones you put in.

I would argue that this means that for wording/phrasing, if you're not proficient enough in a language to know something is wrong or right from the beginning, you can not at this point trust the LLM to make the right choice for you. Ask it to rewrite an already rhyming poem 'to make it rhyme', and you'll get some suggestions that don't rhyme at all. Putting this poem which rhymes very well into ChatGPT and telling it to 'rewrite this so it rhymes' gives results that rhyme 'vague' with 'haze', and 'labyrinth' with 'smith'. The LLM simply isn't smart enough to choose the right words or phrasing, or recognize that what it was fed was already very correct.

For spelling and grammar specifically, I'd ask you (and everyone else who's considering doing this) why you think there's a need to use the LLM for spelling/grammar. Using it for that seems like a case of 'when you have a hammer, everything looks like a nail'. LLMs like ChatGPT aren't specifically designed as tools for spelling/grammar checking. They just take a prompt and use some kind of statistical relationships they learned during training to make some text.

Instead of using the LLM for spelling/grammar, just use a tool that's specifically designed for spelling and/or grammar checking. Most devices and browsers these days seem to have default ones built-in, that give nice squiggly red lines when you make an error. Otherwise, there are plugins like Grammarly. There are a few advantages of these, the most major one being that do not make changes for you, they just give you suggestions. So you can always decide what to keep and what not yourself, and it is faster than having to compare two bits of text to see what changes an LLM made and if these should be kept.

The other serious advantage is that you get to skip the whole discussion about what should/shouldn't be allowed and it makes moderating AI-generated content a lot easier without having endless discussions about how much of a prompt should also be in the final text before something is/isn't considered AI-generated and thus could be removed on sites that have policies banning such content.

13
  • 8
    "just use a tool that's specifically designed for spelling and/or grammar checking... you get to skip the whole discussion about what should/shouldn't be allowed" even grammar tools are finding ways to involve AI: grammarly.com/ai
    – starball
    Jan 10 at 8:09
  • 5
    @starball then don't use that either. It's still a different tool from the default plugin.
    – Tinkeringbell Mod
    Jan 10 at 8:11
  • Totally agree with this answer, however @starball has a valid and sad point: sooner or later, everything will have AI, or be assisted with AI in one way or another. It's already happening, and there's no way to stop it. :/ Jan 10 at 8:35
  • 4
    @Tinkeringbell: first, I don't find your explanation insulting in any way, IMHO it is a solid base for a discussion. However, your answer is pretending my question is only about using an LLM for spelling and/or grammar checking. But I clearly wrote "wording" and "phrasing", and my impression is that is exactly where LLMs have their strengths. I am pretty sure you read that part of my question, but intentionally left it out in your answer.
    – Doc Brown
    Jan 10 at 10:26
  • 5
    @ShadowWizardLoveZelda It's 100% certain that the AI will hallucinate and change your text. You might be able to detect it, or not. But there's only way to fix it: don't do that. That's an exaggeration, and it's not true. ChatGPT if used as a proofreading tool does, in fact, do a more than decent job. Just don't ask it to write an answer for you. Jan 10 at 11:10
  • 2
    @DocBrown I did see that, and I would argue that LLMs do not have strengths there either. They reword and paraphrase, but if you look closer at what they're doing, half the time they're making things worse, not better. And it requires knowledge of the language to recognize this, and if you're a non-native speaker that's already unsure about something, that knowledge probably lacks. If someone can't recognize right from wrong already, what makes you think the LLM that rhymes 'smith' with 'labyrinth' or 'haze' with 'vague' will choose the rights words for someone?
    – Tinkeringbell Mod
    Jan 10 at 11:16
  • 1
    I've given my post an edit to make that point a bit more explicit, it was already sort-of in there when I said that LLMs will change text just because they're told to change text, but I've added a paragraph fleshing that point out.
    – Tinkeringbell Mod
    Jan 10 at 11:18
  • 6
    "I would argue that this means that for wording/phrasing, if you're not proficient enough in a language to know something is wrong or right from the beginning, you can not at this point trust the LLM to make the right choice for you." - sorry, but this argument is quite questionable. Most people (including me), when they know a foreign language, their passive knowledge is a lot better than their active knowledge. And I am not talking about some assessment if something is "wrong or right". I am talking of letting ChatGPT making linguistic improvements (see example in my question).
    – Doc Brown
    Jan 10 at 12:21
  • 5
    @DocBrown You're talking about making ChatGPT make new text for you, based on some prompt. That is, in its very essence, AI-generated content. Talking about it as if ChatGPT is 'making linguistic improvements' is simply not true, the AI generates an entirely new text based upon your prompt and whatever training data it has about what that prompt means.
    – Tinkeringbell Mod
    Jan 10 at 12:27
  • 2
    (as for the 'if you're not proficient enough', take that to mean the passive knowledge is also lacking. If you don't have the active and passive knowledge to know right from wrong from the beginning, an LLM won't save you).
    – Tinkeringbell Mod
    Jan 10 at 12:52
  • 2
    I really strongly object to this perspective that AI has to inevitably make input worse. These tools aren't being built in a vacuum– they're being built to be useful and worthwhile by their makers. Yes, they get a bad rap from people using them for silly things or things for which they're not "designed", but to go as far as saying "don't use Grammarly, a tool built for improving grammar, because it uses AI" is, I don't know how else to put this, absurd. Judge tools by their actual performance, not by some perceived AI boogeyman. You really water down your argument this way.
    – zcoop98
    Jan 10 at 17:50
  • Otherwise, your core argument is very valid– SE already bans folks from machine-translating posts made in other languages because translations "can be inaccurate, and even human translations risk distorting the intended meaning of the post", which very arguably applies strongly to the issue at hand– but your argument is made actively worse by not focusing on this aspect first, and AI second.
    – zcoop98
    Jan 10 at 17:53
  • 2
    @zcoop98 I've seriously considered deleting everything after the first paragraph and just linking to wizzwizz's post instead as they do a much better job of outlining why using a LLM sucks for rewriting/spelling/grammar then I apparently can. But voting sorted stuff already.
    – Tinkeringbell Mod
    Jan 10 at 18:12
4

Let me finally summarize what I learned from the interesting responses, and give my own answer since I think none of the other answers is really reflecting what I think "the right answer" should be.

As expected, the topic is discussed quite controversially in the community. So let's start with the point where we seem to have the most consensus, the question from the title: yes, such an "AI improved" post does count as "AI generated content". When we just copy/paste the output of an LLM into a post (assumed the specific SE site does not radically ban every AI generated content), even when done partially, it should at least be marked as such. This is something I perfectly agree with.

I found @wizzwizz4's answer in particular very interesting, who took the time to analyse my example in-depth. Above all, it showed me that an LLM changes nuances in the meaning of almost every sentence in the course of a linguistic improvement. Others mentioned the - sometimes undesired - change in style of the output.

However, I do not agree to the conclusion that the text becomes necessarily worse as a result, or that you generally cannot or must not use an LLM to improve your texts. In my example, the incorrect rephrasing of a passage by the LLM gave me an indication that this passage was ambiguous, could be misinterpreted by a human reader, and that it makes sense to reword it.

In the end, it clearly depends on how one uses an LLM. It is certainly advisable to use an appropriate prompt to instruct the LLM to work in a way that preserves meaning as much as possible. But even then, one should never blindly replace the original text with the revised text. If, on the other hand, one only takes the revised text as a reflection and then deliberately picks out individual terms or phrases that appear as an improvement (or possibly choose a completely different wording), this is IMHO acceptable. The extent to which one should or must then mark the final text as "LLM improved" certainly depends on the individual case and how much the OP has ultimately adopted from the LLM output.

Let me finally add I wrote this assessment in German and used DeepL for assisting me with the translation. ChatGPT was not involved. That is currently the most efficient way I know to deal with my deficits in English, and one which seems to be accepted by the community. Hence it is indeed questionable if usage of an LLM will bring much benefit over this approach - probably not when writers are able to express themselves in their native language in a decent manner, and a good machine translator to English is available.

11
  • 1
    LLM’s do not “interpret” text, so a poor single generation of a rephrasing is not evidence that it’s likely someone fluent English would misinterpret your meaning. That said people could misinterpret your post no matter how well-phrased it is; the only way to sort that out is to iterate based on feedback from readers. One thing I learned from participating on ELL is that often a sentence I believed had only one obvious meaning is more ambiguous than I thought. The best way to ensure your meaning is getting across is to interact with people to see how they understand it.
    – ColleenV
    Jan 12 at 12:24
  • 1
    @ColleenV: in some sense, LLMs do interpret a text. Don't forget we also use this term for compilers or interpreters(!) of programming languages. But I agree, I could have chosen a better word, let me fix this.
    – Doc Brown
    Jan 12 at 13:48
  • 1
    That is a different type of interpretation - you're mixing two different senses of "interpret" when you compare LLM parsing natural language to a person inferring meaning from a post. LLMs don't "know" anything and can not understand meaning. They understand patterns.
    – ColleenV
    Jan 12 at 13:51
  • @ColleenV: as you see, I removed the offending word "interpretation", do you think it is better now?
    – Doc Brown
    Jan 12 at 13:53
  • It wasn't the word I objected to, it was the idea that an LLM changing the phrasing of something (as you asked it to do) has any correlation with whether a human being would have trouble understanding the original phrasing.
    – ColleenV
    Jan 12 at 14:03
  • @ColleenV: yes, I understand this. I also noted that you did not answer my question if you think it is better now.
    – Doc Brown
    Jan 12 at 14:10
  • Let me be more clear then - however you phrase it, concluding that a human will have trouble understanding a passage because an LLM changed the wording of it is incorrect. Changing the phrasing of an incorrect assertion doesn't make it better if it is still incorrect.
    – ColleenV
    Jan 12 at 14:15
  • @ColleenV: maybe I wasn't clear about this: in my original text, there was an ambiguity which I wasn't aware of, and the "misinterpretation" - sorry - "wrong rephrasing" by ChatGPT made me aware of this. So when an LLM changes the wording in a manner which is against my intentions - yes, that it is a clear sign for me that there is a certain risk (not 100% evidence) that a human could "misinterpret" that sentence, too. P.S. thanks for edit.
    – Doc Brown
    Jan 12 at 14:28
  • This discussion about answers that summarize other answers on ELL's meta might be a little bit helpful, but it was really about Q&A on the main site and not about meta discussions.
    – ColleenV
    Jan 12 at 14:43
  • 1
    @ColleenV: ok, I addressed the title question directly in my answer. Better now?
    – Doc Brown
    Jan 12 at 14:48
  • 2
    @ColleenV: and yes, I think you are right with your other meta post about summarizing answers. But I think part of the idea of writing my own answer was not just to summarize, but also to express how I would answer my own question now, after getting all that feedback. I don't think there is another answer which really reflects my final conclusions, hence my own post.
    – Doc Brown
    Jan 12 at 14:57
3

As a non-native English speaker, I am often a little bit unsure about grammar, wording, or phrasing of my posts. Ignoring the fact that this might be hard to detect by others: if I ask an LLM like ChatGPT to improve my post in a linguistic-only fashion, without changing the meaning, does the result then count as "AI generated content"?

I think we should allow these use cases.

FYI, for reference, some research venues explicitly allow these use cases. E.g. ACL 2023 (which is a top-tiered conference for natural language processing): https://2023.aclweb.org/blog/ACL-2023-policy/: (authors: ACL 2023 Program Chairs)

Here is our take on some cases frequently discussed in social media recently:

  • Assistance purely with the language of the paper. When generative models are used for paraphrasing or polishing the author’s original content, rather than for suggesting new content - they are similar to tools like Grammarly, spell checkers, dictionary and synonym tools, which have all been perfectly acceptable for years. If the authors are not sufficiently fluent to notice when the generated output does not match their intended ideas, using such tools without further checking could yield worse results than simpler-but-more-accurate English. The use of tools that only assist with language, like Grammarly or spell checkers, does not need to be disclosed.

  • Short-form input assistance. Even though predictive keyboards or tools like smart compose in google docs are also powered by generative language models, nobody objected to them, since hardly anyone would try to use them to generate a long, unique and coherent text: it would simply not be practical. Similarly to language tools above, the use of such tools does not need to be disclosed in response to the writing assistance question.

  • Literature search. Generative text models may be used as search assistants, e.g. to identify relevant literature. However, we expect the authors to read and discuss such references, just like the references identified by a regular search engine or a semantic literature recommendation tool. The usual requirements for citation accuracy and thoroughness of literature reviews apply; beware of the possible biases in suggested citations.

  • Low-novelty text. Some authors may feel that describing widely known concepts is a waste of their time and can be automated. They should specify where such text was used, and convince the reviewers that the generation was checked to be accurate and is accompanied by relevant and appropriate citations (e.g., using block quotes for verbatim copying). If the generation copies text verbatim from existing work, the authors need to acknowledge all relevant citations: both the source of the text used and the source of the idea(s).

  • New ideas. If the model outputs read to the authors as new research ideas, that would deserve co-authorship or acknowledgement from a human colleague, and that the authors then developed themselves (e.g. topics to discuss, framing of the problem) - we suggest acknowledging the use of the model, and checking for known sources for any such ideas to acknowledge them as well. Most likely, they came from other people’s work.

  • New ideas + new text: a contributor of both ideas and their execution seems to us like the definition of a co-author, which the models cannot be. While the norms around the use of generative AI in research are being established, we would discourage such use in ACL submissions. If you choose to go down this road, you are welcome to make the case to the reviewers that this should be allowed, and that the new content is in fact correct, coherent, original and does not have missing citations. Note that, as our colleagues at ICML point out, currently it is not even clear who should take the credit for the generated text: the developers of the model, the authors of the training data, or the user who generated it.

A separate, but related issue is use of generative models for writing code. ACL submissions may be accompanied by code, which counts as supplementary materials that the reviewers are not obliged to check and consider, but they may do so if they wish. The use of code assistants such as Copilot is also a relatively new practice, and the norms around that are not fully established. For now, we ask the authors to acknowledge the use of such systems and the scope thereof, e.g. in the README files accompanying the code attachments or repositories. We also ask the authors to check for potential plagiarism. Note that the Copilot in particular is currently the subject of a piracy lawsuit, and may have suggested snippets of code with licenses incompatible with yours. The use of code assistance does not obviate the requirements of authors to ensure the correctness of their methods and results.

And CEUR-WS proceedings. https://ceur-ws.org/ACADEMIC-ETHICS.html:

In the past few months, we have witnessed the emergence of novel large language models (LLM) reaching breakthrough performance on NLP tasks. These include ChatGPT and Galactica, which are AI assistants that can produce long and good quality content that can be seeded for authors’ work. Because of their recent emergence, the norms around the use of such technology is not fully established, yet. Hence, it is important to acknowledge its use and elaborate on how it has been employed.

Specifically, we define three levels of AI assistance usage: insignificant, low and substantial. We will group the different use cases according to these three categories and we will define CEUR-WS stance.

Insignificant. Activities like: i) paraphrasing and refining the manuscript content (using Grammarly or other spell checkers), and ii) smart composition (via predictive keyboards) are widely accepted and do not need any acknowledgement.

Low. The use of AI tools for searching and generating literature review is acceptable upon authors’ checks. Authors must review the content and adjust/add references to line up with the narrative of their manuscript. In case of generating unoriginal content (i.e., definition, or description of well-known concepts) may be acceptable provided that the authors have checked it to be accurate and included proper references to the original content.

Substantial. Using AI assistants for generating new ideas as well as new text is unacceptable. Most of the generated content may derive from existing work. Potential issues with such practice are related to originality, plagiarism, ownership, and authorship, whose consequences and impact are not yet clear.

Regardless of the cases above, CEUR-WS publishes original work from named authors, and thus contributions from AI assistants can only be stated in the acknowledgements and/or by suitable references at the original research papers. We require that all authors and workshop editors adhere to these guidelines. Their violation will lead to the removal of the published paper or the whole volume, similar to our procedures dealing with plagiarism.

As this technology is in current development, we plan to continuously review this policy in the upcoming months.

This policy section is partly inspired by the “ACL 2023 Policy on AI Writing Assistance” available here.

Related documents:

  1. US Copyright Office's Guidance on AI-Generated Material (2023-03-16)

And I'd like to add a note on Tinkeringbell's answer:

I'd ask you (and everyone else who's considering doing this) why you think there's a need to use the LLM for this. It seems like a case of 'when you have a hammer, everything looks like a nail'. LLMs like ChatGPT aren't specifically designed as tools for spelling/grammar checking. It just takes a prompt and uses some kind of statistical relationships it learned during training to make some text.

Not being designed to be a specific tool doesn't make them bad. E.g., LLMs are not designed to do text summarization specifically, however they are state-of-the-art text summarizers. Therefore, the argument is invalid.

7
  • 3
    Thanks. I think Tinkeringbell's argument is questionable because I specificially asked about rephrasing, not just about grammar and spell checking.
    – Doc Brown
    Jan 10 at 12:15
  • 4
    @DocBrown Franck's answer quotes a source that says the same as I am trying to say though: " If the authors are not sufficiently fluent to notice when the generated output does not match their intended ideas, using such tools without further checking could yield worse results than simpler-but-more-accurate English." and "Using AI assistants for generating [..] new text is unacceptable.". The AI assitant when used as proposed in your question makes new text for you, and if you're not proficient enough to consider if that text matches your intended ideas, that backfires.
    – Tinkeringbell Mod
    Jan 10 at 12:24
  • 4
    @Tinkeringbell: I think I am not proficient enough in English to write as good as a native speaker, but I am quite proficient enough to validate if a text matches my intended ideas. That's the typical gap between passive and active knowledge of a language, and it applies to most people who know a foreign language.
    – Doc Brown
    Jan 10 at 12:33
  • 5
    @Tinkeringbell You've cut out a critical part with the "[..]"; the full quote is: "Substantial. Using AI assistants for generating new ideas as well as new text is unacceptable." - OP's usage best matches "paraphrasing and refining" which they count as "insignificant" and "do not need any acknowledgement". When even generating literature review or description of well-known concepts is only counted as "low", there's no way OP's kind of grammar refinement would be considered "significant".
    – SirBenet
    Jan 10 at 12:37
  • 2
    @DocBrown As I said to another commenter under my own answer, I'm not writing these answers just for you, the OP. If you are convinced you're wise enough to decide, then the first part of my answer applies to you: Yes, I count it as AI-generated content that needs attribution because the AI generates the entire content, and whether or not that should/shouldn't be allowed networkwide is not something that can be decided here on MSE. If you make mistakes though, don't blame the LLM.
    – Tinkeringbell Mod
    Jan 10 at 12:41
  • 3
    @SirBenet fair enough, here my non-native speaking got in the way a bit. I assumes 'as well as' was synonymous with 'or', but according to the dictionary it means 'in addition to'. I will still stick to my opinion that whatever an LLM like ChatGPT does though, is taking a prompt like 'rewrite this' and making some new text for you based on your prompt and whatever training data it has, and doesn't work like e.g. a human editor would, keeping all the source text and just suggesting/making needed changes, and as such whatever comes out a.) is AI-generated and b.) needs attribution.
    – Tinkeringbell Mod
    Jan 10 at 12:45
  • @Tinkeringbell: fair enough. Yes, I think attributition of "AI improved answers" should be mandatory. Under which constraints these answers are acceptable at certain sites is indeed something which should be discussed at the individual meta sites.
    – Doc Brown
    Jan 10 at 14:07
3

To stay on the safe side, perhaps you could ask ChatGPT to point out and explain any errors/poor wording rather than asking it to rewrite the post? This should result in fewer unnecessary changes and (since it's you applying the edits) reduce the chance of any hallucination going unnoticed.

Using Bing (GPT-4) for grammar correction

There's definitely some degree of grey area. Common grammar improvement tools like Grammarly and the suggestions in Google Docs/Microsoft Word already make use of machine learning/language models. I wouldn't be surprised if the built-in checkers of browsers/OSs soon move to similar approaches.

To my understanding, the grounds for some SE sites banning AI generated content is primarily the generation of nonsense that is not easily dealt with through votes/flags like regular spam due to appearing legitimate at a glance. On that basis I'd suspect that approaches minimising the risk of nonsense, such as writing the initial post manually and proof-reading the AI-suggested edits, are in theory fine.

23
  • 2
    Yes, we have banned AI generated content because it is mostly spewing nonsense. And we can only successfully fight such content based on the ground that it is AI generated. Because rephrased AI looks very similar (the same) as completely generated content allowing AI usage for rephrasing is basically allowing all AI as we are losing our only viable tools for fighting it. Jan 10 at 12:58
  • 3
    @ResistanceIsFutile I'm not convinced that a human-written post with some AI-suggested fixes would even show up above the noise that AI text detectors inherently have. Other factors, like the user's behavior/history, would also point to human (to whatever extent it does for other human posts).
    – SirBenet
    Jan 10 at 13:03
  • 3
    AI text detectors are not used when detecting AI posts on SE. And yes, people can tell the difference. Even in the above short example, one can detect AI writing. Maybe not enough to warrant immediate post removal, but enough to put the user on someone's radar. So anyone who uses the AI even for merely rephrasing what they have written risks their posts being removed and suspension. And high reputation users are no exception. Jan 10 at 13:21
  • @ResistanceIsFutile "AI text detectors are not used when detecting AI posts" - I don't mean to say that it's the sole factor (the opposite, in fact), but it is definitely at least used by users raising flags. IIRC moderators used to even have a browser extension to streamline feeding posts to huggingface.co/roberta-base-openai-detector but were stopped by SE staff. "[..] risks their posts being removed and suspension" - For reviewed grammar fixes alone to trigger a suspension would suggest to me a huge false discovery rate from overconfidence in ability to detect small text changes.
    – SirBenet
    Jan 10 at 13:45
  • 1
    "IIRC moderators used to even have a browser extension to streamline feeding posts to huggingface.co/roberta-base-openai-detector but were stopped by SE staff. " I don't know from where you got that, but it is not true. Maybe some moderators used AI detectors, but not the Stack Overflow moderators that were responsible for removing the majority of AI posts. Jan 10 at 13:56
  • 1
    It is not a false flag if you can detect that some post was rephrased by AI. Also depends on how large those changes are. For someone proficient in English, the changes might be small enough, but for people that are not changes will be more prominent and AI will be more easily detectable. So when you say "it is fine to use AI to rephrase your post", you are actively suggesting to many people they can use something that can get them into trouble. Jan 10 at 14:01
  • @ResistanceIsFutile "I don't know from where you got that, but it is not true. Maybe some moderators used AI detectors, but not the Stack Overflow moderators" - See here for the claim directly from an SO mod: meta.stackoverflow.com/questions/424979/… "Another userscript allowed us to run posts against AI detectors like Huggingface, which we and other community members used to detect ChatGPT posts."
    – SirBenet
    Jan 10 at 14:09
  • @ResistanceIsFutile "It is not a false flag if you can detect that some post was rephrased by AI" - I don't mean to say here that the grammar-fixed posts themselves would be false positives if detected (although that could be argued) but rather that the sensitivity you'd need to have to suspend users based on text-only grammar fixes, when everything else is legitimately human, would cause a high false discovery rate of entirely human posts.
    – SirBenet
    Jan 10 at 14:25
  • I stand corrected. My source is meta.stackexchange.com/a/389844 Jan 10 at 14:25
  • "when everything else is legitimately human" First of all, on some SE sites, even if you write half of the post using AI, the whole post will be removed. Second, what is "everything else" in this context? If the post is rephrased by AI, people can only see rephrased content, we cannot see and verify that AI made only grammatical fixes. The sensitivity you are talking about does not exist as such. Rephrased content can easily look like completely AI generated. Jan 10 at 14:38
  • @ResistanceIsFutile "Second, what is "everything else" in this context?" - factors used to determine if an account is abusing AI. Are they answering questions at an inhuman rate? Are they giving links/functions/.. that don't actually exist? Or conversely, do their posts have images? Do they mention features that are too new to have been in training data? Nothing conclusive on its own, but plenty to factor into a judgement. "sensitivity you are talking about does not exist as such" - Not certain what you mean by this. If reviewed grammar fixes alone can cause suspension, sensitivity is high
    – SirBenet
    Jan 10 at 14:54
  • You are talking about "abusing AI". On sites where AI content is forbidden, all AI content even partial is not allowed. So ye, grammatical fixes (rephrasing) being done by AI will be detected. And not because "there is high sensitivity" but because it was AI generated or assisted content that was recognized as such. We don't judge quality and correctness of AI answers because there is not enough people on sites who would be able to do that. And not all hallucinations can be recognized by many people. Jan 10 at 15:23
  • 3
    AI rephrases. It does not just fix the grammatical errors. Again, I am not talking about tools, I am talking about people. I don't need any tools to tell that you are also using AI in your posts. To which extent, I don't know, but the point is that it can be recognized. Jan 10 at 15:56
  • 1
    @SirBenet the LLM isn't making a typo fix, it was prompted by the software, and the software presented the result as a typo fix.
    – Kevin B
    Jan 10 at 16:28
  • 2
    @SirBenet within reason... sure. but if you're giving tool X a paragraph of text and it's just sending that text to an llm and getting "reformatted" results and running with it, that's AI generated content. Just because tool X can be used responsibly to improve grammar and fix typos doesn't mean every usage of it isn't producing AI generated content. The example in the question is AI generated content.
    – Kevin B
    Jan 10 at 17:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .