-40

ChatGPT is there and there are good reasons to disallow answers that are just copy and paste from ChatGPT. On the other hand, there are no good ways to tell for sure. GPT-3 detectors detect ChatGPT, but on the other hand they also judge genuine student essays as written by GPT-3.

How can we resolve the problem? Embrace it!

Stack Exchange should negotiate with OpenAI a deal for integration with ChatGPT and provide one "ChatGPT answer" to each question, generated using the OpenAI API and clearly marked as an AI answer.

Benefits for Stack Exchange:

  • Less incentive for users to post unmarked ChatGPT answers
  • Instant answers to simple questions
  • ChatGPT is clearly marked and users know not to trust it without checking
  • Users could be allowed to create answers based on the ChatGPT answer, which improve on it by fact checking, fixing minor problems with the answer, explaining details and so on.

Benefits for OpenAI:

  • Training data from Stack Exchange
  • Publicity
  • Making developers curious about their API

Related: Ban ChatGPT network-wide

6
  • 3
    I think anyone who wants to make ChatGPT answers easier to post here should first try asking it a couple of questions you already know the answers to. Not only does it usually get it wrong, but it can get it wrong differently each time you post the same question. I asked it the same Q twice yesterday & got two very convincing answers… the only issue was the problem I asked it about I already knew there was no viable solution. Didn't prevent it presenting quite credible-looking horse-apples.
    – Tetsujin
    Dec 20, 2022 at 17:20
  • 1
    @Tetsujin At the moment I mostly agree, but AI is evolving at a impressing rate. People already talk about Moores law of AI. Just a few years ago we were happy about AI being able to have a basic understanding of grammar. Today we're talking about its problems when answering complicated questions in dialog form. The complaints about the answers are already at a very high level and the difficulties to distinguish AI answers from human answers shows how advanced current models are. AI has improved by orders of magnitude in the last years and it seems to continue being improved at this rate.
    – allo
    Dec 21, 2022 at 11:11
  • 1
    I think the transition from almost good enough to very useful was from ChatGPT to GPT-4. "It happend". Mar 19, 2023 at 21:29
  • This may be more appropriate at some sites than others. And for some questions than others. It's not a bad idea, really, but network-wide is too much. Apr 13, 2023 at 2:46
  • Would the corresponding ChatGPT answer automatically update whenever a question is edited? (On Reddit, if you downvote a bot's post, it'll auto-delete it. Maybe something similar is an option here.) Apr 13, 2023 at 3:21
  • It would be an interesting idea to see how the answer changes with question changes. Also ChatGPT probably could do something with the diff or a description what was changed, when looking at how it behaves when you give feedback in chat form. Another thing would be generating a new answer when ChatGPT is improved, like when they introduced the GPT-4 version and when they will in future update with an updated dataset.
    – allo
    Apr 13, 2023 at 8:00

4 Answers 4

14

Less incentive for users to post unmarked ChatGPT answers

I do not see this as substantially lessening the incentive. You can already find questions with multiple ChatGPT generated answers posted to them. It does not really seem users who take the cheap and easy way to generate content really care that much.

Instant answers to simple questions

Except the network is built around good answers. There is absolutely no guarantee that an answer generated by ChatGPT is correct. Quite the opposite - the answers are very often wrong. Either inaccurate enough to pass for an answer at a glance and fail at the details, or may even contradict itself within the same paragraph or advocate using non-existent, impractical, or wrong solutions.

The answers generated also often enough fail to actually answer the question, since ChatGPT failed to identify a major component. Like "I want to do I used X but <doesn't work because>" and the answer is "You should use X".

Yes, ChatGPT does also answer correctly. But therein also lies a problem - most simple questions are already answered. They should be marked as a duplicate instead.

ChatGPT is clearly marked and users know not to trust it without checking

Let me be the bearer of bad news. Users do not check the validity of answers. No matter how they are marked. Yes, not all of the users but enough of them.

And we do have very concrete proof of this happening: see this bug which exists in Razer and Docker for Windows. Summary the Razer driver manager and Docker could not be launched together on Windows. The bug was due to an answer on Stack Overflow (screenshot as it is deleted) which initially proposed a faulty mechanism to uniquely identify an executable. It was faulty because it would generate the same ID for any executable. And it was used to prevent an executable running twice. In the case of Docker and Razer Synapse, both thought the other was a second instance of themselves. Thus if Razer Synapse is running, Docker cannot be started.

This is far from the only example. It is, however, the most visible and best for illustrating that users do not verify. There is a highly upvoted comment under the answer pointing out the problem.

It is also quite an important example, as it reveals a misconception many seem to have: the answers posted are not for the question asker. They are for everybody on the Internet. Anybody can ask a question and software engineers over at Docker and Razer can use the answers. Even if one user does verify the contents of an answer, there are many others who will not. And many more users may be taking advantage of content throughout the years.

Which is another reason why it is dangerous to leave wrong content.

Users could be allowed to create answers based on the ChatGPT answer, which improve on it by fact checking, fixing minor problems with the answer, explaining details and so on.

To be honest, I am all for allowing AI generated content in some form. However, I do not believe this is the way. It undermines the foundation of what Stack Exchange is. Reputable answers, each of which ideally stands by itself. Instead, this proposal will fill the network with untrustworthy, potentially wrong, dangerous, or plain misguided answers, and then expects real humans to pick up the slack and verify whether the autogenerated answer works, or not, why and how. And that is a tremendous issue.

7

Interesting idea, but we should first analyze how often ChatGPT gives an incorrect answer to an SE question to see whether it is good enough to be used more systematically, and SE may want to add some mechanism to mark an answer as incorrect that would complement the current voting system (which could also be beneficial for non-ChatGPT answers too).

Also, we'd have to think how to handle new generative question-answering systems and language models (e.g., GPT-4) in terms of updating old AI answers.

Lastly, it's still a bit unclear how a language model can keep track of the provenance of the main knowledge/sources used to generate a given output, and in some cases ChatGPT may go beyond the fuzzy boundaries of fair use: aside from the legal aspect, unreferenced answers sometimes aren't that satisfying (e.g., hard to check the sources and can't read the additional information that the sources may have provided).

1
  • 1
    I think it will often be wrong and should be marked as likely wrong. But it will be (almost) correct for quite a few answers and that may be worth something. Voting can help and editing can help even more to transform a somewhat correct answer into a nice answer. And I suppose OpenAI keeps improving on ChatGPT and together with data from other answers it may be able to answer, for example, many duplicates correctly with things learned from the original question(s). I am also in favor of banning ChatGPT answers from users as possibly dangerous, but adding a canonical AI answer would help there.
    – allo
    Dec 19, 2022 at 22:02
1

Not at the current time, no.

We don't know where things will be in six months or two years, and this MIGHT be viable at some point, and at some future point AI's may even join as individual and productive community members. But right now is not yet time to embrace something so new while it is still in a state of flux and change. The NEXT chatbot could be evil for example.

And of course, it hasn't been demonstrated definitively where the value is, currently.

"We can't always tell, so what the hell" is not good policy.

0

Integrating ChatGPT, it's not as simple as adding a code library or enabling a plugin. The Stack Overflow and Stack Exchange system and the community aren't ready for a ChatGPT integration.

ChatGPT, like any software application, is a helper for the person on the screen; the person on the screen is still accountable for the content they deliver and their decisions. Unfortunately, the current reputation system cannot warrant that people use ChatGPT to help communities reach their goals: build a library of quality answers to any question about the community's scope and workings.


ChatGPT (https://chat.openai.com) is a free research preview intended to get external feedback to improve OpenAI systems that could generate content that doesn't meet the Stack Exchange rules and policies; it should not be integrated into the system that powers Stack Overflow and Stack Exchange sites. The ChatGPT API could be used in specific tasks, but the systems and workings should be improved radically.

ChatGPT power users understand how ChatGPT work. They might use it directly, or tools that use the ChatGPT API in the same way that an expert writer uses a dictionary, a thesaurus, a spelling and grammar corrector, or note-taking, among other tools help to do a task. I.E. ChatGPT power users might find it helpful as :

  • Search Assistant,
  • Writing assistant for tag wikis based on the already tagged questions,
  • Tool to summarize the old community rules, policies and standards,
  • Tool to analyze the discussions, i.e., as a stone checker.

ChatGPT, in any way, should accelerate the individual participation of people who do not have the required competencies to use it responsibly. So far, Stack Overflow and the Stack Exchange sites rely on the moderation done by the community, but many sites haven't a core community; they "survive" thanks to the solidarity of Stack Overflow and Stack Exchange power users that help to keep sites free of spam and to clear the review queues.

Recently, the CEO posted a blog entry about their plans to use AI and an example of the company's experiments. It will take a long time before something like ChatGPT, a highly flexible, interactive tool for community members, might be integrated.


Addendum

I just signup with this account into ChatGPT. It still warns that it's a "research preview", unreliable for generating content.

Welcome part 1

ChatGPT

This is a free research preview.

🔬 Our goal is to get external feedback in order to improve our systems and make them safer.

🚨 While we have safeguards in place, the system may occasionally generate incorrect or misleading information and produce offensive or biased content. It is not intended to give advice.

Welcome part 2

ChatGPT

How we collect data 🦾 Conversations may be reviewed by our AI trainers to improve our systems. 🔐 Please don't share any sensitive information in your conversations.

Welcome part 3

ChatGPT

We'd love your feedback!

👍 This system is optimized for dialogue. Let us know if a particular response was good or unhelpful.

💬 Share your feedback in our Discord server.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .