-26

A few days ago, OpenAI, the people that brought you GPT-3 and DALL-E, launched a new product called ChatGPT. It can answer questions so accurately that it's legitimately scary. Even the most obscure topics known to man are answered swiftly by this bot. It's not 100% accurate, but it's shockingly accurate for an AI. To explain how accurate it is, someone posted on Twitter that they managed to take an SAT test with the bot and received a 1020, which is above the 50% percentile of all its test takers. People are calling this automated system as something that's better than Google.

My question is, could this be a way to accurately answer questions that are on any of the Stack Exchange sites? I entered a question from Stack Overflow relating to the NumPy Python module and it gave a pretty convincing response. The responses are professional, probably even better than similar sites such as Quora where the responses vary in quality.

Would a bot such as this one change the future of question and answer sites?

14
  • 10
    Related on Meta SO: How do you plan on tackling chatGPT answers? Dec 4, 2022 at 4:32
  • 23
    Anyway, a response by ChatGPT: "It is possible that a language model like ChatGPT could be used to provide answers to questions on Stack Exchange sites or other question and answer platforms. However, it is important to note that language models like ChatGPT are not capable of understanding the context or meaning of the words they use to generate responses. They are trained to produce text based on the input they receive, but they do not have the ability to think or reason like a human. As such, the responses they generate may not always be accurate or relevant to the question being asked." Dec 4, 2022 at 5:07
  • 9
    People already using AI to answer questions and get suspended for that these days. Yes, this will make much more people do it, and get suspended, and require lots of efforts from the mods to hunt them down. Change future? lol (People are so blinded by the drawings they forget what the "I" means in AI.) Dec 4, 2022 at 5:20
  • 2
    I feel like just like we have one bot as SmokeDetector in chat for identifying spam posts all over StackExchange we can have one common bot for answers all over the network. However while this idea is good, a bot can't answer comments and also bot answers can be wrong. While the first argument in my previous sentence can't be resolved, the second can be resolved by adding a new option in Flag "Wrong answer by bot". However as in a whole I don't feel we need ChatGPT to answer people's question. Dec 4, 2022 at 7:33
  • 2
    Needs detail/clarity: "My question is, could this be a way to accurately answer questions that are on any of the Stack Exchange sites". Do you mean any question / all questions? Or just good actually answerable questions?
    – starball
    Dec 4, 2022 at 7:48
  • 12
    @JitendraSingh very big no. Smoke Detector has no AI at all, it got totally nothing to do with this. Dec 4, 2022 at 10:17
  • 1
    What is a SAT test? The US college admission test? Dec 4, 2022 at 13:06
  • It was also tried on Wikipedia. Dec 4, 2022 at 13:52
  • 1
    Request for clarity: When you say "could this be a way to accurately answer questions that are on any of the Stack Exchange sites?", do you mean to answer the question here, actually on this site (Stack Exchange)? Or do you mean that the question could be asked not here (not on Stack Exchange), and answered not here (not on Stack Exchange)? That's an important detail.
    – starball
    Dec 7, 2022 at 21:09
  • 4
    The fact that it can score a 1020 on the SAT is perhaps an indictment of the SAT rather than an accolade for ChatGPT... Dec 8, 2022 at 21:57
  • 3
    I was going to be smart and ask ChatGPT this exact question to see what it says, but then I noticed you have to give them your phone number in order to ask it anything, so I gave it up.
    – einpoklum
    Dec 12, 2022 at 22:50
  • @einpoklum not just give number, give access to your phone aka sell your soul to soul-less being. ;) Dec 14, 2022 at 12:17
  • @NathanielFord, the difference is that a human test-taker doesn't have instantaneous access to millions of online documents.
    – WGroleau
    Jun 12, 2023 at 16:50

5 Answers 5

57

While the solutions often look correct to the untrained eye, they are frequently invalid in various subtle ways. As such they are actively harmful as they encourage the question poster to try things that simply don't work.

In one deleted answer the documented return value for COLUMNPROPERTY is either 1 or 2 but the chatGPT answer had code that was predicated on 3 being being returned, that code would never execute and therefore the answer didn't solve the question's issue. Worse than not working though was that it looked like it might be OK to someone unfamiliar with the possible return values of COLUMNPROPERTY.

When the question poster then comments that there is a problem with the answer, neither the answerer nor chatGPT is in a position to respond as neither understand the comment or what to fix in the answer to address it. I pointed out a number of such issues in my Stack Overflow answer

All known chatGPT answers on Stack Overflow have had to be deleted. When the post author didn't do it voluntarily, moderators have done it for them. Be warned that deliberately making work for moderators can result in suspension.

Having the same variable answer quality as Quora is not something to aspire to.

Note that using chatGPT to create answers on Stack Overflow is currently banned

13
  • 5
    I wonder when a spaceship will crash due to programming bug resulting in AI writing the code. I'm 1000% sure this will happen, question is only when. And only then people will finally understand the dangers of AI and stop using it for things it should not be used for. Dec 4, 2022 at 10:19
  • 13
    I don't think you can use the word "quality" in the same sentence as "Quora" Dec 4, 2022 at 12:13
  • 3
    In related news, Quora have had AI moderation bots of unspecified IQ roaming it for several years now. It is a disaster on many levels. Dec 4, 2022 at 12:42
  • 7
    @ShadowTheKidWizard we humans are perfectly able to do that ourselves without any AI involvement. Dec 4, 2022 at 13:00
  • @RobertLongson no doubt! But AI will make it so much worse. People will think it's perfect, just to find it failing over and over again. Dec 4, 2022 at 13:20
  • 1
    Afaik there is no way to identify the chatgpt answers.
    – peterh
    Mar 31, 2023 at 3:14
  • 1
    @peterh you're wrong. If you see enough of them you can even recognise them on sight. Mar 31, 2023 at 6:58
  • @RobertLongson Well I did not see too much of it. First because it wanted my private data, so I registered to it over tor with a fake account, then I can not use anything directly from that because it would make me identifiable :-) But I still find shocking the surprising vehemence with what all the chatbots want our private data (google account, for example).
    – peterh
    Mar 31, 2023 at 16:23
  • 1
    ?? You're concerned about private data, yet you have an account on NSA's biggest competitor?
    – WGroleau
    Jun 12, 2023 at 16:54
  • Partner, not competitor. Jun 22, 2023 at 14:03
  • Alas, the irony of the AI ban is that the usage of StackOverflow is down significantly because many developers use ChatGPT instead of StackOverflow. Aug 28, 2023 at 11:02
  • @justadeveloper Presumably they will come back when word gets around about how bad ChatGPT's answers actually are is when you ask it programming questions. Aug 28, 2023 at 12:01
  • @RobertLongson In my experience, it's been excellent. It not just ChatGPT though. Code Assistants like GitHub Copilot are also excellent. I cannot imagine writing code without AI anymore. My guess is soon everyone will use AI while coding in some shape or form. So this ban will be meaningless. Aug 29, 2023 at 13:40
25

I guess that a fundamental part of the question is what is the real value these answers bring. While in theory machine learning systems can string together plausible answers, a good chunk of what a good answer is involves experience and understanding. Many of these answers look good, but they are useless. One expects tasty cake, and finds frosting covered styrofoam.

Another thing is why are you here if your contributions are literally "type a question into a box, and blindly post the answer here". I've often talked of Stack Exchange as a commons, and essentially posting such answers is basically an act of vandalism. To borrow a term invented by Phillip K Dick, it’s kipple - useless detritus.

I'd argue that SATs, with a distinct, very focused domain and syllabus and a finite question bank, would also be different from Q&A where folks would have very arbitrary questions.

Basically, don't post solutions you didn't try or test yourself, or know for certain that they work, and something from a source like ChatGPT is suspect.

For factual questions – well, please confirm that well, the answer is factual, and not the fever dream of a hundred monkeys on a hundred typewriters.

0
15

It can answer questions so accurately that it's legitimately scary

It will be just as scary when it's wrong, but many won't realize until the consequences™ come.

Would a bot such as this one change the future of question and answer sites?

With the benefit of hindsight, to a degree, yes, of course, and to some degree, they'll continue to do so.


For context, this post's original revisions took a much more defensive stance, viewing the question from an angle of "how useful would it be for AI-generated answers be posted on Stack Exchange?" My previous answer focused heavily on need for quality in questions, and my defense for the continuing value of the Stack Exchange network was based on "garbage in, garbage out"- both for questions and source/training data:

  • If someone doesn't know how to ask a good question, they're going to have a harder time getting a good answer from an LLM tool.

  • For any new field with less information about it included in LLM training data and less SEO juice to get good info found in web search results, likelihood of a LLM + search engine giving useful results is probably going to be low.

A year has passed, and the aspect of this question that I find interesting has changed. Concerning the future of Stack Exchange, my new interest is in the net benefits this technology has provided / is providing- not on the whole world, but here.


could this be a way to accurately answer questions that are on any of the Stack Exchange sites?

Sure? (to an extent.) Nowadays, there are hybrids of search engines and LLMs, so all it needs to do is do a web search, and basically churn and regurgitate any existing answer content.

I've heard tons of people online and my friends say that they find technologies like ChatGPT to be a useful tool for getting answers to questions. Many of them who have newbie experience on Stack Exchange comment that they prefer it over Stack Exchange so they don't have to deal with grumpy experts.

That's what ChatGPT is good at: service with a smile, all the time.

People like that. Traffic has certainly declined (see Did Stack Exchange's traffic go down since ChatGPT?).

Okay, so how the heck do I see benefit in that?

Dealing with problematic question content is costly. It costs time, and it costs patience. It costs that to deal with questions that are missing key details, questions that are (for lack of better description) just a mess, questions that are duplicates, questions where potential for long-term value is covered in three feet of unnecessary context (and usually the hidden gem has already been asked before), etc. And each of those categories abound endlessly. Deal with it enough and it sucks the joy out of building a long-term knowledgebase.

ChatGPT and similar technologies do two things for that:

  • It "tricks" people into doing research and basic troubleshooting.

    I say "trick", but really I mean that it masks research-related activities into a form that people find palatable. For some reason, many people either don't like to, or don't know how to google things (not that googling is a pre-requisite for asking a question on Stack Exchange (see How should we deal with Google questions? and Embrace the non-Googlers), but in my experience, it's usually (for question askers) a better search engine than the one built into Stack Exchange sites). But when the search engine is hidden behind a conversation with a machine, suddenly people find it very attractive (I guess the degree of synthesis and summarization (accurate/faithful or not) is also a factor).

    This isn't always a productive thing. We all know that an LLM can spit out falsehood as well as it can spit out truth. It can mislead people, and that can result in those people posting weird questions on Stack Exchange (Ex. I'm trying to do X and ChatGPT told me to do Y (which makes no sense)).

    But a lot of duplicates are very poor in quality, and less of those is a good thing. It means less curation workload, and less clutter mucking up searchability (bad signposts weigh out good ones). I've also started to see a (very small but hopefully increasing) trend in questions that show actually useful information prompted by conversation with an LLM. While this is slightly disappointing to me, since a lot of basic troubleshooting can be written statically and has no need for neural network involvement, it would seem that many of the people who love this technology are not the type to seek out the loving manual.

  • It (to some degree) deals with the boring, obscure, one-off requirement dumps / questions that just require a lot of synthesis of basic concepts / questions with a bunch of unnecessary context.

    Or at least- that's what it seems to me like people who are trying to develop developer tooling around these technologies are trying to apply it to do.

    People are often too lazy to do or just don't know how to do is generalize their questions (de-localize / create things like Minimal Reproducible Examples) while not blowing up the question scope (staying focused), or to break them down into separate questions that are appropriately scoped for Stack Exchange. I don't get an awful lot of those questions in the tags I answer in, so I don't know how much this situation has changed, but I hope that among the classes of questions that are no longer being posted, there are fewer of these.

And in doing those boring things, it doesn't get grumpy. It has no social battery (well, it consumes a heck ton of energy to run, but that's a different can of worms).

I don't particularly care if ChatGPT wastes someone's time with a bunch of unhelpful info.

To the contrary, I welcome any side-effect that this technology has had on reduction in lower-quality question posts / increase in overall quality of question posts on Stack Exchange.

There are potential hidden costs- fewer people using the sites directly means fewer votes (and voting matters), and potentially missing some gems of questions and losing out on them to the machine. But a lot of people can't vote anyway, and I think if a question is useful enough, as long as this platform itself can retain its reputation as a trusted resource, the gems will find their way here.

5
  • 1
    "If we get some magic-bullet™ that can answer any* question... on the spot, many of the Stack Exchange sites mostly wouldn't be needed anymore." Whoa there! The value of the Stack Exchange ecosystem goes way, way beyond just a collection of not-wrong answers. There is a very real, very large and very vibrant collection of communities of interacting people here!
    – uhoh
    Dec 11, 2022 at 20:40
  • 1
    @uhoh yeah I shouldn't have said that so confidently as such a new community member. Could you elaborate? I might want to edit some of your ideas into my post (with your permission if you'd give it). This also reminded me of something I read from someone from a math SE site: that each person is different, and having something explained in many different ways can be useful too. I'll try to go dig up what post that was.
    – starball
    Dec 11, 2022 at 20:42
  • 4
    SE sites exist to answer questions, not to maintain a community. If a chatbot could feasibly answer any and all questions put to it, the communities could very well go elsewhere to continue being communities, and probably would be better to, since much of the reason for the community would be redundant and the few useful functions left for humans are very suboptimal for socialising and interacting in a less-than-formal way. SE would not be needed in that case, and you are right to say so.
    – Nij
    Dec 17, 2022 at 23:37
  • @Nij The Site Tour begins: "Stack Exchange is a network of 180 communities that are created and run by experts and enthusiasts like you who are passionate about a specific topic. We build libraries of high-quality questions and answers, focused on each community's area of expertise." Essentially, SE is about community, and each community is here for the purpose of building libraries of Questions and Answers on their area of expertise. Perhaps we could then say that SE sites exist to enable communities of experts to build Q&A libraries. Apr 28, 2023 at 11:25
  • @Stevecanhelp the purpose really is the library of answers, not the community around it, for all that we've moved in a squishy direction over the years that departs from that real mission. I wrote about this when some Code of Conduct changes were proposed: meta.stackexchange.com/a/312206/307622
    – Wildcard
    Feb 27 at 9:28
1

"[ChatGPT] can answer questions so accurately that it's legitimately scary." It can also make things up, including citations to non-existent sources. Just say NO.

In response to something I wrote, it generated three paragraphs of politician-sounding pablum on a topic unrelated to my input (and the third paragraph was a repeat of the first).

Asked about the influence of Elvis Presley on the Sex Pistols, its response was 180° opposite from a public statement by Johnny Rotten.

It's been out less than a year, and I've already forgotten the other travesties I've seen. I consider it a minor improvement to an algorithm actually named "Travesty" from decades ago.

It has the potential to be a greater source of falsehoods than Facebook.

5
  • Well, Facebook fake news are coming from humans using bots to spread them. Humans with certain agenda, be it good or bad. But would be much worse to be flooded with shiny things that appear perfect, while actually being empty and false, making people believe in things that don't really exist. That's a great danger, that nobody is willing to see or act against. Those who can prefer to just join the fest and earn their share of the fake cake. Jun 12, 2023 at 14:28
  • And ChatGPT's sources are (supposedly) internet content. Does that include Facebook, which is already helping people believe in things that don't exist? Worse, what if ChatGPT's handlers decide to allow it to scan right-wing nonsense while blocking it from left-wing sources (or vice versa)?
    – WGroleau
    Jun 12, 2023 at 14:39
  • At least one of the ChatGPT interactions I've witnessed was a clear "attempt" to tell the user what he wanted to hear, even contradicting itself when the user pushed back at a statement. A step up from "Eliza" regurgitating the user's comment in the form of a question.
    – WGroleau
    Jun 12, 2023 at 14:48
  • 1
  • Yeah, read about it back then, and don't think it has been improved. Jun 12, 2023 at 17:18
-12

I would think there is definitely scope for AI to assist humans in providing good answers. Especially the latest version ChatGPT4 (with plugins like Wolfram Alpha for maths or ScholarAI or the web browser plugins to give sources) has very good performance quantified based on various benchmarks, e.g. ability to solve university entrance exams correctly (often top10% relative to avg. human) or maths ability. See my answer here giving some examples of how with the help of ChatGPT4 I was able to answer various SO questions, including ones that had never received an answer, despite being highly ranked and having been awarded 4 bounties before. Of course adequate advice on responsible AI use would need to be given, cf. the guidelines of many academic journals like PNAS on the use of generative AI. Especially for programming related questions I think allowing responsible AI usage could be beneficial. The guideline should then include that if ChatGPT or Github Copilot was used to develop code that reproducible code & a benchmark plus output should be given (confirming that the user verified the code works correctly, ChatGPT can't yet run code yet) & that use of ChatGPT should be disclosed & a reputable source should be given. That is very easy to check - much easier than telling if a given answer might have used ChatGPT if not disclosed. I would also not allow for plain copy & paste answers. So critical that appropriate guardrails to counter abuse would be in place. But I definitely see scope for humans & AI to work together to answer some of the questions on SO, especially when it concerns programming related questions (where reproducible code would make them easy to verify) and questions that never received an adequate reply. I would also suggest to add a checkbox "GPT assisted answer" so that user that would not like to see this content could still disable this if they would like. But why not make use of the free knowledge of user paying for ChatGPT4 & those that use it to assist in helping to answer questions on SO? If I look at the questions I ever asked on Cross Validated / Stats Stackexchange there is only 8 out of 20 questions that ever received an adequate answer, 3 of which I provided myself. If I paste the remaining unanswered questions in ChatGPT4 I would have checked 83% of the answers as being mostly or entirely correct (i.e. I would have marked them as correct accepted answers). With a bit of extra work these answers could surely be improved a bit further & it would be valuable to have them as correct, vetted answers.

Also, let's not forget: all those that having Bing as their search engine now by default get a GPT4 produced answer to their web search queries. So I would think that perhaps for most answers GPT now at least provided some inspiration for any answer... The important part is just teaching users how to use this technology responsibly & provide clear guidance on how to use it, as well as to point out its limitations. Imho, instead of fighting AI, humans and AI should work together to lift their abilities to a new level and allow them to solve problems they couldn't solve before. In the same way that we don't expect mathematicians or physicists to solve everything with pencil and paper and stop using computers.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .