Could ChatGPT be a viable way to answer people's questions?

Question

A few days ago, OpenAI, the people that brought you GPT-3 and DALL-E, launched a new product called ChatGPT. It can answer questions so accurately that it's legitimately scary. Even the most obscure topics known to man are answered swiftly by this bot. It's not 100% accurate, but it's shockingly accurate for an AI. To explain how accurate it is, someone posted on Twitter that they managed to take an SAT test with the bot and received a 1020, which is above the 50% percentile of all its test takers. People are calling this automated system as something that's better than Google.

My question is, could this be a way to accurately answer questions that are on any of the Stack Exchange sites? I entered a question from Stack Overflow relating to the NumPy Python module and it gave a pretty convincing response. The responses are professional, probably even better than similar sites such as Quora where the responses vary in quality.

Would a bot such as this one change the future of question and answer sites?

Answer 1

While the solutions often look correct to the untrained eye, they are frequently invalid in various subtle ways. As such they are actively harmful as they encourage the question poster to try things that simply don't work.

In one deleted answer the documented return value for COLUMNPROPERTY is either 1 or 2 but the chatGPT answer had code that was predicated on 3 being being returned, that code would never execute and therefore the answer didn't solve the question's issue. Worse than not working though was that it looked like it might be OK to someone unfamiliar with the possible return values of COLUMNPROPERTY.

When the question poster then comments that there is a problem with the answer, neither the answerer nor chatGPT is in a position to respond as neither understand the comment or what to fix in the answer to address it. I pointed out a number of such issues in my Stack Overflow answer

All known chatGPT answers on Stack Overflow have had to be deleted. When the post author didn't do it voluntarily, moderators have done it for them. Be warned that deliberately making work for moderators can result in suspension.

Having the same variable answer quality as Quora is not something to aspire to.

Note that using chatGPT to create answers on Stack Overflow is currently banned

Answer 2

I guess that a fundamental part of the question is what is the real value these answers bring. While in theory machine learning systems can string together plausible answers, a good chunk of what a good answer is involves experience and understanding. Many of these answers look good, but they are useless. One expects tasty cake, and finds frosting covered styrofoam.

Another thing is why are you here if your contributions are literally "type a question into a box, and blindly post the answer here". I've often talked of Stack Exchange as a commons, and essentially posting such answers is basically an act of vandalism. To borrow a term invented by Phillip K Dick, it’s kipple - useless detritus.

I'd argue that SATs, with a distinct, very focused domain and syllabus and a finite question bank, would also be different from Q&A where folks would have very arbitrary questions.

Basically, don't post solutions you didn't try or test yourself, or know for certain that they work, and something from a source like ChatGPT is suspect.

For factual questions – well, please confirm that well, the answer is factual, and not the fever dream of a hundred monkeys on a hundred typewriters.

Answer 3

On Unanswerable Questions

If you're not asking solely about answerable questions, and instead asking generally about all / the average question(s),

Ever heard of "garbage in, garbage out"?

This is like the "oh no- computers are learning to code and now they're going to take our jobs- oh wait- for the computer to solve a client's problem, the client will have to accurately describe what they want".

Maybe instead, people should try to build things to:

• detect unanswerable questions and ask the asker to clarify on points that need clarifying.

  I mean- what is ChatGPT going to do for a "can you help me?" question on Stack Overflow?

• detect images of text and transcribe them, images of code and format them as code blocks with the right language classifier, images of tables and format them as tables.

I think (at least some) Stack Exchange sites need those more than possibly-accurate answers to answerable questions.

On Highly Localized / Unnecessarily-Non-Generic Questions

Maybe instead, people should try to build things to un-localize / generic-inate highly localized questions. (Though personally, I'd still want such automation to be an assisance tool gated by human supervision instead of an unsupervised thing).

We have dedicated badges to incentivize editing posts (including questions) (editor, strunk and white, copy editor) and editing them along with answering them (explainer, refiner, illuminator) because many questions have room to be improved for the benefit of people other than the original asker. A chat AI sounds like it's designed to chat with and help one person. That's not what Stack Exchange is (although that's what many new users think it is, and that's what often happens).

I entered a question from Stack Overflow relating to the numpy Python module and it gave a pretty convincing response

What was the question? If it was one of the many "please help me figure out what is wrong with my code" questions...

Maybe we should leave it at that. Maybe leaving it at that is a good thing for everyone. Even if an AI could provide correct, quality answers to any posted question (... again, how do you answer an unanswerable question?), then the value it would provide to the Stack Exchange network (I think) would be to make it so questions that aren't on-topic or don't meet community guidelines could be easier asked and answered elsewhere.

On Constructive, Subjective Questions

Would a bot such as this one change the future of question and answer sites?

Yes and no? If we get some magic-bullet™ that can answer any* question (that is on-topic and meets community guidelines for topics that are a good fit for the Stack Exchange Q&A framework) on the spot, many of the Stack Exchange sites mostly wouldn't be needed anymore.

We're generally a repository of information in the form of fact-based, objective Q&A.

What such an AI by-definition could not replace, is answers to constructive subjective questions where answers are based on peoples' personal experiences. A human experience cannot be synthesized. They can be gathered, which is one of the things we do here.

On Questions that Don't Meet Other Community Guidelines

Take a look at this MSO discussion: "Should I answer questions that include images of code (or violate other guidelines)?" (the answer is "No. Those questions should be closed.")

I mean- sure. Anyone who wants can write an answer if it's an honest attempt to answer the question. But if the question does not meet community guidelines, it should be closed (unless it has enough historical value to justify keeping it, in which case it should be locked).

On Misc Other Questions

What's it going to do when you throw something completely new at it? Ex. question about a new programming language?

My Closing Unsolicited Personal Thoughts: Do AI belong here on SE?

I think the Stack Exchange network is a community of Q&A for people.

When I read the Stack Exchange tour page, Here's how I read it (with my personal-thought inserted in parentheses, and emphases added):

Stack Exchange is a network of 180 communities that are created and run by (human) experts and (human) enthusiasts like you who are passionate about a specific topic. We build libraries of high-quality questions and answers, focused on each community's area of expertise.

Why do I read it that way? To me, a chat AI is not an "expert". I'm not sure how to explain it well. To me, an expert actually knows what they're talking about and what they're saying. Does a chat AI really know what it's talking about and what it's saying? And it's not an "enthusiast" either. It's mechanical. It has no motive (don't come after me, sci-fi people).

In 2008, Jeff Atwood and Joel Spolsky launched Stack Overflow, a site where (human) programmers could help solve each other's problems. Online. For free. Since then, millions of people have jumped at the chance to help a (human) stranger. And thanks to those people like you, the Stack Exchange network has grown to include 180 different communities visited by over 100 million monthly unique (human) visitors.

Also, the idea of people getting rep for an answer (even if the answer is correct) they got by blindly copy-pasting from ChatGPT feel... wrong to me.

The help center page on reputation says:

Reputation is a rough measurement of how much the community trusts you; it is earned by convincing your peers that you know what you’re talking about. The more reputation you earn, the more privileges you gain and the more tools you'll have access to on the site.

If someone uses ChatGPT only the write an answer and doesn't personally verify its correctness, then that person didn't know what they were talking about, and the AI that generated the response didn't really know what it was talking about either.

Answer 4

"[ChatGPT] can answer questions so accurately that it's legitimately scary." It can also make things up, including citations to non-existent sources. Just say NO.

In response to something I wrote, it generated three paragraphs of politician-sounding pablum on a topic unrelated to my input (and the third paragraph was a repeat of the first).

Asked about the influence of Elvis Presley on the Sex Pistols, its response was 180° opposite from a public statement by Johnny Rotten.

It's been out less than a year, and I've already forgotten the other travesties I've seen. I consider it a minor improvement to an algorithm actually named "Travesty" from decades ago.

It has the potential to be a greater source of falsehoods than Facebook.

Answer 5

I would think there is definitely scope for AI to assist humans in providing good answers. Especially the latest version ChatGPT4 (with plugins like Wolfram Alpha for maths or ScholarAI or the web browser plugins to give sources) has very good performance quantified based on various benchmarks, e.g. ability to solve university entrance exams correctly (often top10% relative to avg. human) or maths ability. See my answer here giving some examples of how with the help of ChatGPT4 I was able to answer various SO questions, including ones that had never received an answer, despite being highly ranked and having been awarded 4 bounties before. Of course adequate advice on responsible AI use would need to be given, cf. the guidelines of many academic journals like PNAS on the use of generative AI. Especially for programming related questions I think allowing responsible AI usage could be beneficial. The guideline should then include that if ChatGPT or Github Copilot was used to develop code that reproducible code & a benchmark plus output should be given (confirming that the user verified the code works correctly, ChatGPT can't yet run code yet) & that use of ChatGPT should be disclosed & a reputable source should be given. That is very easy to check - much easier than telling if a given answer might have used ChatGPT if not disclosed. I would also not allow for plain copy & paste answers. So critical that appropriate guardrails to counter abuse would be in place. But I definitely see scope for humans & AI to work together to answer some of the questions on SO, especially when it concerns programming related questions (where reproducible code would make them easy to verify) and questions that never received an adequate reply. I would also suggest to add a checkbox "GPT assisted answer" so that user that would not like to see this content could still disable this if they would like. But why not make use of the free knowledge of user paying for ChatGPT4 & those that use it to assist in helping to answer questions on SO? If I look at the questions I ever asked on Cross Validated / Stats Stackexchange there is only 8 out of 20 questions that ever received an adequate answer, 3 of which I provided myself. If I paste the remaining unanswered questions in ChatGPT4 I would have checked 83% of the answers as being mostly or entirely correct (i.e. I would have marked them as correct accepted answers). With a bit of extra work these answers could surely be improved a bit further & it would be valuable to have them as correct, vetted answers.

Also, let's not forget: all those that having Bing as their search engine now by default get a GPT4 produced answer to their web search queries. So I would think that perhaps for most answers GPT now at least provided some inspiration for any answer... The important part is just teaching users how to use this technology responsibly & provide clear guidance on how to use it, as well as to point out its limitations. Imho, instead of fighting AI, humans and AI should work together to lift their abilities to a new level and allow them to solve problems they couldn't solve before. In the same way that we don't expect mathematicians or physicists to solve everything with pencil and paper and stop using computers.