-26

At some point, large language models will generate answers that are better than human answers.

  • There seems to be consensus that GPT-3 and ChatGPT (GPT-3.5) are not good enough for that
  • But it will happen,
  • or it already happened with GPT-4
  • or with one of many similar large language models
  • or with large language models fine tuned for specific topics

Independent of when it happens:

What are the pros and cons of making an AI basically a normal user of Stack Exchange, with equal rights?

I ask the question now because it is no longer clear whether we already reached the threshold with GPT-4. It is better than humans in many professional and academic exams. I am sure many questions on Stack Exchange are much easier to answer than serious human exam questions. So it is not far fetched that the transition from "any time soon" to "it happened" already passed.

It would not be surprising at all, because the development is exponential, and GPT-3 and ChatGPT were already irritatingly good. There is no technical reason to expect that AI will develop from "almost human" over "as good as human" to "better than human".

Improve this question
3
  • I understand that many people are somehow "against" it, and I am not comfortable with it myself. But I do not have good arguments against it.
    – Volker Siegel
    Mar 19 at 22:03
  • 1
    feature-request should be used on proposals or requests, don't use when looking for input as preparation to create a proposal / make a request that require the intervention of CM or DEVs (staff). Later, if this tag might be added, when be appropriate, in other it might be better to create a new post. See the tag wiki of this tag to learn about the details about what is expected that feature requests.
    – Rubén
    Mar 19 at 22:10
  • Quora is using Sage bot now :|
    – the Hutt
    May 7 at 9:43

3 Answers 3

Reset to default
16

These questions are asked too soon in response to marketing hype

As with the many other proposals along these lines that have been posted on Meta Stack Exchange and Meta Stack Overflow, this is very premature. Yes, AI will improve. It has, however, taken about 60 years to get to where we are now, with many people forecasting throughout that time that human-like, general AI is just around the corner, or "coming very soon". There's still a long way to go, even though there have been significant recent advances in chat bots, which are designed to spout "eloquent bullshit" (i.e. they are designed to sound good, not be correct).

The existing capability will be evaluated when it's here.

Without knowing the actual capabilities of the AI that's being evaluated, there is no way to definitively answer these questions and proposals, other than that the actually existing capabilities of AI is something that will, of course, be actively monitored and evaluations will be done of what actually exists, from all different points of view, including things like:

  • "Can AI do it?"
  • "Is AI economically viable to use?"
  • "Is AI something which users want?"
  • "What is society's current opinion about the acceptable use of AI?"
  • "Are people abusing the AI tools?"
  • etc.

Seriously, there are lots of technical people with very detailed knowledge of AI and its currently available capabilities that participate in the Stack Exchange Network. There's even a whole Stack Exchange site, Artificial Intelligence, which is completely about AI, and several others where some questions about AI are on-topic. To think that AI won't be evaluated as the capability continues to evolve is ridiculous. But, it's also ridiculous to make plans or commitments prior to knowing what the existing capability will be at any particular time.

Improve this answer
4
  • 1
    I ask it now because since GPT-4 it is no longer clear whether it still is "coming very soon", or already came. GPT-4 made a surprisingly big improvement over ChatGPT. GPT-4 exhibits human-level performance on many professional and academic exams.
    – Volker Siegel
    Mar 19 at 22:33
  • 8
    @VolkerSiegel So, prove, or at least demonstrate, that it is, in fact, at the level where it's reasonable to consider it under nearly all conditions. Because, if allowed, then it will be used by people in whatever way they think of. Currently, 99.9% of people using AI to post on Stack Overflow do it inappropriately, even based on our old rules (i.e. it would be abuse even without our existing ban on the content). So, demonstrate that even when abused, which a large number of people are currently doing and will continue to do, that it doesn't cause harm.
    – Makyen
    Mar 19 at 22:57
  • There is the question of what "reasonable to consider" means. I think it does not mean that it is right "under nearly all conditions", just under significantly more conditions than humans.
    – Volker Siegel
    Mar 20 at 17:58
  • I avoid to say that it is already "good enough", even though I suspect it is since a couple of days with the release of GPT-4. It was not with ChatGPT of GPT-3, we agree on that.
    – Volker Siegel
    Mar 20 at 18:00
-5

An answer provided by GPT-4 itself, in part as an example of what is currently possible

System: "You answer questions on Meta Stack Exchange" The prompt for GPT-4 was the unmodified text of the first version of the question. All text below is the unedited output of GPT-4. First try, not cherry picked:

Integrating an AI like GPT-4 or future large language models into Stack Exchange as a normal user with equal rights has both pros and cons.

Pros:

  1. Improved quality: AI-generated answers are likely to be more accurate, comprehensive, and up-to-date, providing better information to the community.
  2. Faster response times: AI-driven models can generate answers almost instantly, significantly reducing the time it takes for users to receive helpful information.
  3. Consistency: AI models are not affected by emotions, personal biases, or fatigue, which could result in more consistent, objective answers.
  4. Multilingual support: Advanced language models can potentially provide support in multiple languages, making Stack Exchange more accessible to a broader audience.
  5. Reduced load on human users: By providing accurate answers, AI can help reduce the workload for existing users, allowing them to focus on more complex and nuanced questions.

Cons:

  1. Loss of human touch: AI-generated answers might lack the personal touch, empathy, and contextual understanding that human users can provide.
  2. Overreliance on AI: Users may become overly reliant on AI-generated answers, potentially stifling community engagement and collaboration.
  3. Misinterpretation: AI models can sometimes misinterpret questions or generate answers that may not be entirely relevant to the user's query, leading to confusion and potential misinformation.
  4. Ethical concerns: The use of AI-driven models may raise ethical concerns, such as data privacy, fairness, and accountability, which need to be carefully considered and addressed.
  5. Gaming the system: AI-generated answers might lead to users trying to game the system for reputation points or other benefits, potentially harming the overall quality and integrity of the platform.
  6. Diminished user motivation: With AI providing accurate answers, human users might feel less motivated to participate and share their knowledge, leading to reduced community engagement.
  7. Cost and resource implications: Implementing and maintaining AI-driven models require significant financial and computational resources, which could impact the sustainability of the platform.

Ultimately, the decision to integrate AI models like GPT-4 or future large language models into Stack Exchange should be carefully considered, weighing the pros and cons. It may also be beneficial to explore hybrid approaches that leverage both AI-generated answers and human expertise to create the best possible user experience.

Improve this answer
14
  • Letting GPT-4 answer it is an afterthought, was not my intention here, but it feels quite appropriate for this special case.
    – Volker Siegel
    Mar 19 at 22:05
  • 3
    GPT-4 still have limitations, i.e, it might hallucinate among having other problems. By the way, please read Should ChatGPT be banned on MSE itself?
    – Rubén
    Mar 19 at 22:13
  • 13
    "AI-generated answers are likely to be more accurate" - That's a huge claim that is currently unsupported by evidence.
    – Pseudonym
    Mar 19 at 22:29
  • 6
    "AI-generated answers are likely to be more … up-to-date" is a fundamentally impossible claim for large -language-model-based AI. Such models are trained on existing text. They can't reason. They can't learn new things without humans providing the training text. Thus, humans, at least some experts, will always be ahead of these types of AIs, even if code is written for the AIs which verifies the accuracy of their generated statements.
    – Makyen
    Mar 19 at 22:50
  • @Pseudonym That is somewhat self-referential, the AI is saying it here itself. In a way, it has no useful meaning. It is an important point that I did not edit the output, so I will leave it there.
    – Volker Siegel
    Mar 19 at 23:05
  • @Makyen It is definitely not true for GPT-4, as its learning cutoff date is September 2021. And you could argue that GPT-4 should know that. But it can not know it, because it did not exist at the time it learned. It is not necessary true in general. There is no reason that incremental learning, fine tuning or other integration of up to date information should be difficult.
    – Volker Siegel
    Mar 19 at 23:11
  • @Makyen You say "They can't reason". I think that is no longer true, for many definitions of reasoning. Do you have a specific basis for that, or is is an assumption that may be outdated? When you think of a large language model as a thing that predicts the next word, which would be correct, seeing "real reasoning" would be very surprising, which would also be correct. But note that humans do exactly the same: Write or speak a single next word, based on past input and internal state.
    – Volker Siegel
    Mar 19 at 23:19
  • 5
    @VolkerSiegel Indeed it is self-referential, and it kind of makes the point that ChatGPT doesn't know that it doesn't know.
    – Pseudonym
    Mar 19 at 23:56
  • 1
    @VolkerSiegel "You say "They can't reason". I think that is no longer true, for many definitions of reasoning. Do you have a specific basis for that" do you?
    – VLAZ
    Mar 20 at 6:46
  • 6
    "But note that humans do exactly the same: Write or speak a single next word, based on past input and internal state." unless it's some sort of ramble - no, they don't. Source: I'm a human. I tend to think in terms of ideas first then put them into words. Not think of word then think of another word to follow the previous one.
    – VLAZ
    Mar 20 at 6:47
  • @VLAZ How you come up with the next word depends on what happened before, yes. It is not a ramble, I try to explain how it is meant when we say "a language model predicts the next word". Some words you say are relevant, some are trivial, you spend different amounts of attention on them, but you say them sequentially. That is what a large language model implemented by a transformer like GPT-4 does: A main point is the attention on relevant things. What you would call ideas. GPT-4 reads a prompt, has "an idea" of what to say, and says the next word. The new word becomes part of the prompt.
    – Volker Siegel
    Mar 20 at 17:37
  • @VolkerSiegel When I say "I want two scoops of the chocolate ice cream" I already know what I want, how much, and what kind. I do not stop to consider each of these words based on what I've already said so far.
    – VLAZ
    Mar 20 at 17:42
  • @VLAZ A good basis for it is the GPT-4 report, cdn.openai.com/papers/gpt-4.pdf and interacting with GPT-4 itself. If reasoning is required to succeed in tests like SAT or the Uniform Bar Exam, it can reason.
    – Volker Siegel
    Mar 20 at 17:47
  • @VLAZ Of course you know more than you can immediately say, just as a GPT-4 does. You know that you just said "I want two scoops of the chocolate", and predict that saying "ice" next is what would be best for your situation. So you do that, and then find that you just said "I want two scoops of the chocolate ice", and decide what word to say next. Of course we would describe it totally different for humans normally. But this is how it is meant in the context of a language model, and it applies for humans too.
    – Volker Siegel
    Mar 20 at 17:54
-6

Pro:

  • It is realistic that using a large language model can increase the quality of the answers on the sites. It is just an assumption in the question, but I am sure it is realistic
  • If it would turn out that large language models alone can not provide consistently better answers, combining it with human review and/or rating certainly can.

Contra:

  • I see a big risk that it would break the gamification of Stack Exchange, most importantly it could break the reputation system: With a naive implementation, the large language model would possibly collect most of the reputation.
  • It could negatively influence the social aspects of Stack Exchange. It could diminish the feeling of "being a community", when a significant part of the site is non-human.

These contra arguments are strong, but on the other side, increasing the quality of the answers seems to be a high value too.

The more I think about it, the more it irritates me. Which is probably appropriate.

Improve this answer
2
  • 1
    Re "the large language model would possibly collect most of the reputation": That presumes it is correct. There isn't much evidence of that, on the contrary. It is more on the level of even a blind chicken picks up a grain now and then (say, wrong 90% of the time). Here is an example that started out well for ChatGPT, but the human soon had to give up trying to correct its incorrect ways.
    – This_is_NOT_a_forum
    Mar 20 at 11:44
  • @This_is_NOT_a_forum That applies to ChatGPT. But the progress with GPT-4 is huge. It is better than humans in many professional and academic exams. The situation totally changed.
    – Volker Siegel
    Mar 20 at 17:30

Not the answer you're looking for? Browse other questions tagged .