-20

This question is meant to be constructive in tone, not another channel to start debates or express opinions or frustrations on what happened around AI in the past weeks. (There are plenty, more appropriate questions to do that.)


In GPT on the platform: Data, actions, and outcomes Philippe concludes

At the end of the day, it all boils down to this: We have to walk the middle path.

I've been trying to picture what this means and where we could go from here. I try to take into consideration that SE, Inc. and the community of users have different, but also many common interests. Very briefly (and a bit contrived) these interests can be summarized by: SE wants (needs) traffic, users want quality answers that are easy to find, they both want SE to continue to be a unique Q&A site.

It seems likely that SE is going to experiment with system-generated answers to questions.* If so, then maybe is this the area where we can find middle ground? Can it be implemented such that

  • SE gets its traffic.
  • The community can keep dealing with human answers largely the way it was used to, the way that was developed and tried in many years of consensus-driven community moderation.
  • We all benefit?

Can we achieve these goals? I think it's possible if we make these answers visible and for the rest as little disruptive as possible. What follows is an attempt at an implementation plan that meets these conditions.

  • The answer is clearly marked as generated.
  • The answer should be generated after (at least) 5 minutes, i.e. the time revision history will start recording. - On second thoughts, this should last much longer, otherwise posting poor questions will be too rewarding. It would almost be a shortcut to using ChatGPT directly.
  • It is signed by a new type of community user or marked Community Wiki (CW).
  • The answer can be voted on/accepted. Downvotes have the usual reputation penalty.
  • As long as the answer is not accepted, the question will show as unanswered even if the answer receives votes. This is important because it will not change the workflow of people looking for "unanswered" questions.
  • Only human answers contribute to the answers count. This is important for people who were used to looking for questions that have zero answers.
  • Posting the generated answer doesn't bump the question nor change its activity status. This will preserve the current front page experience.
  • Generated answers can't be edited. This allows learning models to exclude their own pure answers and thus prevents them from confirming their own bias.
  • Generated answers can't be flagged and/or deleted. I.e. except for voting, they are out of the community's hands, and fully under SE's control.
  • We can't comment on generated answers. The constructive way to point out any flaws is to post a better answer.
  • The generated answer will always be on top in any sort mode (the sort options box is below it), therefore...
  • A generated answer can be collapsed to help answerers focus on the question alone and other readers to focus on human answers. It will keep its collapsed status when sorting is changed.

For human answers:

  • The existence of a generated answer may make it less tempting to post another solely generated answer.
  • Network policies regarding the use of AI in human answers (SE-wide and network-specific) are fully determined by community consensus (these policies, what they are now and how they should develop, is not a subject of discussion here).
  • Community moderation works as usual.

Possible consequences:

  • This restrains a question sooner than we are used to. Normally, a question shouldn't be changed essentially once it has received answers (then revisions should only make the question more answerable), so users will have less time to change the question considerably. Or the generated answer should be regenerated 5 minutes after the question was edited.
  • Answering questions will become harder because (serious) answerers will always have to look at the generated answer as well as, obviously, the question.

So am I missing anything (very likely!)? Am I hallucinating? Are there better, or additional ways to find middle ground? AI is a fact. We have to find ways to integrate it naturally and helpfully into our everyday lives. Please contribute in finding ways how to do this on our SE platform.


*Not that I'd like it, but let's suppose it's inevitably going to happen.

37
  • 9
    I wouldn't trust keeping things 'entirely' under SE's control + it dosen't solve the problem of 'naively' posting ChatGPT answers and thinking its ok Commented Jul 8, 2023 at 12:51
  • 6
    Few other points - I'd say a generated answer should be subject to existing quality rules - including comments and votes. That feedback can be useful in deciding if to try an answer. It would be nice to be able to use said feeback to improve generated answers too Commented Jul 8, 2023 at 12:56
  • 1
    The best way is to let the elected mods to determine if it is GPT post. If yes and not marked, then deleted. If it is the second offence, suspend the user.
    – Nobody
    Commented Jul 8, 2023 at 12:56
  • 11
    This suggestion seems a bit like the Quora model, where the bot sits at the top of the list of answers. Do we really wan to be more like Quora? i.sstatic.net/aaDdd.png
    – Tetsujin
    Commented Jul 8, 2023 at 12:57
  • 1
    @Nobody I'd assume these are system generated :D Commented Jul 8, 2023 at 12:57
  • 1
    @Nobody Yes, that's the idea. Commented Jul 8, 2023 at 13:00
  • 1
    @Nobody I'd say any 'non human' generated content should be clearly labelled Commented Jul 8, 2023 at 13:01
  • 3
    @GertArnoldisonstrike - tbh, I'm not much of a fan of Quora anyway& very rarely go there. There seems to be little quality assurance on there. I've spotted several 'large answerers' sound remarkably like ChatGPT themselves;) I just noticed this new bot had sprung up at the top recently.
    – Tetsujin
    Commented Jul 8, 2023 at 13:03
  • 5
    "AI is a fact. We have to find ways to integrate it naturally and helpfully into our everyday lives." We don't.
    – JonathanZ
    Commented Jul 8, 2023 at 14:33
  • 1
    Id say having SE generate a dedicated AI answer for every question is probably not that useful, as most likely it wont be accurate anyway (based on the outputs generated by current popular ai tools). More helpful options would be human user, who want to do so, using AI as a tool to help in generating their answers. May be they can use AI tools to generate a first draft of heir answers and edit out the mistakes. Does said AI tool have to be built into SE? Not really. The interested users can directly go to chatgpt (or any AI generator of their choice) or use a possible future SE inhouse AI tool
    – user13267
    Commented Jul 8, 2023 at 15:37
  • 1
    regarding your last paragraph, I think "integrating AI into the SE platform" is not the important issue. What should stop is the misguided aggression some people have here against any use of AI at all, who think responsible use of AI to help generate posts should not be allowed for users who want to do so, and go around downvoting anything that doesn't outright say all AI use must be banned.
    – user13267
    Commented Jul 8, 2023 at 15:50
  • 3
    "AI is a fact." If humankind ever develops a true artificial intelligence, it will revolutionize our society in ways we can't even yet imagine. It will revolutionize our philosophy, the way we view the world around us (through our eyes versus through an AI's eyes), how we define reality itself (and who/what defines it). We will become equal to the Creator of the universe - atleast in one aspect, and we will have to decide if we are worthy and ready. Do we have the right to 're-program' an AI? To power it off? Have we developed the ethics to know the answers to those kind of deep questions?
    – ouflak
    Commented Jul 8, 2023 at 18:39
  • 7
    True Artificial Intelligence is distant speculative dream that may never happen. It might not even be possible. It is not a 'fact'.
    – ouflak
    Commented Jul 8, 2023 at 18:39
  • 9
    "AI is a fact" Yea, not yet, it ain't. What we're talking about here are pre-trained transformers. They're not "intelligent", they're not "creative", they're not useful in SE's question / answer format (At least not in any form we've seen up until now)
    – Cerbrus
    Commented Jul 8, 2023 at 19:07
  • 2
    @user13267 I am saying that your solution is unrealistic because there is not enough people that would be able to moderate that additional content and there will be much more such content because it is easier to produce AI posts than write your own. Commented Jul 9, 2023 at 18:11

7 Answers 7

1

Generally speaking, I think the ideas you have proposed are not terrible. They're pretty sensible. This is not to say I think that it's a good idea in general to execute, but that I think you've got a commendable coverage of thought into how to not execute it massively terribly, if it were to be executed. That being said, I don't think you've considered everything, and have missed some pretty important considerations, which I'll touch on along with some misc comments:

  • Generated answers can't be edited. This allows learning models to exclude their own pure answers and thus prevents them from confirming their own bias.

And what about when whatever model is used is updated in any way such that the ways it responds changes, such as a new and improved version coming out? What will happen to the answer? Edit it? And what will happen to the votes? What if the new answer is dramatically different than the previous one? Reset the votes? Post it as a new answer? (I already anticipated brought up this issue way back when the blogpost dump happened).

Add on to that that the world moves and grows. People already complain loudly about outdated content and information, and it's a real pain point. What should happen when an AI-generated answer becomes outdated? Generate another one as a new, separate answer post? I for one dislike that idea. But what else, then? Nuke the old AI answer and rewrite it? What should happen to the votes on it? How would an outdated AI-generated post even be detected?

What about non-deterministic models like ChatGPT? Do we just give a temperature of zero? If not, who decides which answer gets used? What if the answer generated the first time is not the best answer the model could give?

  • The existence of a generated answer may make it less tempting to post another solely generated answer.

But sadly, there will still be those that have low morals and will break the authentic usage policy to try to farm rep without any real work.

  • This restrains a question sooner than we are used to. Normally, a question shouldn't be changed essentially once it has received answers (then revisions should only make the question more answerable), so users will have less time to change the question considerably. Or the generated answer should be regenerated 5 minutes after the question was edited.

I assume this would most often happen for questions missing details. In seriousness, one could actually consider throwing another model into the question asking process to guide on question improvement (which I suggested all the way back on Dec 4th, though I don't necessarily think its the best, simplest, or most robust approach to solve the problem).

  • Answering questions will become harder because (serious) answerers will always have to look at the generated answer as well as, obviously, the question.

Nah. The only thing I'd look at the generated answer for is to see if it gave the answer I'd give, in as efficient a way as I could give it. That shouldn't be too hard.

1
  • Thanks for your thoughts on this and understanding what I was up to! And what about when whatever model is used is update - anyone is free to improve their answer always. But I wonder if SE would re-generate thousands of answers. Anyway, it's up to them. As for making it less tempting for users to generate answers, yes, they'll keep doing it. As I realized later, it may even get worse ("If they do it, I can do it"). I'm all for using AI to help users use the site better. As we know, the first efforts weren't that successful, but I think that's were they should their efforts on. Commented Jul 11, 2023 at 6:55
8

Well, the middle part would involve finding a compromise to start with.

I'd start by saying between big firms scraping whatever they want, and a certain lack of success with generative AI assisted question writing—I feel like Stack Overflow's strategy is a little flawed. No one's going to pay for what they can simply take elsewhere, and a bit more differentiation would be necessary IMO to avoid embarrassment, let alone succeed

As a former management major, maybe it’s worth looking at Stack Exchange's Strengths, Opportunities, Weaknesses and Threats (AKA a SWOT analysis)

Strengths -

I'd argue in a period where folks are sometimes finding value in generative AI, but also frustrated, keeping the core of the site about quality, human-curated content you can trust is a strength. Our communities have consistently put in the work to keep these things going, and even SE's own blog admits that language models often don't know what they are talking about. Collectively we do. We Think, and We know things. It could even be argued that the data we produced has extrinsic value - there's a lovely article by Bruce Schneier that suggests a model for this as a form of a dividend. We don't expect cash, we just want to preserve that value.

Weaknesses -

I'd say there are two weaknesses—that we're losing experts, and other engaged users, and we're not gaining new users fast enough. I'd note many of SE's attempts to do the latter, end up alienating the former.

Opportunities -

Considering the fear of 'Oh, they're going to scrape our data, the Opportunity here is that we're the source. It’s an unfiltered, unhallucinated source of information. Someone wrote it and they can back it up. We can build on a reputation for quality and reliability.

Threats—that people go to other sources of information like ChatGPT—or we'll face the end that the Hyphen Site had, and go out of relevance, grasping like a drowning man for every last possible source of revenue.

I'd also say that—Stack Overflow Inc can't compete, IMO with organisations that have multibillion dollar venture capitalist budgets like OpenAI or Google playing their game. The Formatting assistant was a thin frontend over OpenAI's software.

Now that I've been all negative, let’s consider a few things

  • Good interesting questions are novel and need some degree of knowledge. This isn't 'really' what generative AI is good at. There's also no point adding 'additional' ChatGPT answers to a question with existing answers

  • If you can make up a correct answer, it’s already there and someone's answered it. If not, maybe having a generative AI answer and having users vote on it might be of some value. In which case having some idea of its sources would be nice.

    However, the existence of an organic answer would remove it—maybe with a positive score on that answer, or outscoring the generative AI one.

  • The complaints of many is that the community is unfriendly, and yet the network is an important enough resource that they keep trying to get it changed from the outside

  • If ChatGPT and other tools are not free or their restrictions and data incest is sufficiently bad, they lose value, and ChatGPT answers pollute our sites. Here's an article on that

  • People with bigger wallets are going to outperform a company that's smaller and well, less well resourced. SE isn't going to win the current trend by following everyone else.

I think a successful strategy looks at current end user pain points and not the latest fads.

Looking at all this—a useful use of generative AI would not be a tool that posts answers. It’s one that takes a question, sorts through existing answers and gives users the information they need—basically SE without the need to deal with the community. Don't reprocess the answers any more than that's needed. Use the feedback to do better search. Basically don't make up an answer. Give them existing answers across the site. Maybe explain why the system 'thinks' it’s a suitable answer (if possible).

Failing which, it walks a user through the question creation process to maximise chances of success. It labels the question as such. It would be nice to use comments to encourage the poster to edit things. It tells them their work looks like homework and tells them to ask their TA or lecturer, and uses other tools to 'fix' or encourage fixing common pitfalls like images of code.

It clearly states where the source is, and that its tool assisted

It’s also something novel. It’s not Copilot, which writes your code for you. It’s not ChatGPT which sometimes lies to you. It’s a tool for helping you use this site.

It even outsources our babysitting newbies a little.

8
  • Thanks for this rich and cooperative answer. Interesting links there! However the existence of an organic answer would remove it is worth considering. Its a tool for helping you use this site. Yes, let's do that too. Commented Jul 8, 2023 at 13:27
  • Alas quite a lot isn't up to me 😁 Commented Jul 8, 2023 at 13:27
  • 1
    Wouldn't you say the weakness is that the model requires enough traffic from new users and enough regular users who know how the site works? And not being able to maintain this balance, especially in the face of AI competition, is a threat?
    – tripleee
    Commented Jul 9, 2023 at 8:29
  • 1
    Another weakness is that the model appears to be unsustainable on its own (even with advertising) and so a threat is that the owners will be under pressure to divert attention to whatever robber baron tactics their investors think are sexy this quarter.
    – tripleee
    Commented Jul 9, 2023 at 8:44
  • 1
    Possibly! But I feel like to some extent, efforts have focused more heavily on 'attracting' new users than retention. And I was thinking about the sites more than the company since parts of the company work in very mysterious ways. Also I really don't want to go out and say the company is a threat to itself in this context from where I am standing :D Commented Jul 9, 2023 at 8:47
  • "data incest" may not make sense without reading the reference. "Data incest arises due to multiple usage of identical information as if it were independent information." Commented Jul 9, 2023 at 11:36
  • your proposal misses to touch on how the "summarized info" will be voted on. A solution which will not point back to the answers which the summary is comprised of and which makes it impossible to vote on those answers would not be a good solution I think.
    – starball
    Commented Jul 12, 2023 at 7:14
  • Cause I don't think that's a good idea at all. Gen AI isn't the correct answer - but rather refocusing on the fact that we're community software designed for humans. Commented Jul 12, 2023 at 7:17
6

The answer is clearly marked as generated (by Stack Exchange).

The bloated bulge of generated content has already overwhelmed what shrinking number of such volunteers we have. So problem not solved.

The answer should be generated after (at least) 5 minutes, i.e. the time revision history will start recording.

I'd say 12 hours. Minimum.

It is signed by a new type of community user or marked Community Wiki (CW).

But what if the answer is gibberish, a hallucination, or even dangerously incorrect? Should that really be marked as a CW? And what about plagiarism? Do we really want that material marked CW?

The answer can be voted on/accepted. Downvotes have the usual reputation penalty.

I may have misunderstood your earlier point. But if it's community, does reputation matter?

Posting the generated answer doesn't bump the question nor change its activity status. This will preserve the current front page experience.

Does the 'front page experience' really mean that much? I don't know myself. There have been meta posts about it, so I'm sure it's important to some here. I'm just not sure it's the audience that comes here looking for a quick answer.

Generated answers can't be edited.

Even if a minor edit is the only thing that solves a critical problem with an otherwise high quality post?

This allows learning models to exclude their own pure answers and thus prevents them from confirming their own bias.

Is this really SE's problem? Even if they were to accept your proposal outright, I'm not sure Stack Exchange wants to really have to cater to chatbot LLMs... unless they are getting a cut of the action. Then that's different.

Generated answers can't be flagged and/or deleted. I.e. except for voting, they are out of the community's hands, they fully under SE's control.

But one of the core facets of our Charter is to be a repository of knowledge. If we allow egregious amounts of almost always entirely incorrect, untested, uncuratable (assuming the earlier parts of your proposal are enacted) postings to flood the site... well what are we here for? Didn't SE just have a round of layoffs? Do they have the staff and domain knowledge to go through and do what's right now being done by volunteers?

We can't comment on generated answers. The constructive way to point out any flaws is to post a better answer.

It probably wouldn't matter. The system that posted the generated answer is not truly artificially intelligent and would very likely not have the domain knowledge to respond to any useful comments anyway.

The generated answer will always be on top in any sort mode (the sort options box is below it), therefore...

Any particular reason? I am not asking critically (it doesn't really matter to me where the content would go), but I'm not sure of the point of topping it, bottoming it or anything like that. I guess I could see grouping such content together though. Is that related to...

A(ll) generated answer(s) can be collapsed.

Which makes sense I guess.

Some issues which I think you still need to address with your proposal (unless I missed it).

  • Problem 1. Large amounts of generated posts by the system that need moderation. Even generated content can sometimes violate the CoC, plagiarize without attribution (or false attribution), be politically incorrect, and so on....

  • Problem 2. A worryingly low percentage of these generated answers are correct, far lower than the equivalent human average.

  • Problem 3. Whether generated or human, correct or incorrect, all such posts need to be reviewed, and often need active curation. And the fact is that we've got a shrinking volunteer workforce on that front.

  • Problem 4. On some sites on the network in particular, but really any site depending on the context, allowing deceptively well-formatted incorrect information (that you say can't be deleted!) can be dangerous and ruin the lives of anyone deceived into thinking that it is an 'answer'. How does your proposal address this problem?

10
  • It seems I didn't make myself clear enough that it's about system-generated answers. Commented Jul 8, 2023 at 13:04
  • @GertArnoldisonstrike, Wait. You mean Stack Exchange generates loads of answers? Well that's a whole different thing altogether. How many answers does SE decide to post to the same question? Do they limit themselves? How do they know which answer is right? How do they know it isn't a hallucination? What about topics where the answer could be dangerously misleading if your answer is wrong? Sorry if I missed that critical point. I may have a go at rewriting my answer (something that couldn't be done if it was generated - as there would be no comments allowed).
    – ouflak
    Commented Jul 8, 2023 at 13:09
  • Even generated content can sometimes violate the CoC ... Yes, I'm struggling with that a bit. Maybe we should be able to flag for that, but I'm trying to find a way that keeps it all away from moderators, Maybe something like "report this answer" similar to the current Adds proposal, so it'll be up to SE to deal with violations they cause themselves. As for the expected poor quality of the answers, I agree. That's why I want them to be clearly marked and ignorable and again, something for SE to deal with. But yeah, it's a problem. Commented Jul 8, 2023 at 13:54
  • How many answers does SE decide to post to the same question? Just one, maybe overwritten after editing. No retroactively, by the way. Commented Jul 8, 2023 at 14:14
  • I'd say 12 hours. Minimum. Yes, rethinking this, probably far longer than 5 minutes, otherwise posting poor questions will be far too rewarding. Commented Jul 8, 2023 at 14:25
  • @GertArnoldisonstrike, The highest scoring answer I have on StackOverflow is actually the second highest voted answer on that question. How does SE know how many times to hit the 'Generate' button?
    – ouflak
    Commented Jul 8, 2023 at 14:46
  • Sorry, I don't understand your question. I didn't intend to say that what happens to human answers is related to SE regenerating answers, I seem to have suggested there is some connection there. Commented Jul 8, 2023 at 15:02
  • @GertArnoldisonstrike, Many questions can have several good answers. A StackExchange staffer could hit the 'Generate' button once, and that might generate one good answer. But what if there are multiple possible good answers? How many times does the SE staffer click the 'Generate' button on a question? How would they even recognize that they should? And how do they know if they've really got some unique solutions that fit the different possible scenarios?
    – ouflak
    Commented Jul 8, 2023 at 18:09
  • @GertArnoldisonstrike, "Just one, maybe overwritten after editing. No retroactively, by the way. " Then there is no point in topping it (or positioning it anywhere). If it's just a single incorrect (very likely) answer, does it matter where it is if it is alongside multiple human answers, or no other answers?
    – ouflak
    Commented Jul 8, 2023 at 18:21
  • Let me clarify that I don't propose generated answers here. As for me it's not going to happen, but I think it's inevitable that SE will do something along those lines in the near future. I tried to find ways to make that as unobtrusive as possible. To be clear: it should be 1 answer per question. No staffer is going to generate answers. Positioning them at the top is an effort to give SE what they want. Many of the other points are aimed at letting the community do what it's good at with the least possible amount of disturbance by generated answers. Commented Jul 8, 2023 at 18:54
5

I'm not convinced this is financially feasible in the first place.

OpenAI reportedly pays $700k per day to host on the order of 1,5 billion visits per month.

ChatGPT Plus is currently priced at $20 per month which gives you access to GPT-4 and priority in the user queue, though still not unlimited access to either GPT-3.5 or GPT-4.

Stack Overflow currently gets around 3,200 questions per day (all of Stack Exchange 5k, give or take?)

Parenthetically, Stack Overflow for Teams currently costs $13.50 per user per month for the Business tier.

Now, on to speculations.

Even if we assume 90% margin, that's more than $1 per user per month in operating costs for Stack Exchange. We are imagining that each paying user also bears the cost of 9 non-paying users of the public site.

OpenAI is running an expensive gamble. They probably won't be offering a free tier forever, even if they can reduce the costs drastically. They have certainly attracted a large number of paying customers already, and will probably be tweaking down the limits for free access, or even disable the free demo once they have enough paying users to be able to rely on other types of advertising to fund further growth.

As a quick back-of-the-envelope calculation, let's assume that OpenAI would offer Stack Exchange 5000 answers per day for $15,000 per month (that's $500 per day).

It is not altogether a bad deal, but is it really worth it, given the dubious quality of ChatGPT answers, and how would you prevent abuse? Especially if free access to OpenAI gets more restricted over time, you would expect some users to discover ways to use Stack Exchange simply to get free access to ChatGPT going forward. Question limits for new users would probably have to be tweaked down, and staff and moderators would have to divert some of their attention to hunting down users who manage to abuse the system via bot accounts, creative prompts, etc.

Personally, I would not proceed based on these numbers. They are obviously very much a shot in the dark, but the real beef here is that LLMs at scale are not free, and the quality of the current generation of solutions is not good enough to justify the cost (even if it was cheaper than I'm guessing) and additional hassles.

3
  • OK, that's mainly about whether to introduce system-generated answers or not. I don't think we have much to say in that. I share your worries about the influx of users trying to abuse it. That would make it harder for regular users to get due attention and answerers would have to wade through more garbage. I can only hope it's never going to happen. After all, people who want generated answers can find them elsewhere. Commented Jul 11, 2023 at 7:04
  • The calculation in starball's question lands on a lower cost estimate for ChatGPT, though still on the order of hundreds of dollars per day.
    – tripleee
    Commented Jul 14, 2023 at 8:40
3

I have a couple of questions which you seemingly failed to address in your proposal:

  1. What is the reason for storing unhelpful answers? Shouldn't they be deleted, to not propagate false information, if they are just plain wrong and heavily downvoted? Or if the answer is not a proper answer, why shouldn't moderators have a chance to delete it?
  2. Why cannot the community explain errors in generated answers, expand some nuances, or provide alternatives through comments?
  3. Some tags, on SO for example, have no reason for such answers to exist. Time and time again it was proven that ChatGPT can successfully answer a question only if it was previously asked many times. But this question should be closed, and not answered, even by a bot. For novel hard questions in that tag, ChatGPT generates consistent crap. Its attempts are as good as asking a three year old to write a regex.

Some additional info provided in chat when I tried to gather reasons why "automatic answers" is not the way to go (I'm sorry for not providing credit at the moment, I'll do my best to find those in a couple of days, once I'm back home):

  1. There is no such thing as "the answer from ChatGPT". It changes every time on generation. How will an autogenerated answer, especially provided with delay, stop users from FGITWing their own answers from GPT (maybe with their own personal take on prompt modification)?
  2. (For a somewhat similar idea, but with additional quality check queue) we don't have enough SMEs as it is. Adding the burden of validating at least one answer for every question is unsustainable for the current model.
    I believe this can be similarly applied to your idea: we have problems with voting as it is. Even bad answers get a couple of up votes. We don't have enough active downvoters to reasonably apply quality gate to those autoanswers based on votes. Additionally they always look good, and will attract up votes from passers by solely based on that fact.

Some more of my thoughts; I for one am not interested in spending my time curating garbage produced by unsupervised LLMs. I'm sure many active curators will agree. Also it might become the last straw for many.

I obviously don't know how much, but let's assume 30% of the currently striking users dislike this idea. Are you sure repelling 500 active curators is an acceptable cost for introducing your idea?

3
  • Let me emphasize that these auto-generated answers are not my proposition, not at all. I only expect SE to start experimenting with them and I tried to proactively think of how that could be fit in. I'm against storing unhelpful answers but I'm afraid it's going to happen in one way or another. Commented Jul 8, 2023 at 21:15
  • So if it's going to happen, I'd prefer it to be something that we can ignore as much as possible. Commented Jul 8, 2023 at 21:17
  • I'm genuinely interested in reasons why "automatic answer" is not the way to go. I've read rumors that they would come but couldn't find them either in the vast volume of comment that were posted the past few weeks. Commented Jul 8, 2023 at 21:25
2

I'd want to add:

  • If a non-accepted AI-generated answer score drops below zero, it is immediately auto-deleted.

One of the main problems is the moderation burden; this is how Reddit bots are treated (except for the "accepted" part).

But honestly, I don't think the community is ready to accept AI answers, and AI answers aren't quite up to scratch. I recall a time when it was embarrassing to admit you learned something from the *gasps for dramatic effect* Internet. Then the same thing happened with Wikipedia. Now it's AI's turn.

If you're going to trial something like this, may I suggest doing so at language-related sites first. LLMs are, after all, particularly good at languages. And it's not the huge burden to check AI answers to language-related questions.

Maybe Stack Exchange needs an in-house LLM, which actually understands the specific needs of Stack Exchange.

3
  • If you're going to trial something like this, may I suggest doing so at language-related sites first. That may be a good idea (if it's going to happen). Of course we have to hope someone up there is listening. Commented Jul 9, 2023 at 17:43
  • 2
    LLMs can generate a diverse collection of example sentences much faster than humans and of better quality than through web crawling or databases (ultra-narrow context; not suited to language learners). They're good at tedious questions like "introduce mathematics vocabulary in Chinese" (verifying ChatGPT's answer = minutes; writing your own answer = hours) or questions which require understanding nuance in two languages e.g. "is there an idiom which means [insert English idiom]?" If an AI answer-bot can't survive at language-learning sites, then it can't survive anywhere on the network. Commented Jul 10, 2023 at 0:12
  • auto-deleted and then what? response regenerated? would the downvote cost rep? would it contribute to the daily vote count?
    – starball
    Commented Jul 11, 2023 at 0:18
-3

Generated answers are fine, as long as they've been curated and checked by the expert user who is posting them.

Simply put, there's nothing wrong with an expert using AI-generated text as a starting point for writing their own answer. They can use the AI to generate a baseline so that they're no longer working from a blank page, and then edit it for accuracy. By doing so, they combine the strengths of the AI (rapidly-generated text drafts) with the strengths of an expert human user (domain knowledge).

As such, your suggested solution is an inferior solution, since it's cutting out the input and supervision of the human expert user. Human plus AI is strictly better than human or AI alone.

If anything, there should just be a "generate draft answer" button that produces AI-generated text built into the page, that then requires the user to spend a minimum amount of time vetting and editing the answer for accuracy.

There's no need to specially mark them or reduce the reputation the user earns from them, because, ultimately, the AI is just a tool to assist the expert human user in writing answers more easily.

8
  • 2
    It seems I didn't make myself clear enough that it's about a system-generated answer to a question. For the rest I agree that there's nothing wrong with using AI to generate a first draft of an answer. I'm not sure if there should even be a built-in function for that. Commented Jul 8, 2023 at 13:17
  • @GertArnoldisonstrike Yes, I know. I'm saying that your solution is inferior to allowing users to generate and curate AI drafts on their own, perhaps with the system facilitating this.
    – nick012000
    Commented Jul 8, 2023 at 13:20
  • Sure, I agree. In my view the generated answer would be a clearly separate entity leading its own life. Around it, it's business as usual, which may include allowing users to generate and curate AI drafts on their own. So it's not either-or. I try to imagine what it should look like given that SE is going to generate answers. Commented Jul 8, 2023 at 13:37
  • 6
    "that then requires the user to spend a minimum amount of time vetting and editing the answer for accuracy." What's to stop a user form generating 20 answers on 20 tabs, going out for lunch, and then submitting them?
    – Cerbrus
    Commented Jul 8, 2023 at 14:02
  • 1
    This is the only sensible answer. Just use AI for what it is, a tool to help find solutions. If you want to use it to generate posts, just use it responsibly and make sure what it generated/what you are posting, actually is correct and helps others
    – user13267
    Commented Jul 8, 2023 at 15:40
  • 6
    And from where are these experts going to be recruited to engage in this mammoth volunteer effort? And how are we going to get those experts, if we do somehow manage to attract them to volunteer their time, to stay and not burnout? We're talking about hundreds of answers being spewed across the sites every day. We were already hurting for mostly non-'expert' reviewers and curators even before the ChatBot LLM content onslaught. So are these experts just going to spring up from the ground at the wave of a wand or something?
    – ouflak
    Commented Jul 8, 2023 at 18:02
  • @ouflak "And from where are these experts going to be recruited to engage in this mammoth volunteer effort?" You. Me. Everyone on this site. AI should be integrated into every answer.
    – nick012000
    Commented Jul 9, 2023 at 1:20
  • 6
    @nick012000 That's the imagination, but is it realistic? We are struggling to get enough eyeballs on each new answer as it is.
    – tripleee
    Commented Jul 9, 2023 at 9:09

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .