New blog post from our CEO Prashanth: Community is the future of AI

Question

For posts following this one, see:

• An example of a generative AI tool: Suggesting better question titles
  • AI/ML Tool examples part 3 - Title-Drafting Assistant
  • Stack Overflow will be testing a title-drafting assistant, and we’d like your involvement
• AI/ML Tool examples part 2 - Extracting questions from Slack transcripts and finding similar/duplicate SO for Teams questions
• Stack Overflow will be experimenting with a question formatting assistant, and we would love for you to be involved

---

Today, we released a blog post from Stack Overflow CEO Prashanth Chandrasekar about AI and its intersection with communities. Because of the timely nature and the content of it, I’m reproducing it here for comments or questions from community members. You can find the original blog post here.

Senior members of the company staff will be reviewing everything here and will respond to questions, though it may be 24 hours or so before answers are ready, just because of the amount of interest that this is likely to get. Please leave any questions that you have below as individual answers.

Community is the future of AI

By Prashanth Chandrasekar

Throughout history, great thinkers have made predictions about how new technology would reshape the way in which humans work and live. With every paradigm shift, some jobs grow, some change, and some are lost. John Maynard Keynes wrote in 1930 that new technology meant humans would be working 30 hours a week or less, and that the main challenge would be what to do with all our free time. So far, predictions of this nature haven’t exactly come true. As new technology empowers us, we push ourselves to new heights and reach for previously unattainable goals.

Over nearly 15 years, Stack Overflow has built the largest online community for coders to exchange knowledge, a place where anyone with an internet connection can ask or answer questions, free of charge, and learn from their peers. Stack Overflow for Teams, our enterprise SaaS product, is trusted by over 15,000 organizations to serve as their internal knowledge bases. With the recent advent of dramatically improved artificial intelligence, many industries are wondering how technologies like ChatGPT will change their business. For software development, the answer seems more immediate than most. Even before the latest wave of AI, a third of the code being written on popular code repositories was authored by an AI assistant. 

Today, sophisticated chatbots, built on top of cutting edge large language models (LLM), can write functional code for a website based on nothing more than a photo of a rough sketch drawn on a napkin. They can answer complex queries about how to build apps, help users to debug errors, and translate between different languages and frameworks in minutes. At Stack Overflow, we’ve had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? How can our business adapt so that we continue to empower technologists to learn, share, and grow?

It’s worth reflecting on an important property of technological progress. The Jevons Paradox shows us that, as innovation allows us to do more, we settle on a new normal, moving the goal posts for what we expect of people and organizations, then competing to see who can find new ways to pull ahead of the pack. For knowledge work, as the cost of an action diminishes, we often do more of it. Abstracting away repetitive or tedious tasks frees technologists up to make new discoveries or progress innovation.

If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we’ll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them. We are enthusiastic about welcoming this next generation of developers and technologists, providing them with a community and with solutions, just as we have for the last 15 years. We’ve got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams and will have some exciting news to share this summer.

I’ve heard similar sentiments expressed recently by Microsoft founder Bill Gates, by Geoff Hinton, the godfather of the neural network approach that produced today’s AI revolution, and by Stephen Wolfram, a pioneer across computer science and mathematics. Each sees in today’s AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive. Given that, we want to help democratize knowledge about these new AI technologies, ensuring that they are accessible to all, so that no developers are left behind. 

I talk to developers of varying experience levels all of the time, and I’ve been hearing anecdotes of novice programmers building simple web apps with the help of AI. Most of these stories, however, don’t begin and end with an AI prompt. Rather, the AI provides a starting point and some initial momentum, and the human does additional research and learning to finish the job. The AI can debug some errors, but is stymied by others. It can suggest a good backend service, but often can’t solve all the points of friction that arise when integrating different services. And of course, when a problem is the result not of instructions from a machine, but human error, the best answers come from other people who have experienced the same issues. 

For more experienced programmers, AI will be an amplifier of their existing skill, making them more ambitious in their projects. The result, as Jevons would predict, is that they spend more time with AI, but also more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. They feel empowered to reach farther beyond their traditional skillset and to push the boundaries in terms of the kind of work they want to take on.

We are excited about what we can bring to the fast moving arena of generative AI. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will “hallucinate” facts and figures if they feel it fits the pattern of the answer a user seeks. Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. We want to help coders stay in the flow state, allowing them to create with the latest tools with the confidence that they will be able to document and understand the provenance, source, and context of the code being generated. 

Community and reputation will also continue to be core to our efforts. If AI models are powerful because they were trained on open source or publicly available code, we want to craft models that reward the users who contribute and keep the knowledge base we all rely on open and growing, ensuring we remain the top destination for knowledge on new technologies in the future.

AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. Allowing AI models to train on the data developers have created over the years, but not sharing the data and learnings from those models with the public in return, would lead to a tragedy of the commons. It might be in the self-interest of each developer to simply turn to the AI for a quick answer, but unless we all continue contributing knowledge back to a shared, public platform, we risk a world in which knowledge is centralized inside the black box of AI models that require users to pay in order to access their services. 

As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish.

Our thesis on community as the center of a safe, productive, and open future for AI also offers some exciting prospects for our business. Stack Overflow for Teams, our enterprise, private version of Stack Overflow, helps to power a community-driven knowledge base inside of 15K+ organizations like Box, Microsoft, and Liberty Mutual. Decades of institutional knowledge, shaped and curated by subject matter experts and experienced teams, allows the employees at these organizations to more easily collaborate, improving productivity and trust. 

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains: from new hires being able to onboard more quickly, to speed up developer workflows, as users are able to quickly ask questions and retrieve answers tapping into the company’s history, documentation and Q&A.

The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams, and they have energized everyone at our company. We’ll be working closely with our customers and community to find the right approach to this burgeoning new field and I’ve tasked a dedicated team to work full time on such GenAI applications. I’ll continue to share updates through channels such as my quarterly CEO blog, but I’ll be back in touch soon to announce something big on this topic. In the meantime, thank you to our community and customers for continuing to help us on our mission to empower the world to develop technology through collective knowledge.

Answer 1

I'm not quite as excited as Prashanth to be participating in training AI, I participate here because I like teaching people, and I'm not particularly impressed by the job ChatGPT is currently doing of that, despite all the existing material it's been trained on.

In particular, I've recently seen a new flavor of question here on sites like Biology.SE that fits the format: "ChatGPT told me XYZ, can you explain this to me?" This pattern isn't new, "My professor/teacher/textbook told me XYZ and I'm confused" is pretty common. The difference is, no one should be learning from ChatGPT right now. It's not capable of understanding, it's a method of generating carefully crafted BS that appears to be knowledgeable without actually having knowledge.

From generating citations to completely fake papers to thinking Romania is land-locked to lifting plagiarized text, right now one of the biggest challenges to everyone else is filtering and fact-checking this BS. While versions of AI-assisted code writing have been around for quite awhile, the latest iterations are something quite different and I fear the hype will drive expectations for productivity that are not achievable without creating massive problems and security risks where the people paying the consequences will not be the same as the people collecting the benefits.

I don't really get what "Community is the future of AI" is supposed to mean, but for now, I fear it means that Our Community is not being treated respectfully by purveyors and users of recent AI technology releases.

Answer 2

This entire announcement makes me question the CEO's understanding of what his company's platforms actually are. There's a reason AI has been banned on several of the sites. We're actively fighting it over on Stack Overflow. This worries me. Don't get me wrong; I love AI/ML; I find it highly interesting, and I do look forward to the development of it into the future, but it has no place on SE.

Honestly, my first impression was: "wtf?" This seems so detached from the reality at the SE sites, which is highly worrisome, for someone that should know the reality, and is in charge of their future.

Please do not add GenAI to Stack Overflow. You're free to experiment with it, but don't pollute Stack Overflow with it. We're not interested in fact-checking AI content; we are interested in generating the content ourselves.

layer a conversational interface on top of this wealth of information

But why...? This is a distraction. These sites exist to easily let us find answers to the questions we seek, not be bombarded with noise. What purpose does this serve?

I find this post very demotivating. I’m not sure if there’s any point in continuing to moderate and curate Stack Overflow. You have made me insecure about my efforts to SE. Are they lasting? Will my work be undone? I don’t know. And a simple «no» in response, is not enough to cure that lack of trust I have.

Other problems I take issue with:

• MSE is a second class citizen to the blog
• The CEO doesn't post this himself on MSE
• You don't link to the MSE discussion from the blog
• The MSE discussion is not visible in the sidebar on any of the other sites and Meta sites. As such, it's invisible, because there's no link to it from the blog.
• My comment on the blog linking to this MSE discussion takes forever to be accepted; many people will have missed it by the time it finally gets put there. Also, it's not my job to do that; it's yours. Resolved one day after being featured.

Answer 3

Do not even think about incorporating something like ChatGPT into this site, or anywhere on the network.

To do so would be a slap in the face of all of the real contributors and their hard work and diligence to ensure that the content that is being produced comes from actual subject matter experts and those knowledgeable in the field.

The attitude from the tech industry is that AI is this panacea that will solve problems or make things easier, and I have the impression that this perspective is not changed within Stack Overflow. This worries me; y'all seem eager to solve a problem that y'all have had next to no success in quantifying.

One cannot claim to be supportive of creative minds or individuals when these AI models are generated and built on top of so much unlicensed, unprivileged data. (This is why I have reservations on things like GitHub Copilot, for instance.) It really doesn't take a lot of snooping around to make this observation.

I will absolutely not consent to having my data processed in pursuit of any AI integration, and I'll be looking at what my options are to prevent that in the coming weeks, should you move forward with this approach.

Answer 4

feature-request

Please provide an official interpretation of the CEO's wording for regular Meta Stack Exchange users.

The above post, as it was originally intended for a different audience, uses a communication style that might be magnificent for the target audience and the publishing platform, but the "storytelling methodology" has resulted in the writer including a lot of stuff that looks to be irrelevant for this context and space.

AFAIK, most of Meta Stack Exchange's regular users aren't CEOs, aren't top decision makers handling advertising budgets, aren't finance market stock buyers, aren't top-notch tech influencers. Most users are "regular" folks interested in having one or more healthy Q&A communities. For many of these communities, Stack Overflow is the template for the Q&A model and it informs their initial and ongoing operation. It's clear that without Stack Overflow the other sites that support these communities can't exist, that is not being questioned.

While many of the current communities were proposed and supported by Stack Overflow users from their conception to their launch, each community has developed their own culture and workings. This observation is relevant as most communities are being affected by ChatGPT and have reviewed the possibility of banning generated text content. Many of them have done this.

What are the official implications and repercussions of the current CEO's words for the Stack Exchange communities? I'm not asking that you write down the roadmap or describe the features that are coming as it's clear that you will do that when you are ready. I'm not asking you to add content, but rather communicate the same content in a way that is easier to understand for the regular audience of this space.

Answer 5

I’ve tasked a dedicated team to work full time on such GenAI applications.

Let's get things straight. The company has the budget to waste on "GenAI applications" (whatever that is supposed to mean), but doesn't for:

• keeping Traducir open-source (in case anyone happens to not know why it's important: it powers community-lead efforts for providing translation strings for localized versions of Stack Overflow);
• fixing issues and improving the public Stack Exchange API (I am not even counting MSE and MSO requests, just the Stack Apps ones);
• fixing issues reported on MSO and MSE, as well as implementing feature requests (MSO, MSE);

The list is far from being exhaustive. In light of the following statement:

Community and reputation will also continue to be core to our efforts.

How does syphoning limited resources from the enormous list of problems and improvements that can be implemented across the network to something that is considered wasted effort at best by many members of the said community (the general tone of the discussion under the post is proof enough of that) correlate with the claim that the community is at the core of the company's efforts?

Answer 6

Apparently, even before this initiative picked up steam, the result of this has been, and to use the exact term I used before - "a large negative impact to staffing and resources". There was a significant reduction in staffing - including folks working on public Q&A.

These would a have direct impact on quite a few requests from the community, as well as the company's ability to support the community.

Each sees in today’s AI the potential for the loss of certain jobs

Well looks like that's happened already

To me - this means a reversion to pulling resources from the public Q&A sites to fund potential projects or ones that were failing, and a clear failure to support the community.

As much as the new shiny things - we need to make sure that fundamental needs of these community - support both in terms of community and development needs are met.

As things stand there has been an immediate failure in the company's commitment to the community.

---

Assuming that this isn't SO/SOFT only - what does this actually mean for the communities - and what are the benefits for us?

There are fundamental parts of the network that have not really gotten attention - including some relatively 'trivial' feature requests over design changes (like better access to meta and chat) , and reimplementation of features that have been ignored, or put on the back burner due to lack of resources (notifications), or other priorities.

We've also have a significant amount of workload from folks 'naively' using ChatGPT and other such LLM/ML tools. I'm not purely sceptical of these tools in context, but I'm wondering looking at the costs of other organisations doing this - with massive GPGPU farms and such. It is incredibly expensive - and well, when features and requests that directly benefit the community now are delayed, I'd ask if this is the right call - and would be concerned that resources that could be put to more direct use is diverted to a moonshot.

I've been here long enough to see the rather painful effects that the company getting a bet like this wrong can have. Pivoting to the new hot technology while it’s hot makes sense for a startup, but without a solid idea of what the benefits are is concerning.

So practically, what are the benefits to the communities, and what are the guarantees that if this fails, that there wouldn't be a large negative impact to staffing and resources as has happened in the past?

Critically - is there a plan to turn 'ideas' into a revenue source and clear goals for this team to meet, as opposed to a sense of FOMO?

At Stack Overflow, we’ve had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person?

As an active user of the site? Experience, and the synthesis of knowledge. Humans make links and put their collected knowledge together. I've even occasionally gotten bespoke answers to unusual issues because someone found it interesting. ChatGPT writes like a 13-year-old trying to do a literature review.

We are excited about what we can bring to the fast moving arena of generative AI. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will “hallucinate” facts and figures if they feel it fits the pattern of the answer a user seeks.

Yes, we noticed. We've even banned the use of many of these tools on many of our sites. They're basically emitting entirely plausible garbage, and that's worse than complete garbage.

. Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. We want to help coders stay in the flow state, allowing them to create with the latest tools with the confidence that they will be able to document and understand the provenance, source, and context of the code being generated.

While you have a more focused corpus of information, it doesn't necessarily mean that the generative AI/LLM models you use will actually understand what you're doing. Maybe this might lead to better search, but I feel like outside the lowest hanging fruit, unless you've got something up your sleeve OpenAI and Google doesn't have, it’s hopelessly optimistic.

I'd also question how feeding information into an LLM model works with our licence. Would GenAI-processed results still be under the CC-BY-SA licence and how would you properly attribute the original source of the question?

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information.

My previous job was with a vendor for the local government. Outside the 'data cannot leave the country', the very idea of feeding our data to an external source would freak my former boss out. There's a news article of information leaking as well at Samsung, including confidential source code. This might not be as good an idea as it seems.

---

RESPONSE by Philippe:

The goal is to leverage the latest technology to add value for the Stack Overflow community. We are working closely with community members throughout this process to experiment with GenAI to build solutions to solve for historical pain points with the site. We would love to explore the following areas with community members and will remain open to feedback as we test and learn about these subjects:

• Guided question asking to improve question quality
• Tooling for the site and content moderation tooling to reduce the load on moderators
• Content recommendations to improve discovery and lower duplication
• Personalization of search experiences to improve content discovery
• Trends and insights on emerging topics in the community to highlight new knowledge contributions and subject matter experts

For example, historically, community members have manually provided feedback to newer community members asking questions on the site. Leveraging GenAI to give newer community members real-time feedback on asking questions that are appropriate on Stack Overflow might reduce the load on community members.

We will also work with the Moderators and power users on the site to test and refine the application of these AI solutions going forward: we are committed to building, updating, and curating high-quality knowledge on our site and ensuring these voices are an important part of the feedback cycle as we test these newer solutions to existing community challenges. -- Philippe

Answer 7

It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow.

This is obviously not correct!

How can somebody who understands anything about Stack Overflow think this is correct? The relevance of Stack Overflow has nothing to do with AI generated content. AI generated content is not even allowed there, and the community overwhelmingly supports that ban.

Did anyone check this announcement before it was published? Is there anybody at the company who can tell the CEO he is totally wrong about this?

Answer 8

Currently about 60 sites in the Stack Exchange network ban the use of ChatGPT in them.

Now that Stack Exchange it taking a leap towards AI, does it mean it will also override those bans and allow ChatGPT (and other AI generation programs) posts all around the network?

(Personally I really hope that they'll let the bans stay, but have to ask.)

---

RESPONSE by Philippe:

The existing bans on ChatGPT content are intended to do a few things:

1. prevent plagiarism on the sites, since ChatGPT does not attribute content.
2. prevent a tidal wave of content that LOOKS like it might be correct but isn’t.

There remains a need for both of those things, and we will support those SE sites that choose to continue to ban answers that are created by generative AI programs like ChatGPT.

Internally the work that we are doing toward integrating AI solutions is geared toward using versions of this technology to support the community’s efforts and goals. We are committed to building solutions that benefit the community, with the quality of the content on our sites always at the front of mind. We are committed to building trust in the collective knowledge of the community on the site. This means we will highlight where knowledge came from and prohibit the creation of knowledge that looks superficially solid but isn’t accurate (such as using AI to create answers that have not been reviewed, edited or verified by a community member). We also will not be using AI to create unattributed answers. -- Philippe

Answer 9

I am not here to debug ChatGPT.

I am not here to debug generative LLMs in general.

Answer 10

Disclaimer: I am personally fully against AI-generated answers/solutions on SE. While this answer is somewhat harsh, I do feel the criticisms within it are fair.

---

... John Maynard Keynes wrote in 1930 that new technology meant humans would be working 30 hours a week or less, and that the main challenge would be what to do with all our free time. ...

Can we get a citation for that "30 hours a week or less"? NPR and The Guardian both say 15 hours, not 30. Additionally, assuming you are referring to this document written by Keynes, it says "Three-hour shifts or a fifteen-hour week may put off the problem for a great while." (page five).

... Even before the latest wave of AI, a third of the code being written on popular code repositories was authored by an AI assistant.

Again, according to what? I haven't looked, but what specifically does "popular code repositories" actually mean? And how is this (un-named) source determining how much of the code is AI-assisted? Is it some statistic from a product like GitHub Copilot? Also, when exactly do you consider the latest wave of AI to have started? When GH Copilot was released? Or when ChatGPT was released? Or some other point in time? You make a claim, but it is largely unclear and contains no source at all.

Today, sophisticated chatbots, built on top of cutting edge large language models (LLM), can write functional code for a website based on nothing more than a photo of a rough sketch drawn on a napkin. They can answer complex queries about how to build apps, help users to debug errors, and translate between different languages and frameworks in minutes. At Stack Overflow, we’ve had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? How can our business adapt so that we continue to empower technologists to learn, share, and grow?

That's more fair. I've seen the project about generating the website - I agree, it's pretty interesting. At the same time, though, AI-generated code from chatbots has many limitations, perhaps most notably that it is often flat-out wrong, even if it sounds confident/correct. This is a large part of why SO and other SE sites have banned ChatGPT-generated answers.

It’s worth reflecting on an important property of technological progress. The Jevons Paradox shows us that, as innovation allows us to do more, we settle on a new normal, moving the goal posts for what we expect of people and organizations, then competing to see who can find new ways to pull ahead of the pack. For knowledge work, as the cost of an action diminishes, we often do more of it. Abstracting away repetitive or tedious tasks frees technologists up to make new discoveries or progress innovation.

Fair enough, that's reasonable.

If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we’ll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them.

That’s quite the run-on sentence.

We are enthusiastic about welcoming this next generation of developers and technologists, providing them with a community and with solutions, just as we have for the last 15 years. We’ve got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams and will have some exciting news to share this summer.

At risk of being harsh/rude, WHY?! Please no. A very large amount of work and effort has been put into finding and removing AI-content on SO, and the rest of the network (when it is banned on a site). ChatGPT answers are often wrong and not wanted. I cannot speak for others, but I certainly do not welcome that, nor am I excited for that in any way. Harsh, but that feels like you are flat-out oblivious to the fact that (overall) the community does not want ChatGPT answers (per the vote-counts on the ChatGPT ban announcement on MSO a.k.a. the absolute highest scoring post on MSO).

[image caption] Community members and AI must work together to share knowledge and solve problems

No, we certainly don't. SO and SE has been solving problems and sharing knowledge for many years without ChatGPT. It is certainly not a "must".

I’m not alone in thinking AI might lead to an explosion of new developers. I’ve heard similar sentiments expressed recently by ... Bill Gates, ... Geoff Hinton, ... and by Stephen Wolfram ... Each sees in today’s AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive. Given that, we want to help democratize knowledge about these new AI technologies, ensuring that they are accessible to all, so that no developers are left behind.

Sure, some developers are excited about it. I also find it interesting. But when you say "we want to help democratize knowledge about these new AI technologies", does that mean you want to share information about those technologies? I don't have an issue with that, and doing so is good (assuming it is on-topic, of course)... but actually adding AI into SE is different. Can you clarify which you're referring to?

I talk to developers of varying experience levels all of the time, and I’ve been hearing anecdotes of novice programmers building simple web apps with the help of AI. Most of these stories, however, don’t begin and end with an AI prompt. Rather, the AI provides a starting point and some initial momentum, and the human does additional research and learning to finish the job. The AI can debug some errors, but is stymied by others. It can suggest a good backend service, but often can’t solve all the points of friction that arise when integrating different services. And of course, when a problem is the result not of instructions from a machine, but human error, the best answers come from other people who have experienced the same issues.

Yes. Exactly. Sometimes, it does it correctly. Sometimes, it can't. The best answers do come from other people, not chatbots.

For more experienced programmers, AI will be an amplifier of their existing skill, making them more ambitious in their projects. The result, as Jevons would predict, is that they spend more time with AI, but also more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. They feel empowered to reach farther beyond their traditional skillset and to push the boundaries in terms of the kind of work they want to take on.

I'm not convinced of your first sentence, but that's just my personal viewpoint, though.

One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will “hallucinate” facts and figures if they feel it fits the pattern of the answer a user seeks. Grounding our responses in the knowledge base of ... questions on Stack Overflow (and ... knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. ...

That's a good goal... but, how exactly will you "ground ... responses in the knowledge base of 50 million asked and answered questions"? Specifically, how will you ensure only correct answers? This is way easier said than done - if it was practically do-able, it would have likely already been done.

Community and reputation will also continue to be core to our efforts. If AI models are powerful because they were trained on open source or publicly available code, we want to craft models that reward the users who contribute and keep the knowledge base we all rely on open and growing, ensuring we remain the top destination for knowledge on new technologies in the future.

Yep, the community is very important. But could you elaborate on this a bit? Like, community members will get rewarded when a chatbot answer based on theirs is marked helpful? What exactly does this mean?

AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data – for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. Allowing AI models to train on the data developers have created over the years, but not sharing the data and learnings from those models with the public in return, would lead to a tragedy of the commons. It might be in the self-interest of each developer to simply turn to the AI for a quick answer, but unless we all continue contributing knowledge back to a shared, public platform, we risk a world in which knowledge is centralized inside the black box of AI models that require users to pay in order to access their services.

I'm still not convinced of any benefit from adding AI content into SO, even if it is free. Also, what "symbiotic" relationship? The one in which SO flat-out banned it? Not exactly what I’d call symbiotic at all

As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish.

Yes. Innovation is good, as is progress. Productive discussion of those technologies is also good.

The next paragraph is little more than an ad for SO Teams, so I’m omitting it.

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains: from new hires being able to onboard more quickly, to speed up developer workflows, as users are able to quickly ask questions and retrieve answers tapping into the company’s history, documentation and Q&A.

Would it be limited for SO for Teams? Or would it go to actual SO? Not trying to be overly harsh, but can you elaborate on what information a chatbot would be able to give that wouldn't be accessible by searching?

The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams, and they have energized everyone at our company. We’ll be working closely with our customers and community to find the right approach to this burgeoning new field and I’ve tasked a dedicated team to work full time on such GenAI applications. I’ll continue to share updates through channels such as my quarterly CEO blog, but I’ll be back in touch soon to announce something big on this topic. In the meantime, thank you to our community and customers for continuing to help us on our mission to empower the world to develop technology through collective knowledge.

NO please.

I do NOT want generative AI in the public SO. Also, is everyone at the company actually excited about this? I suppose maybe, but it seems like an exaggeration. Not going to name any names, but what about the previous statement (from a staff member, on MSE), that SE would "work internally progresses on identifying these posts and making our systems more resilient to issues like this in the future" (referring to ChatGPT-answers). That answer made me think SE was going to help communities deal with AI-answers. Can you clarify if that answer is still true?

Also, my answers on SE are intended for humans, not for training a machine learning model with.

As someone that has personally put in a large amount of effort and time into dealing with AI-generated answers on SE, I'm extremely disappointed to hear that you may incorporate generative AI models into SO.

Answer 11

After noticing some worrisome exchanges about "SE is going to use generated content" in the chat rooms, I came here to verify those rumors and to determine concretely what the actual "plan" was.

Yet after reading the "announcement" multiple times... I don't yet know what the plan is. I can't see what you are talking about. I can't see what your ideas are. I can't see what you are going to do. Nor do other users apparently since most answers start with a "if you mean to...".

This announcement could be summarized as a haiku and no actual meaning would be lost.

GenAI is future
Many cool guys say that too
Let's jump the wagon

All I can see is that (as Zoe pointed out in a comment) this actually feels more like an "early announcement" than a "we want your feedback on our plan" post. If anything, because if you wanted actual feedback we would at least need to know WHAT the plan actually is.

Oh, yes, you then say that "soon" you will be telling us what the "big plan" is, but that would only mean that we can't have any question now, right? Also, does this "soon" mean "after we already committed to it, you will get to know when we have already decided and your feedback won't matter anymore"? What is the point of asking us to "leave any questions that you have below" NOW if we actually have to GUESS YOUR INTENT because you didn't tell us what this is all about???

tripleee speculated about three different scenarios in their post. For starters... I guess it would be nice to know if any of these actually fit what your are planning to do.

Answer 12

Please do not integrate this algorithm into the site.

It plagiarizes

The algorithm will reproduce whole sections of one to one content without citation nor warning. It then strings these sections together with transitional words such as however, additionally, or furthermore.

It cannot generate

Aside from the issue of reproducing existing works without citing them, the algorithm cannot actually generate legitimate content which fails at its own primary goal. Instead, any time the algorithm generates content, it simply goes with the most likely to occur series of content; as opposed to the correct series of content. This is dangerous at worst, and "hallucinating" at best.

It acts as an authority

The algorithm presents all of these flaws, the issues with citation and of lying hallucinating, as if they are canon; as if each statement, code, or demonstration were 100% accurate and trustworthy. Integrating something which presents content in such a manner is negligent for a place aimed at being a repository of knowledge.

---

There is no need to integrate an algorithm which has these traits. It is dangerous, negligent, and ethically questionable.

Answer 13

Note that I've flagged ~800 GPT answers on Stack Overflow. I was honestly hoping for a bit more information about how Stack Exchange as a company (rather than the individual sites) plans to handle the deluge of unverified AI answers that all too often turn out to be hallucinations.

Instead, I ended up not quite sure what was being announced, even after reading it through twice. I do turn to ChatGPT myself for its ability to re-word and summarize information, so I decided to ask it to summarize the post to see if there was something I missed:

Prashanth Chandrasekar, CEO of Stack Overflow, believes that artificial intelligence (AI) will not replace programmers but rather will create millions of new software developers from fields like finance, education and art who previously didn't have access to AI-powered tools. Sophisticated chatbots, built on top of cutting-edge large language models, can already write functional code and answer complex queries about how to build apps, debug errors, and translate between different languages and frameworks in minutes. While AI provides a starting point and some initial momentum, humans do the additional research and learning to finish the job. AI will amplify existing programming skills and help programmers spend more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. Stack Overflow plans to add GenAI to its products, democratize knowledge about new AI technologies, and provide a community and solutions to the next generation of developers and technologists. (emphasis added)

This was really in line with my personal reading of the post. Since everything but the last sentence seems to me to be rather commonly accepted, I guess the "announcement" can be boiled down to the final sentence, which still leaves things quite open.

Answer 14

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains.

I think it'd be more beneficial and productive to work on better search functionality, and better possible-duplicate detection in the Ask Question module

Generative AI is working with past information- probably taking from our past Q&A.

1. If it can answer a new question, I'd have suspicions about the new-ness of the question.
2. If it can't (likely when faced with a new question), then you should just leave it to your community of experts who actually know what they're talking about.

In either case, what's important on the asker's side is to search, and what would be more useful to them (and anyone else looking for possible duplicates of a question) is better search functionality.

I'm confused about whether this is coming to SO/SE or just SO for Teams

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information.

This paragraph talks about integrating AI into SO for Teams and doesn't mention SO, but the next paragraph mentions both:

The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams.

So which is it? Are you just adding AI to SO for Teams? Or also to SO (/ the rest of the public Stack Exchange network)? I don't care much about SO for Teams. But if AI is coming to SO or the rest of the Stack Exchange sites, I really hope it's put in a completely separate area in each network site than regular Q&A. For example, I don't want to see AI-written answers under question posts.

Either that, or you've got to (not really, but it sure would be nice if you'd) talk with us and think real hard about

• How to make sure bad AI-generated content is easy for people to vote down (I'm thinking of rep thresholds)

• How that content is going to age over time. What would you do when the language model updates? What would you do with already-written answers with older language models?

• What moderation burden is this going to put on the community and its moderators?

---

For the rest of my thoughts, see also my answers to

• Could ChatGPT be a viable way to answer people's questions?

• The future role of Stack Exchange vs. emerging AIs

---

various other commentary:

We are excited about what we can bring to the fast moving arena of generative AI

The community has already been bringing it since inception. We write content licensed under CC-BY-SA. The language models are using our community-written content (whether the way they use it is allowed under CC-BY-SA, I don't know. I'm not a lawyer).

AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow.

So yes. Agreed :) and with the rest of that paragraph. Except that I wouldn't say "symbiotic". Maybe "parasitic" (with Stack Exchange being the host organism). Even when people "contribute" AI-generated answers to questions here, it's often a hallucination (and against the site rules because of the hallucinations and other reasons like moderation and curation burden).

Our thesis on community as the center of a safe, productive, and open future for AI also offers some exciting prospects for our business

Most of what was written up to that point was pretty grounded / concrete (at least- I thought so). But I don't really get this statement. It seems a bit wishy-washy to me. I suppose community is important in that that's where voting and editing comes in (curation), but I'd say the bigger source of value in what we do here is freely contribute expertise. The experts are (in my current view) the starting point of what makes Stack Exchange valuable. I don't think it's true to say that all the experts here are here for community. They might be attracted to here because of the standard of quality and name-recognition of the large network sites though (not sure what either of those things has to do directly with community). Or they might be here for magic internet points and/or to alleviate an inferiority complex. Regardless of what is central to the future of AI, what is really central to Stack Exchange (which is what I care about) is fact-based Q&A. To say community is central to Stack Exchange (not that anyone has said that here) could be misleading, since we're not a discussion platform and we don't do chit-chat (except in chat subdomain sites).

Side note: Another problem I've seen with ChatGPT answers on SO is that they often have a bunch of "buffer info"- info that may be useful, but really isn't necessary in the context of the question. One of the selling-points of Stack Exchange (see the tour page) is that we avoid noise.

Answer 15

Quite simply, if people want to use some kind of AI assistance to answer their questions quickly, then they can use those.

If they want an actual human, often expert, to answer their questions, they will log into Stack Overflow.

I don't understand what the benefits of mushing these two approaches together would be. Stack Overflow's unique selling point is that it (should be at least) expert humans answering questions using their personal experience. Why risk killing this to add something people can do on another site¹?

^{1 I know the answer is $$$}

Answer 16

Welcome to the enshittification of Stack Overflow!

No one at the helm is reading or cares about this page.

Make no mistake, within less than a year new visitors will get a first answer or at least a question rewording suggestion provided by a ChatGPT-alike.

Answer 17

The GenAI hype-train manages to (just about) clear the bar for usefulness set by its illustrious recent predecessors (web3.0 and metaverse). But regardless of its own merits, it's important to recognise that it is currently a hype-train.

Public interest is high, the news both tech and non-tech are excitedly reporting about it left and right and the average Joe is lapping that up. But this level of public giddiness isn't sustainable - there will be something else new and exciting that will come along soon and capture the public's interest and generative AI will continue but it won't be grabbing every third headline any more.

This isn't a new phenomenon - humans have been getting bored with the latest and greatest for a very long time, just ask NASA how quickly the general public lost interest in the moon-landings.

So if SO Inc is wanting to get in on this area - are they prepared for what happens in 3, 6, or 9 months when it's not headline news any more? Is this something that is genuinely in the best interests of the company and the network and would add real value without that hype? Because if the answer is no - then I'd question the wisdom of throwing any real resources at this, and instead cash in on the hype by writing a few blog posts, have some people discuss it on the podcast or whatever. Sure you probably lower your ceilings on potential rewards doing that - but you avoid throwing millions at something and having it turn out to be a boondoggle.

Answer 18

I must be a very unlucky person. These things that seem to sprout like flowers in other people's IT gardens just don't happen to me. Every question I ask the AI comes with errors at best, that's when the AI decides to be creative and starts spewing a bunch of information taken from I don't know where.

I'm not saying that AI is useless or anything like that. Far from it, I like AI and I like the promise of social progress that it brings to humanity but what I mean is that it's premature for us to have this type of integration in the community that is the foundation of activities directly and indirectly related to Information Technologies.

Asked about the possibility of its participation in the ChatGPT communities, ChatGPT responded:

My participation in the ChatGPT communities is limited to providing responses to your questions and engaging in conversation with you. As a language model, I am not capable of actively participating in the same way as humans do. However, I am here to assist you in any way I can and provide you with the information and insights you need.

I am convinced that the training of AI is currently not robust enough to meet the needs of our communities. An error made with the endorsement of AI has the potential to spread virally and be incorporated into various systems used around the world.

I believe that any AI system must be tested, improved, specialized, and retested before it is made available to our public, and that its responses should be finally curated by members of the Stack Exchange community.

Without being polite, I feel like they're saying to me:

-Okay, thank you for these years of unpaid work but we don't need you anymore. From now on, we'll have a little system that will work for free just like you, but with one difference, the system it's a bit imprecise and doesn't complain when it sees the mistakes we've made and in fact, it doesn't even see them.

Answer 19

Please, no. AI is not remotely close to any real "artificial intelligence" (in the literal sense) these days, and it does not belong as a contributor on SO/SE sites in any shape or form at the current time, unless you mean to recast the sites as training grounds for future generations of GPTs. If that's the choice, I expect such a change of direction to alienate and drive away many of the longtime good-faith contributors, myself included.

IMHO the very suggestion of integrating "generative AI technologies" shows a huge disconnect between sweet dreams of (future) AIs helping make everything better vs. the reality of (current) AIs being mistakenly trusted to provide the answers, which are often wrong, but may also be deceivingly plausible many times.

We've seen this on Math SE already, where users have been quoting complete nonsense written by AIs with much nonchalance and self-assurance, then wondering whether they were "missing something".

(A random example from the last hour is here, which was deleted since, but still visible to 10k+ rep users. The AI confidently stated that "two of these n numbers must have the same parity, which means that one of them must divide the other". It takes very little math knowledge to realize how hilariously nonsensical that assertion is. Of course, humans can and do make silly mistakes, too. Difference, however, is that most of us refrain from posting every one of them out of a sense of propriety, and, anyway, we would not be able to generate them at the rate of hundreds per minute even if we wanted to for whatever trolling reason.)

To end on the same note I started off with, AI is not remotely close to any real "artificial intelligence" (in the literal sense) these days. The gap can be closed in one of two ways. Either keep making AI smarter until it gets close. Or, dumb down the collective "intelligence" until it matches today's GPTs. Unfortunately, SO/SE embracing generative AI now looks like it's leaning towards the latter. Please, no.

Answer 20

Let's get something straight: this is just another big-tech (or, in this case, big-tech wannabe) jumping on the bandwagon. And jumping on the bandwagon usually doesn't lead to good long-term decisions.

A CEO, who as far as we know has never posted a question on Stack Overflow, should not be the one making the call on whether or not we like AI on our sites. I have no problem with Stack Overflow having a CEO or making a profit, but corporate has absolutely no standing to say how to better run our sites. The community's decision to ban ChatGPT has become the most upvoted meta post ever by far. Why does this one hotshot tech CEO get to override all of our voices?

Even if we ignore all the practical obstacles other posts have pointed out, the very idea that the CEO would put out a statement that clearly contradicts the sentiment of everyone on this network is a straight-up insult to the whole network. Not to mention this guy has repeatedly bragged about how "democratic" Stack Overflow is. For god's sake, just let the people who know the site run the site.

Answer 21

I agree with the general consensus that ChatGPT in its current state can be dangerous when used incorrectly (which is especially likely with inexperienced coders). However, when used correctly, it could be helpful.

Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use.

This is the key point that I think a lot of people here are missing. LLMs could point users to existing answers, which doesn't give it an opportunity to hallucinate information. They can also easily solve simple issues and provide tailored guidance for question askers.

Many ChatGPT answers are not easy to verify, but that doesn't mean that all of them are like that. If I make a simple mistake such as forgetting a return statement, ChatGPT can see that and point it out to me. I see the potential for a system where an LLM can respond to particularly simple questions before they are even posted, reducing the flood of low-quality questions.

There are currently LLM solutions that aim for trustworthy answers that cite sources (perplexity.ai comes to mind). These are of course only as good as the training data, but that training data is by definition as reliable as the answers given by humans. In this way, it could be used as a smarter search engine that is customized for Stack Overflow.

Another safe way to integrate LLMs into SO is by providing question-asking guidance to new users. It could be possible for an LLM to say, for example, "Hey, it looks like you are encountering an error, but you haven't included a specific error message in your question. If you haven't already, could you copy-paste the error you're getting directly into your question?"

Overall I don't think it will be easy to use LLMs/GenAI in a non-disruptive and safe way, but I am cautiously optimistic that it will be possible to do so. One thing to keep in mind is the self-driving car evaluation problem: do we aim for perfection, or do we just aim for being better than the average humans who are currently doing the task? Perfection is ideal, but better-than-human performance is still worth using.

Answer 22

Let me try to synthesize an analysis of some possible interpretations and my (and I expect, the broader community's) attitude to these variations.

• "Yikes, no!"

  Interpretation: "We will start polluting the site with ChatGPT content based on its current capabilities, and with our usual rigorous testing and interaction with the broader user community" (that's sarcasm for "quickly, surprisingly, with bad timing, and without proper thought or coordination").

• "Mmmmmmaybe..."

  Interpretation: "We want to jump on the current hype bandwagon and spend months on an expensive project which will probably ultimately be cancelled. But if it's successful, we expect to see some sort of next-gen AI which actually doesn't try to answer when it doesn't know, which could really boost the site's capabilities and your user experience. For the time being, though, don't be nervous; this is just something we put out to hopefully instill some new market confidence in our stock evaluation." (The last sentence only illustrates how bad I am at market-speak, I'm afraid. I hope you get the idea.)

• "Yes! You're on the right track!"

  Interpretation: "Our focus is on reducing the number of badly articulated beginner questions by answering them before they are actually published on the site. We have a proof of concept for a new question model which shows some promise already."

Answer 23

There are several things that large language models and their ilk can be useful for in the context of Stack Exchange. Let me list some of them:

• Recommending titles for the questions. Humans are very bad summarizing content in terms that are relevant. I'm sure anyone can dig up several examples and discussions about this. Language models are good at crunching big texts and summarizing it.
• In the same vein, it could be used for making suggestions about the content to add information. The prompt would be something like "what information this question lacks that there are comments in similar questions asking for?". Basically, offloading the work of readers to the machine so it reduces the friction in the amount of comments left on the question, the need to close questions because they lack information, and the user perceiving user interactions as hostile.
• Summarizing duplicates. We've tried this before by suggesting related questions titles with not much success (measured by the number of users that either alter their question or decides not to ask it). Instead of that, something akin to "of these related questions, summarize the 3 top positively scored answers, and include a summary of the question itself, asking if it's what they are trying to do". Obviously, improving the way to detect duplicates would help here, to make sure that semantic equivalence is maintained.

I would be very sad if those three branches of research aren't explored.

Answer 24

This is less about the content of the blog post and more about what SE should do from preventing this from backfiring. I'm a little surprised that nobody's explicitly put this out yet, considering that there are...28 answers and counting, but please: listen to the community.

Yes, the CEO has some sort of vision about wedding SE with AI in some sort of warped matrimony. And perhaps there's merit to it, or at least to parts of it (In my opinion, there isn't, but what do I know?). But the overwhelming consensus here in the answers, comments, and votes is that no, we don't want it, certainly not what's been implied in this vague-but-foreboding blog post.

SE's biggest failures have been when they've just steamrolled ahead with a plan without seeking input from the community, or when they get input from the community and proceed to ignore it. But some of their best moments are when they ask for our feedback, acknowledge our feedback, and respond to it by, at the very minimum, factoring it into the decision-making.

So whatever you do, please do it gradually, please do it thoughtfully, and please let us have a voice in some of the decisions. And please do it having fully considered the effects that this would have on the community.

Answer 25

Things I look forward to (then a question)

1. Finding citeable answers faster with ML guidance

With a ML-driven chatbot, or similar interface, I can imagine it will be easier to find human-written answers without actually posting a question. The image below is a screenshot from an ML-driven search engine (not shown are the citations below the answer, probably the most valuable part):

2. Writing better questions with ML guidance

I can easily imagine the Ask Wizard evolving to help newcomers and old-timers alike improve the quality of their questions. Including:

• answering them without posting a question (see above)
• helping them find resources that might help them answer their own question

(Obviously I could use ML tools to do these today, no integration necessary.)

3. Generate hypothetical answers I can test, approve and then publish

I'm most skeptical of this one, but with the right gamification it might be possible. (Harsh penalties for anyone publishing nonsense.)

4. Continue to have the Stack Overflow data dump publicly available under BY-SA 4.0

(I assume this still exists, I sometimes miss the news.) This public data dump of the community's hard work is a social asset. I would love to see SO release it as an LLM model compatible with Dolly 2.0 or whatever the open format ends up being. Failing that, keep the data openly and publicly available so that others can try is a must.

(Obviously this doesn't apply to Stack Overflow for Teams data.)

My question

Am I being overly optimistic to think that these are the kinds of ideas Stack Overflow is considering, rather than generating work for volunteer moderators?

Follow-up

Journeyman Geek points to Phillipe's response in his answer (emphasis mine):

... We are working closely with community members throughout this process to experiment with GenAI to build solutions to solve for historical pain points with the site. ... Leveraging GenAI to give newer community members real-time feedback on asking questions that are appropriate on Stack Overflow might reduce the load on community members.

And in Phillipe's post "An example of a generative AI tool: Creating better question titles":

We know that title creation is difficult for new askers, and this [ML powered process would be] a low-friction way to make that process easier for both askers and moderators alike.

So I think Stack Overflow are indeed planning labor-reducing tools. Whether they implement my brilliant ideas is another story ;)

Answer 26

And just who is supposed to review and moderate all of the crap AI content posted by users farming for no-effort reputation, in order to weed out the worryingly low percentage of correct answers from the incorrect ones? Perhaps the CEO, who as far as I'm aware has never actually curated a single post on the site, is somewhat unaware of the (free) labor shortage issue he has in this regards. We're already hurting for reviewers. It often takes a subject matter familiar/expert to spot that an AI-generated post is indeed factually completely wrong. It can easily be argued that we're already overwhelmed on some reviewing fronts (and that this was perhaps the case even before the flood of incorrect AI-generated content started coming in).

Also, what about Stack Overflow's reputation? As word gets out that more and more answers on the site are just plain wrong despite being so deceptively well formatted and well written, this site's value will tank and it may not ever recover. I don't know how you get your reputation back from that disaster.

Any good executive always has the long-term interests of the company as priority. I think the CEO needs to explain how allowing a bulge of untested, far-too-often incorrect, information on this site will benefit the community or the corporation in the long term or even the short term.

Answer 27

If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we’ll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them.

Tell me you have never trained a junior developer without telling me you have never trained a junior developer. Even then, a junior developer is someone who (hopefully) wants to be writing software, nevermind individuals with little-to-no technical aptitude. This pie-in-the-sky fantasy that non-technical people will be able to chat with an LLM only about their domain knowledge and end up with a usable, novel, or deployable application is misguided at best.

Prashanth implies that people from non-technical backgrounds will start using "AI" to write code. To make my point as concise as possible: LLMs will not enable millions of non-developers to suddenly be capable of writing software. Reason I bring up junior devs, which are far more likely to have either an aptitude or interest in software development, still require a great deal of training to understand delivering software beginning to end. LLMs may prove helpful in bringing juniors up to speed and augmenting people already in the field, not workers that may prefer painting.

Answer 28

I have just one simple question, yes or no, but would like to get honest answer even if not pleasant.

Were the recent layoffs part of this future project(s) the blog post mentions? It would make sense, as such projects need lots of money, and business can't afford leaking too much money without compensating it.

Answer 29

Each [of the three people mentioned] sees in today’s AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive.

The bit about farmers caught my eye. A little searching turned up this graph, which I knew must exist:

I'm pretty relaxed about ChatGPT, but for the sake of argument, let's suppose AI tools are to programming as tractors are to farming. By analogy, there would end up being fewer programmers (as a percentage of total population) each doing a greater share of the available work. I have no opinion if this is good or bad for society as a whole.

What about Jevons paradox? Here's a quote from the linked article:

In 1865, the English economist William Stanley Jevons observed that technological improvements that increased the efficiency of coal use led to the increased consumption of coal in a wide range of industries. He argued that, contrary to common intuition, technological progress could not be relied upon to reduce fuel consumption.

This isn't talking about the composition of the labor force! I suspect that as technology has improved, the percentage of people operating power-generating machines has decreased since 1865. (I didn't look for that chart, however.) Instead it's a phenomena of supply-and-demand when increasing efficiency can increase consumption of a resource due to increasing demand from decreasing effective price. The key determinant is whether the demand is elastic or inelastic.

It helps to think of a specific application. I work for College Confidential and we currently use natural language generation (NLG) to create profiles of colleges based on a dataset we subscribe to. Recently we tried generating some content using ChatGPT and shared the results with our community. Then we used a corrected version to update the school profile.

I think the final result using ChatGPT is better than the NLG system in terms of creating an interesting page. That's not the important thing though. What matters is what students and their parents are looking for in a school profile. If a better profile results in more demand (elastic demand) than we will probably put more effort into using ChatGPT and similar technologies. But if they don't care any more about this version of the profile than the previous version, we'll decide based on other factors, such as cost, whether to continue this work.

My guess is that students are going to be more interested in seeing what our community says about these schools and that ChatGPT is only advantageous because we are seeking their input. But that remains to be seen.

Like any gold rush, the people who stand to make the most money are the outfitters followed by the con men followed by the lucky few who strike it rich (without immediately losing it to the con men). Large language models (LLMs) sure are shiny and if I were to invest, I'd put my money on companies that provide the computing power for these models.

Answer 30

Since there's a few complaints over the writing style of the blog post - so I made an attempt to 'translate' this to something a little more accessible. This is a 'classic' community wiki - I may miss things or misinterpret things, so feel free to add, subtract or correct my interpretation of this. I feel rather than a literal interpretation between the 'business'/'management' writing style - annotations make more sense.

I'm quoting, but not strictly following the paragraph structure Prasanth had originally, for ease of reference. There's also some bits where I feel like it’s more about the 'intent' than the exact wording - these are in the form of 'editorial' notes.

It would be nice to whip this into shape as a full blown, stand-alone document, but there's a bit more work for that.

Throughout history, great thinkers have made predictions about how new technology would reshape the way in which humans work and live. With every paradigm shift, some jobs grow, some change, and some are lost. John Maynard Keynes wrote in 1930 that new technology meant humans would be working 30 hours a week or less, and that the main challenge would be what to do with all our free time. So far, predictions of this nature haven’t exactly come true. As new technology empowers us, we push ourselves to new heights and reach for previously unattainable goals.

A bit of rough background of how the effects of technology was expected to affect work, and where such predictions have fallen short, ending with how technology could enable reaching more difficult goals.

Over nearly 15 years, Stack Overflow has built the largest online community for coders to exchange knowledge, a place where anyone with an internet connection can ask or answer questions, free of charge, and learn from their peers. Stack Overflow for Teams, our enterprise SaaS product, is trusted by over 15,000 organizations to serve as their internal knowledge bases.

(ed: A brief introduction to Stack Overflow and Stack Overflow for Teams)

With the recent advent of dramatically improved artificial intelligence, many industries are wondering how technologies like ChatGPT will change their business. For software development, the answer seems more immediate than most. Even before the latest wave of AI, a third of the code being written on popular code repositories was authored by an AI assistant.

Today, sophisticated chatbots, built on top of cutting edge large language models (LLM), can write functional code for a website based on nothing more than a photo of a rough sketch drawn on a napkin. They can answer complex queries about how to build apps, help users to debug errors, and translate between different languages and frameworks in minutes.

Essentially the thesis here - that LLM/AI tools have had an effect on software development, with significant amounts of code written by AI assistance. There's also chatbots that can translate various human inputs to code and answer questions.

At Stack Overflow, we’ve had to sit down and ask ourselves some hard questions. What role do we have in the software community when users can ask a chatbot for help as easily as they can another person? How can our business adapt so that we continue to empower technologists to learn, share, and grow?

This seems to reflect some self doubt - with Stack Overflow questioning if it has a place where a chatbot can answer questions, and if change is needed for the business to remain relevant

It’s worth reflecting on an important property of technological progress. The Jevons Paradox shows us that, as innovation allows us to do more, we settle on a new normal, moving the goal posts for what we expect of people and organizations, then competing to see who can find new ways to pull ahead of the pack. For knowledge work, as the cost of an action diminishes, we often do more of it. Abstracting away repetitive or tedious tasks frees technologists up to make new discoveries or progress innovation.

Jevon's Paradox - essentially, as work gets easier, there's more of it, not less in this case

If new AI systems make it possible to create software simply by chatting with a computer, my prediction is that, far from the job of programmer disappearing, we’ll end up with millions of new software developers, as workers from fields like finance, education, and art begin making use of AI-powered tools that were previously inaccessible to them.

These tools won't remove the need for software developers, but make it easier for folks who are not 'traditional' software developers to use these tools to perform tasks

We are enthusiastic about welcoming this next generation of developers and technologists, providing them with a community and with solutions, just as we have for the last 15 years. We’ve got a dedicated team working on adding GenAI to Stack Overflow and Stack Overflow for Teams and will have some exciting news to share this summer.

(Nothing to add here for the moment)

I’ve heard similar sentiments expressed recently by Microsoft founder Bill Gates, by Geoff Hinton, the godfather of the neural network approach that produced today’s AI revolution, and by Stephen Wolfram, a pioneer across computer science and mathematics. Each sees in today’s AI the potential for the loss of certain jobs, yes, but also, if history is a guide, a future in which a great variety of more highly skilled work becomes available to an even larger group of people. Just as tractors made farmers more productive, we believe these new generative AI tools are something all developers will need to use if they want to remain competitive. Given that, we want to help democratize knowledge about these new AI technologies, ensuring that they are accessible to all, so that no developers are left behind.

Fairly influential tech leaders believe that while AI could cause certain jobs to become obsolete, there's 'new' types of skilled work that is accessible to more people. There's a belief that generative AI tools are essential, and accessible to anyone who needs them.

I talk to developers of varying experience levels all of the time, and I’ve been hearing anecdotes of novice programmers building simple web apps with the help of AI. Most of these stories, however, don’t begin and end with an AI prompt. Rather, the AI provides a starting point and some initial momentum, and the human does additional research and learning to finish the job. The AI can debug some errors, but is stymied by others. It can suggest a good backend service, but often can’t solve all the points of friction that arise when integrating different services. And of course, when a problem is the result not of instructions from a machine, but human error, the best answers come from other people who have experienced the same issues.

AI tools can help someone get started on a project, but when it comes to errors, and more complicated problems it’s sometimes not very good. It’s still not a replacement for human experience - when people have had the same problems, they know how to resolve the problem better.

For more experienced programmers, AI will be an amplifier of their existing skill, making them more ambitious in their projects. The result, as Jevons would predict, is that they spend more time with AI, but also more time creating new ideas, researching new topics, and asking new questions that had not occurred to them before. They feel empowered to reach farther beyond their traditional skillset and to push the boundaries in terms of the kind of work they want to take on.

Rather than replacing more experienced developers - it might allow them to do more innovative things, and expand their abilities.

We are excited about what we can bring to the fast moving arena of generative AI. One problem with modern LLM systems is that they will provide incorrect answers with the same confidence as correct ones, and will “hallucinate” facts and figures if they feel it fits the pattern of the answer a user seeks. Grounding our responses in the knowledge base of over 50 million asked and answered questions on Stack Overflow (and proprietary knowledge within Stack Overflow for Teams) helps users to understand the provenance of the code they hope to use. We want to help coders stay in the flow state, allowing them to create with the latest tools with the confidence that they will be able to document and understand the provenance, source, and context of the code being generated.

Current LLM systems don't really differentiate between whether answers are correct or wrong and will make things up that look like they answer the questions a user has. By basing our responses on the data on Stack Overflow and Stack Overflow for Teams, they know what the model they are using is trained on. This would allow them to use these tools know what they were trained on (ed: "Flow State" is the current marketing slogan Stack Overflow seems fond of).

Community and reputation will also continue to be core to our efforts. If AI models are powerful because they were trained on open source or publicly available code, we want to craft models that reward the users who contribute and keep the knowledge base we all rely on open and growing, ensuring we remain the top destination for knowledge on new technologies in the future.

(ed: I'm not actually sure if models here means LLM model, or models of doing things, and it does feel like it’s talking about integrating)

AI systems are, at their core, built upon the vast wealth of human knowledge and experiences. They learn by training on data - for example open-source code and Stack Overflow Q&A. It is precisely this symbiotic relationship between humans and AI that ensures the ongoing relevance of community-driven platforms like Stack Overflow. Allowing AI models to train on the data developers have created over the years, but not sharing the data and learnings from those models with the public in return, would lead to a tragedy of the commons. It might be in the self-interest of each developer to simply turn to the AI for a quick answer, but unless we all continue contributing knowledge back to a shared, public platform, we risk a world in which knowledge is centralized inside the black box of AI models that require users to pay in order to access their services.

AI systems are only as good as the data they are using, and open source code and Stack Overflow Q&A are examples of this. That AI learns from humans and humans can use AI is why community platforms will be relevant. However if AI learns from data, and developers use AI for quick answers, without knowledge contributed back, knowledge will end up behind a paywall.

As the AI landscape continues to evolve, the need for communities that can nurture, inform, and challenge these technologies becomes paramount. These platforms will not only offer the necessary guidance to refine AI algorithms and models, but also serve as a space for healthy debate and exchange of ideas, fostering the spirit of innovation and pushing the boundaries of what AI can accomplish.

(ed: Not much to say here)

Our thesis on community as the center of a safe, productive, and open future for AI also offers some exciting prospects for our business. Stack Overflow for Teams, our enterprise, private version of Stack Overflow, helps to power a community-driven knowledge base inside of 15K+ organizations like Box, Microsoft, and Liberty Mutual. Decades of institutional knowledge, shaped and curated by subject matter experts and experienced teams, allows the employees at these organizations to more easily collaborate, improving productivity and trust.

(ed: It's a pitch for Stack Overflow for Teams)

Incorporating generative AI technologies into the organizations using Stack Overflow for Teams will allow us to layer a conversational interface on top of this wealth of information. We believe this could lead to tremendous productivity gains: from new hires being able to onboard more quickly, to speed up developer workflows, as users are able to quickly ask questions and retrieve answers tapping into the company’s history, documentation and Q&A.

(ed: This does feel like they are talking about a chatbot over SO data)

The example above is just one of many possible applications of GenAI to our Stack Overflow public platform and Stack Overflow for Teams, and they have energized everyone at our company. We’ll be working closely with our customers and community to find the right approach to this burgeoning new field and I’ve tasked a dedicated team to work full time on such GenAI applications. I’ll continue to share updates through channels such as my quarterly CEO blog, but I’ll be back in touch soon to announce something big on this topic. In the meantime, thank you to our community and customers for continuing to help us on our mission to empower the world to develop technology through collective knowledge.

People are apparently excited about this, and there's a specific full-time team for Generated AI applications. There's some big news coming soon.