Stack Overflow needs to change to become less restrictive to adapt to an environment where ChatGPT is available

Question

I want to write a post on the meta (here) to raise awareness of a realization that I had this morning while trying to find some information via the Internet. Sorry for the length of this post; I think the serious nature of the matter warrants a detailed post.

What is the TL;DR for this?

• I had a question which was quite open ended. I initially thought of asking on the Unix & Linux Stack Exchange site, but then decided against it given that (extensive) previous experiences suggest to me that the question was unlikely to be received positively, and therefore posting on SE would be pointless.
• Instead I asked ChatGPT. I am concerned by the fact that I made this decision. I think it hints at problems with the SE system in general, and my concern is that if SE does not change to become more open then it is simply going to be replaced by ChatGPT and I don't think this will be a good thing for any of us who use SE sites.

I want Stack Overflow to become better

• I want Stack Overflow and the wider Stack Exchange network to continue to exist and thrive
• I am not interested in starting a controversial flame war between ChatGPT users and SE users who do not use ChatGPT
• In this post I will be quite critical of the ways in which I think Stack Overflow is failing, particularly in a context in relation to the existence of ChatGPT which is becoming a comparable competitor. Please do not take this personally. Sometimes to improve a system we have to say the system fails in a certain aspect of how it functions. This is all I am trying to do here: Improve SE and particularly Stack Overflow
• It's not a rant. If you disagree, just say you disagree. Don't say it's a rant

With that out of the way, let me jump into it.

Some background (your career, probably)

I think it is useful to have some background context before moving on to explain my observation from this morning.

As a developer, your journey probably went something like this:

• You picked a programming language (in my case BASIC, but maybe you tried something more modern like Python, MATLAB or JavaScript) and you started using it. Your first programs probably included "Hello, World!" and a for loop which prints the numbers 1 to 10.
• For some period of time (perhaps several years) you are in a phase of learning where you can ask very specific questions, which Stack Overflow likes. Examples may include "how do I fix this compiler error?", "how do I install this dependency?", "how do I modify this configuration file to do X?".
• Eventually if you have been in the game long enough you start to become responsible for building things. Perhaps you become team lead of something within an organization or even CTO.
• Your questions now fundamentally change. Sometimes you will still need to ask simple, specific, very focused questions. "I know how to do this in language X, how do I do it in language Y which I have never used before?"
• However, many questions become more high level and as a consequence more vague. Systems design questions are obvious examples. These questions do not appear on Stack Overflow, because they are too high level. This is often mistaken to be "opinion-based". Because there is more than one way to skin a cat.
• I think what makes these questions fundamentally different is that you want to ask "give me some possible ideas about how to do [something] such that I can make an informed decision about what direction I want to go in*. You want to become better informed and share, improve and collaborate on ideas.
• In my experience, it is very, very rare to find people IRL who you can discuss such things with. This combination of creativity and such a high level of domain-specific experience in a niche area make such people exceptionally rare, so the obvious place to look to find them is the Internet.

Observation

This morning I was looking for some information via the Internet. At a most basic level, this is what I was trying to do: I had some questions, and I wanted to use the Internet to obtain some answers. These questions were closer to initial ideas than to very detailed and specific questions.

To give some idea about what I wanted to ask:

• I run a server at home which hosts a self-hosted Community Edition of GitLab. The disk onto which the data is saved is RAID 1 mirrored. This protects against data loss under some conditions, but not all. The data is not "backed up", because there is only one copy of it on a single system. I wanted to find out information about what methods and techniques might be available for use to improve redundancy and to protect against data loss.

As you can see, this is already quite an open-ended question, and because of this it is unlikely to be received well by the SE community. It would probably be closed as being "opinion-based" despite the fact that it is not opinion-based. There may be more than one possible solution, but that does not make it opinion-based.

Things become more complicated when you add in the fact that I had some initial ideas.

• Perhaps there could be a way to get GitLab to recognize it is connected as part of a cluster of GitLab instances and mirror/backup the data across multiple nodes
• Perhaps Unix tools, like rsync, could be used to periodically backup the data
• Perhaps a self-hosted file syncing service, like Nextcloud, could be used to back up and synchronise data from the server hosting GitLab to another machine
• Perhaps there are other possible solutions, since I don't know which would be the most effective it would be useful to know about a range of possible ideas
• I wanted to ask what are the advantages and disadvantages of these ideas, what other possibilities might there be which I have missed, and tell me about your experience of using these things

This adds two further things into the mix which are likely to be received negatively:

• Answering your own question, with the suggested answer in the question. Except the answer isn't a proper answer; it's just some initial ideas for an answer
• Asking for "all possible solutions", which is a perfectly legitimate thing to want to to to be better informed and to make an informed decision, but it is absolutely not something you want to do if you want a question to be well received on Stack Overflow

In my opinion, this hints at a more serious underlying problem within the community. It is difficult to explain exactly what this is. I would characterize it as "being against learning". I do not know if this is because the rules and policies are set up that way (presumably inadvertently) or if this is just the culture which has developed over time.

If the Internet is not for learning, then what social good does it perform? Surely the most important purpose of the Internet is to facilitate access to information to enable education and learning.

I post quite regularly (primarily) on Stack Overflow. Many of my questions are downvoted. This is often for the reasons I describe above. Sometimes it may be because the question is poorly worded, or just not well thought out. This is not my concern. My concern is that Stack Overflow ceases to exist or ceases to become useful, because people simply opt for the ChatGPT alternative.

I think those who have been long time users of the site would agree the quality of information you can obtain from asking questions has declined over time. In recent years this has probably been most noticeable because of the appearance of ChatGPT as an alternative. However, the decline was notable before ChatGPT existed. I think that's because the rules about what can be asked became too rigid.

Why I am concerned by the possibility of Stack Overflow disappearing

Finally let me provide some comments on Stack Overflow vs. ChatGPT. I post these to point out that Stack Overflow has some real advantages which can never be replaced by a machine. (Real people use the site and interact with each other.)

Ways in which ChatGPT is better than Stack Overflow:

• ChatGPT is never rude to me. I can ask whatever questions I want, regardless of how badly thought-out, poorly formatted, worded and structured, and those questions do not get downvoted, deleted or closed
• Answers are (near) instantaneous. The latency is much lower

Ways in which Stack Overflow is better than ChatGPT:

• There is human oversight and accountability. Answers and critiqued and a good answer can be replaced by a better one. This creates a free marketplace of ideas where (in theory) competition results in the best possible answer arising given a long enough period of time
• Sometimes ChatGPT will give incorrect information, something which can often happen if it has misunderstood your prompt. Sometimes it goes around in circles. It never says "no" when sometimes it should (meaning "no what you are asking for is not possible"). However, these flaws are being gradually fixed over time and one day this may no longer be a concern
• Do we really want to become dependent on a machine? Especially one where the company who controls it can change the algorithms which could limit our ability to access information. It could become paywalled at any time, and there isn't any limit to how high the paywall price could be.

Answer 1

There's nothing stopping you from using both, and in some cases, I've found ChatGPT handy in adopting information from a post for my specific conditions. I've seen people do the reverse and ask why something they asked ChatGPT about didn't work... which leads me to the first point.

ChatGPT will not judge you, but as per an old IBMism...

An AI chatbot gives you an answer. It may not be correct. It will not follow up. It won't challenge you when you want to use a toaster in the bath.

The funny thing is, as designed, the broader Stack Exchange ecosystem covers for many of these things, but aspects of this have been left to moulder, or attempts made to reinvent the wheel, but in trendier shapes.

The culture of "small specific questions" as opposed to overarching broad architecture questions means that quite often questions and answers are reasonably scoped and are reusable knowledge artifacts. That there's separate sites for programming questions, software engineering, system administration and end user software and hardware use isn't an accident.

Essentially, there are better places to document long-form problems than a Q&A site, and while there were attempts made to cater for that (Documentation), it didn't work out.

Historically, chat was a great place to discuss ideas, and concepts and often lead to questions on the site. As long as you weren't there to just ask questions (main site please!) many of the more active chats might have gone "OOH SHINY! I MESSED WITH THIS IN MY HOMELAB!" and lead to a question. or go "Wait, why not just... pull the whole repo, that's the whole point of gitlab... all you need is a way to put the repo back into the Git repo..."

You'd need to find the right chat, and chat activity, and discoverability are not what they used to be.

I'd argue to an extent that SE's partially in the mess its in now because it neglected its core users - the sort of people who stick around, and hang out in chatrooms, for trying to pander to people and goals that had nothing to do with the community. History has had a few "low quality", less focused Q&A networks—Yahoo Answers comes to mind, that haven't had the same utility we've had. Ironically people want us to change from what makes us useful, because they find us useful. What we really need to do is get people to the parts of the network they need better.

Trying to be more 'like' ChatGPT or going with a more free-spirited forum-like model takes away the core advantages of the Q&A format: That you get a focused answer from someone who hopefully knows what they're doing.

I'd argue that that you're complaining about SO is part of the problem. There might be other sites or parts of the network that might serve your specific needs better than asking a question, but visibility is poor. Heck, on some sites, I'd ask for feedback on a potential question, within reason on their chat or meta, just to make sure I have the best chance of success.

I've also self answered for documentation purposes, and generally done well because the questions were interesting. Alternatively, I've asked a question and waited for an answer, so it would be documented. My most recent one seemed simple, but wasn't documented on the network and I found it useful. I do have a deep knowledge of what works on my site though, but self answers do not always mean downvotes.

We've a system that works and if we disappear, it's because the folks managing the place decided to chase a new shiny thing rather than pay attention to what they got. It's happened lots of times, and pretty recently too.

They are different tools for different situations. You don't generally use a hammer to tighten a nut, even if at a pinch a good wrench can be a hammer.