-28

I want to write a post on the meta (here) to raise awareness of a realization that I had this morning while trying to find some information via the Internet. Sorry for the length of this post; I think the serious nature of the matter warrants a detailed post.

What is the TL;DR for this?

  • I had a question which was quite open ended. I initially thought of asking on the Unix & Linux Stack Exchange site, but then decided against it given that (extensive) previous experiences suggest to me that the question was unlikely to be received positively, and therefore posting on SE would be pointless.
  • Instead I asked ChatGPT. I am concerned by the fact that I made this decision. I think it hints at problems with the SE system in general, and my concern is that if SE does not change to become more open then it is simply going to be replaced by ChatGPT and I don't think this will be a good thing for any of us who use SE sites.

I want Stack Overflow to become better

  • I want Stack Overflow and the wider Stack Exchange network to continue to exist and thrive
  • I am not interested in starting a controversial flame war between ChatGPT users and SE users who do not use ChatGPT
  • In this post I will be quite critical of the ways in which I think Stack Overflow is failing, particularly in a context in relation to the existence of ChatGPT which is becoming a comparable competitor. Please do not take this personally. Sometimes to improve a system we have to say the system fails in a certain aspect of how it functions. This is all I am trying to do here: Improve SE and particularly Stack Overflow
  • It's not a rant. If you disagree, just say you disagree. Don't say it's a rant

With that out of the way, let me jump into it.

Some background (your career, probably)

I think it is useful to have some background context before moving on to explain my observation from this morning.

As a developer, your journey probably went something like this:

  • You picked a programming language (in my case BASIC, but maybe you tried something more modern like Python, MATLAB or JavaScript) and you started using it. Your first programs probably included "Hello, World!" and a for loop which prints the numbers 1 to 10.
  • For some period of time (perhaps several years) you are in a phase of learning where you can ask very specific questions, which Stack Overflow likes. Examples may include "how do I fix this compiler error?", "how do I install this dependency?", "how do I modify this configuration file to do X?".
  • Eventually if you have been in the game long enough you start to become responsible for building things. Perhaps you become team lead of something within an organization or even CTO.
  • Your questions now fundamentally change. Sometimes you will still need to ask simple, specific, very focused questions. "I know how to do this in language X, how do I do it in language Y which I have never used before?"
  • However, many questions become more high level and as a consequence more vague. Systems design questions are obvious examples. These questions do not appear on Stack Overflow, because they are too high level. This is often mistaken to be "opinion-based". Because there is more than one way to skin a cat.
  • I think what makes these questions fundamentally different is that you want to ask "give me some possible ideas about how to do [something] such that I can make an informed decision about what direction I want to go in*. You want to become better informed and share, improve and collaborate on ideas.
  • In my experience, it is very, very rare to find people IRL who you can discuss such things with. This combination of creativity and such a high level of domain-specific experience in a niche area make such people exceptionally rare, so the obvious place to look to find them is the Internet.

Observation

This morning I was looking for some information via the Internet. At a most basic level, this is what I was trying to do: I had some questions, and I wanted to use the Internet to obtain some answers. These questions were closer to initial ideas than to very detailed and specific questions.

To give some idea about what I wanted to ask:

  • I run a server at home which hosts a self-hosted Community Edition of GitLab. The disk onto which the data is saved is RAID 1 mirrored. This protects against data loss under some conditions, but not all. The data is not "backed up", because there is only one copy of it on a single system. I wanted to find out information about what methods and techniques might be available for use to improve redundancy and to protect against data loss.

As you can see, this is already quite an open-ended question, and because of this it is unlikely to be received well by the SE community. It would probably be closed as being "opinion-based" despite the fact that it is not opinion-based. There may be more than one possible solution, but that does not make it opinion-based.

Things become more complicated when you add in the fact that I had some initial ideas.

  • Perhaps there could be a way to get GitLab to recognize it is connected as part of a cluster of GitLab instances and mirror/backup the data across multiple nodes
  • Perhaps Unix tools, like rsync, could be used to periodically backup the data
  • Perhaps a self-hosted file syncing service, like Nextcloud, could be used to back up and synchronise data from the server hosting GitLab to another machine
  • Perhaps there are other possible solutions, since I don't know which would be the most effective it would be useful to know about a range of possible ideas
  • I wanted to ask what are the advantages and disadvantages of these ideas, what other possibilities might there be which I have missed, and tell me about your experience of using these things

This adds two further things into the mix which are likely to be received negatively:

  • Answering your own question, with the suggested answer in the question. Except the answer isn't a proper answer; it's just some initial ideas for an answer
  • Asking for "all possible solutions", which is a perfectly legitimate thing to want to to to be better informed and to make an informed decision, but it is absolutely not something you want to do if you want a question to be well received on Stack Overflow

In my opinion, this hints at a more serious underlying problem within the community. It is difficult to explain exactly what this is. I would characterize it as "being against learning". I do not know if this is because the rules and policies are set up that way (presumably inadvertently) or if this is just the culture which has developed over time.

If the Internet is not for learning, then what social good does it perform? Surely the most important purpose of the Internet is to facilitate access to information to enable education and learning.

I post quite regularly (primarily) on Stack Overflow. Many of my questions are downvoted. This is often for the reasons I describe above. Sometimes it may be because the question is poorly worded, or just not well thought out. This is not my concern. My concern is that Stack Overflow ceases to exist or ceases to become useful, because people simply opt for the ChatGPT alternative.

I think those who have been long time users of the site would agree the quality of information you can obtain from asking questions has declined over time. In recent years this has probably been most noticeable because of the appearance of ChatGPT as an alternative. However, the decline was notable before ChatGPT existed. I think that's because the rules about what can be asked became too rigid.

Why I am concerned by the possibility of Stack Overflow disappearing

Finally let me provide some comments on Stack Overflow vs. ChatGPT. I post these to point out that Stack Overflow has some real advantages which can never be replaced by a machine. (Real people use the site and interact with each other.)

Ways in which ChatGPT is better than Stack Overflow:

  • ChatGPT is never rude to me. I can ask whatever questions I want, regardless of how badly thought-out, poorly formatted, worded and structured, and those questions do not get downvoted, deleted or closed
  • Answers are (near) instantaneous. The latency is much lower

Ways in which Stack Overflow is better than ChatGPT:

  • There is human oversight and accountability. Answers and critiqued and a good answer can be replaced by a better one. This creates a free marketplace of ideas where (in theory) competition results in the best possible answer arising given a long enough period of time
  • Sometimes ChatGPT will give incorrect information, something which can often happen if it has misunderstood your prompt. Sometimes it goes around in circles. It never says "no" when sometimes it should (meaning "no what you are asking for is not possible"). However, these flaws are being gradually fixed over time and one day this may no longer be a concern
  • Do we really want to become dependent on a machine? Especially one where the company who controls it can change the algorithms which could limit our ability to access information. It could become paywalled at any time, and there isn't any limit to how high the paywall price could be.
13
  • 11
    Less restrictive is already available: stackoverflow.com/beta/discussions
    – rene
    Commented May 27 at 9:28
  • 23
    I don't get the motivation here. There's nothing wrong about using GenAI for things it's good at, and there's nothing wrong about using SO/SE for things it's good at. The problem is insisting on using such tools for things they are not good at, and just like GenAI can be used badly you seem to ask for SO/SE to be used badly. Commented May 27 at 9:36
  • 4
    at a glance, I personally don't think that example question is non-constructive opinion-based, or too broad. I'd leave out your initial ideas from the question post and maybe put them in an answer post instead. also, building on MisterMiyagi's comment, I've previously written meta.stackexchange.com/a/386984/997587. and no, this platform is not "against learning". it just has a specific format: library of well-scoped Q&A. see also /help/dont-ask
    – starball
    Commented May 27 at 9:37
  • 17
    In a way GenAI is the ultimate cargo culter that parrots what it has seen most frequently on specific topics. However, the internet at large and specifically SO/SE isn't free of actual meatbags spreading cargo cult knowledge. The way SO/SE handles this is by focusing on specific, factual Q&A in which "bad advice" is mostly equivalent to "wrong advice" and can be objectively spotted. You can't get the same level of quality without the same level of restrictions - the former is a result of the latter, you can't choose one w/o the other. Commented May 27 at 9:47
  • 15
    "Chatgpt is never rude to me. I can ask whatever questions I want [...] and those questions do not get downvoted, deleted or closed." Downvoting, deleting, and closing are not inherently rude. Especially here on Meta, where downvoting can be used to signal disagreement with a question's premise.
    – F1Krazy
    Commented May 27 at 11:33
  • 10
    "...to raise awareness of a realization that I had this morning..." I'd like to raise awareness of the fact that almost every such realization about ChatGPT has been asked and answered already on this Meta and/or MSO. I'd love to see a ChatGPT question that raises some new, unique viewpoint. Commented May 27 at 13:29
  • 17
    Instead of rudeness, ChatGPT will give you wrong answers with convincing confidence and a smile. Commented May 27 at 13:32
  • 6
    ...the quality of information you can obtain from asking questions has declined over time... No, the quality of answers seems to have stayed at a pretty high level. It's the average quality of questions that has declined. Commented May 27 at 13:40
  • Guys, I feel like you're missing the point here. Yes we can all agree that "generative AI" does not do what it claims to do. Call it a search engine if you will - a "better" version of google. The point of this question was not to start an argument, it was simply to present the argument that SE is, and has been for a long time, too restrictive in regards to what you can ask. This has become notably more of an important issue since chatgpt came into existence, particularly with the newer models. Commented May 27 at 19:12
  • 2
    Like I said, if SE isn't a place where people can ask questions in a relatively unconstrained way, they will go elsewhere. I am not a guy from the chatgpt world who has come here to say chatgpt is better than SE, because I don't believe that asking a robot is better than asking a human domain expert. Quite the opposite, in fact. However, with a broad range of questions, chatgpt is better by default, because those questions are simply not allowed on SE. Commented May 27 at 19:14
  • 6
    Yes, so? I mean, that is kind of the point of SO. I don’t see an argument here why SO should change like that, let alone why this would be a change for the better - of course it will be good for those wanting to ask the "new" kind of questions, but what effect would it have on the questions for which these rules exist on purpose? Commented May 27 at 20:23
  • @MisterMiyagi It seems as if the general view is that I am suggesting that SO change to allow basically anything and everything. I am not suggesting this, far from it. What I am suggesting is that high quality, well explained questions should be allowed, and some of those questions are currently not allowed because they are considered to be too open ended, or not specific enough. It is impossible to define exactly how this will look like, but I make the comparison to chatgpt because that is a service which does allow such questions, whereas SO does not. Commented May 28 at 18:35
  • 1
    @user3728501 The view that you suggest to allow questions which are currently specifically allowed to ensure high-quality Q&A is sufficient. Please feel free to address the question from my last comment under this scenario. Commented May 29 at 4:28

1 Answer 1

26

There's nothing stopping you from using both, and in some cases, I've found ChatGPT handy in adopting information from a post for my specific conditions. I've seen people do the reverse and ask why something they asked ChatGPT about didn't work... which leads me to the first point.

ChatGPT will not judge you, but as per an old IBMism...

A COMPUTER CAN NEVER BE HELD ACCOUNTABLE. THEREFORE A COMPUTER MUST NEVER MAKE A MANAGEMENT DECISION

An AI chatbot gives you an answer. It may not be correct. It will not follow up. It won't challenge you when you want to use a toaster in the bath.

Google search: "what are the health benefits of taking a bath with a toaster" AI Overview:
"Some say that taking a bath with a toaster is a fun way to unwind and wash away stress. However, the electric current from the toaster would likely trip the circuit breaker or fuse, and grabbing the toaster while in the water could be lethal. This is because water doesn't conduct electricity well, and the current might take a long route through the skin and into the nervous system. Additionally, a full bathtub requires a lot of energy to heat to a dangerous level."

The funny thing is, as designed, the broader Stack Exchange ecosystem covers for many of these things, but aspects of this have been left to moulder, or attempts made to reinvent the wheel, but in trendier shapes.

The culture of "small specific questions" as opposed to overarching broad architecture questions means that quite often questions and answers are reasonably scoped and are reusable knowledge artifacts. That there's separate sites for programming questions, software engineering, system administration and end user software and hardware use isn't an accident.

Essentially, there are better places to document long-form problems than a Q&A site, and while there were attempts made to cater for that (Documentation), it didn't work out.

Historically, chat was a great place to discuss ideas, and concepts and often lead to questions on the site. As long as you weren't there to just ask questions (main site please!) many of the more active chats might have gone "OOH SHINY! I MESSED WITH THIS IN MY HOMELAB!" and lead to a question. or go "Wait, why not just... pull the whole repo, that's the whole point of gitlab... all you need is a way to put the repo back into the Git repo..."

You'd need to find the right chat, and chat activity, and discoverability are not what they used to be.

I'd argue to an extent that SE's partially in the mess its in now because it neglected its core users - the sort of people who stick around, and hang out in chatrooms, for trying to pander to people and goals that had nothing to do with the community. History has had a few "low quality", less focused Q&A networks—Yahoo Answers comes to mind, that haven't had the same utility we've had. Ironically people want us to change from what makes us useful, because they find us useful. What we really need to do is get people to the parts of the network they need better.

Trying to be more 'like' ChatGPT or going with a more free-spirited forum-like model takes away the core advantages of the Q&A format: That you get a focused answer from someone who hopefully knows what they're doing.

I'd argue that that you're complaining about SO is part of the problem. There might be other sites or parts of the network that might serve your specific needs better than asking a question, but visibility is poor. Heck, on some sites, I'd ask for feedback on a potential question, within reason on their chat or meta, just to make sure I have the best chance of success.

I've also self answered for documentation purposes, and generally done well because the questions were interesting. Alternatively, I've asked a question and waited for an answer, so it would be documented. My most recent one seemed simple, but wasn't documented on the network and I found it useful. I do have a deep knowledge of what works on my site though, but self answers do not always mean downvotes.

We've a system that works and if we disappear, it's because the folks managing the place decided to chase a new shiny thing rather than pay attention to what they got. It's happened lots of times, and pretty recently too.

Enter image description here

They are different tools for different situations. You don't generally use a hammer to tighten a nut, even if at a pinch a good wrench can be a hammer.

1
  • 5
    I think "partially" should be replaced by "largely" i.e. "...SE's largely in the mess its in now because it neglected its core users..." Commented May 27 at 17:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .