14

Rerunning this query by starball for new questions and answers posted across the network generated the following graph:

Graph of questions and answers over time since 2018

It plots posts by their creation date, and includes deleted posts.

The decline in questions asked and answers given keeps continuing over the past year. What intrigues me is: what happened in April 2023? There's a sudden drop by roughly 20% of both questions and answers, with an otherwise continuing, linear trend. Fewer questions to answer could very well result in fewer answers, so what caused this drop in questions asked in an otherwise linear trend?

The yearly roomba which hasn't run would result in more posts being present since April 2023, due to them not having been deleted yet. Also, the 365 day roomba requires no answers on the question, whereas the drop in answers is of roughly equal magnitude.

I'd not expect the moderation strike to have a big influence either. First, it started in June 2023. Second, moderators (users and diamonds alike) aren't the biggest question asking group and besides, they close (and subsequently delete) questions. Lack of that action would result in more questions being present.

ChatGPT's launch on 30 Nov 2022 is well visible in the graph. Could the release of GPT-4 on 14 March 2023, which also included third party plugin access1, drive even more people from Stack Exchange towards LLMs?

35
  • 11
    Roomba deleted posts? There is a year rule that removes posts.
    – rene
    Commented Apr 3 at 7:11
  • 1
    @rene that's a year ago now, true. But wouldn't a lack of the yearly roomba action result in more posts present, rather than less (due to them not being deleted yet)? Besides, the 365-day Roomba requires no answers to be present, but the drop in answers is roughly the same as the drop in questions.
    – Adriaan
    Commented Apr 3 at 7:17
  • 2
    @rene The 365-day Roomba rule doesn't run on meta sites, including this one. Commented Apr 3 at 7:44
  • 4
    @SonictheAnonymousHedgehog even though the SEDE URL (and UI) contain Meta.SE, this is actually a cross-site query (and it doesn't matter on which site you run it, data.stackexchange.com/stackoverflow/query/1759268/… gives the same results)
    – Glorfindel Mod
    Commented Apr 3 at 8:57
  • 7
    (and the amount of posts on Meta is dwarfed by main site posts, so that shouldn't make a difference)
    – Glorfindel Mod
    Commented Apr 3 at 8:59
  • 4
    Why April 2023? There's a downwards trend from middle 2020 Commented Apr 3 at 9:49
  • 3
    @DanubianSailor That the network has been getting less traffic over the years (even from 2015 onward, depending on the statistic used) is well known. However, it has been more or less smooth for most of the time since 2020. April 2023, however, has a sudden drop of roughly 20% traffic, rather than the 3-5% drop-off of the current trend. This appears to be an isolated, one-off event.
    – Adriaan
    Commented Apr 3 at 9:51
  • 1
    I think I have seen this same graph somewhere else (probably looking at posting trends on SE after release of chatgpt), possibly its even the same query, so although not likely its possible it could just be the effect of chatgpt
    – user13267
    Commented Apr 3 at 10:20
  • 5
    Given the fluctuations in the plot, trying to tie April to something specific is likely unwarranted. Just an overall roll off of participation.
    – Jon Custer
    Commented Apr 3 at 14:08
  • 6
    I don't know that "total number of questions" or "total number of answers" can be influenced by any single factor or how we could even validate any of the factors we might blame. But you've left a few possibilities out of your consideration; for example, we make changes pretty much continuously to block out known spammers and other ill-doers, and sometimes this can impact a lot of garbage at once. It might be useful to include in your analysis how many of those posts in March/April of last year were actually useful (lasted more than a day, had x number of views, y number of votes, and so on).
    – Aaron Bertrand Staff
    Commented Apr 3 at 14:55
  • 7
    Also things like what % of the posts in March had flags, were closed, deleted, etc. compared to April. Just because there are a lot of questions doesn't mean they're all useful, and just because the total goes down doesn't mean there's any negative impact. In any case, you'd still only be theorizing - I really don't think this question can possibly have a verifiable answer or a useful outcome aside from stirring up more speculation.
    – Aaron Bertrand Staff
    Commented Apr 3 at 14:58
  • 4
    @Fe203 Not pretending anything. But it's also unhealthy - and certainly far from productive/constructive - to jump to conclusions that there could only possibly be a single elephant. I've suggested multiple other factors that can contribute to a decline in posts that you haven't even proven is a qualitative decline, it's just a number. And even if you could prove that you're right, which is impossible, then what? What is your goal? What kinds of factual answers do you expect? What new information do you expect that hasn't been covered before? meta.stackexchange.com/q/387278/165455
    – Aaron Bertrand Staff
    Commented Apr 3 at 20:52
  • 1
    The most pertinent sentence from @AaronBertrand's link seems to be "Conversely, in April of this year, we saw an above average traffic decrease (~14%), which we can likely attribute to developers trying GPT-4 after it was released in March."
    – ronno
    Commented Apr 4 at 8:27
  • 2
    I'd be curious if we are seeing a trend in "easy" or homework questions seeing the largest downward trend, vs more "complex" questions remaining more constant. Not sure how to define those, but I could see that more users could be finding it more efficient to get easy/homework questions from their gen-AI tools, rather than SO.
    – emmabee Staff
    Commented Apr 4 at 19:01
  • 3
    @starball "if one could somehow understand why these people left, it could better inform plans for how to stem the tide of "stagnating participation" - wait, if we admit that the "stagnation" is caused by people who left, then we can also assume that those were pasts the 15 rep "wall of doom" that they plan to remove. Removing the entry barrier only makes sense if they think the "stagnation" is caused by new users not joining, not by "old 15+ rep users alienated , feed up and leaving". Sadly, I don't even think they see old users leaving as an issue, they just want NEW users to join
    – SPArcheon
    Commented Apr 5 at 8:07

1 Answer 1

-7

Speaking from an "SO user" perspective: now that (free) ChatGPT (since Nov '22) can both write and critique code without judgmental overtones, and word of its capabilities has spread, users - especially newbies - have gone where they feel their predicament is being met with greater understanding and patience (imagine that!).

(Still with SO perspective:) Used to be 'n' questions each week about Harvard's CS50 course exercises. Now such questions are extremely rare... SO tends to reject fumbling questions from beginners

Question incomplete or ill-composed and requires revision. Question has been Closed!

On the other hand, the LLM's do their best to help the neophyte (with infinite patience.)


The (not to be named) new kid on the block arrived in Nov 2022.
In March 2023, that same new kid had a version upgrade from 3.5 to 4.0.

These dates appear to coincide with the months showing significant dips in the OP's graph.

Then, again, this may be only a coincidence...


I'm "Singing in the Rain" :-)

The movie of that title depicts the upheaval felt by a silent film studio and its actors when "Talkies" began to tap-into a significant portion their audience (who had previously had few alternatives.) After some years, "colour" enhanced the consumer's experience.

Evolution is known to manifest itself in gradual change, and, on occasion, seemingly radical leaps-and-bounds.

The analogy is (imho) poignant. The unpopularity of the observations made here, regardless of their perspective, does not bode well for the future of the site.


In light of a shocking revelation in another thread here on Meta-SE, I re-visit this answer and its observation that the dips in the OP's graph show a strong correlation with public availability/interest in AI generated content.

Seems no-one at SO noticed a particular user suddenly tripling their baseline rewards when GPT-4 came online. I wonder what the statistics for the volume of their SO contributions would show.

It's a bitter pill to swallow, but the problem will only fester if not addressed.

10
  • 3
    @Adriaan The question seems to be: "What happened in April '23?" My supposition is that the graph of ChatGPT usage would show the opposite trend to the one presented above (on the same timeline)... for the reasons presented in this (admitted) rant. A new player is in the game, but it is unwelcome (banned) to sit at the SO table. When did ChatGPT "gain traction"? When did SO impose the ban on AI generated content? SO has been judged "toxic" for a long time. What has SO done to improve its public image in the face of a new competitor?
    – Fe2O3
    Commented Apr 3 at 9:37
  • 2
    Your answer showed up in the Low Quality queue, and I VTLOed, but can you edit out that ranty parts and, especially, the parts that seem to be feature requests? You can turn that into a question of its own, perhaps, but it seems tangential at best here.
    – Joachim
    Commented Apr 3 at 10:02
  • 4
    This is not an answer to the question. The question is not about the decline, but about the hiccup in the decline in April.
    – nvoigt
    Commented Apr 3 at 10:07
  • 5
    This is a bad-faith answer. The goal here is a sour grapes rant using the question as a mere pretext for the rant. Commented Apr 4 at 15:38
  • @PresidentJamesK.Polk I admit that the "1st edition" did venture a bit too far "into the weeds". The question asks "What happened in April?" Examine the edit history of the question. Credit has been given to my account as the first one to relate the arrival-of and update-to a new competitor to Stackoverflow. Dismissing uncomfortable feedback, rather than considering it, merely demonstrates an attitude of infallibility.
    – Fe2O3
    Commented Apr 4 at 21:19
  • 1
    @nvoigt Version 1 of this answer was too "in-depth". I admit that. It was the first and only "answer" to mention ChatGPT's release in 11/2022. It failed to mention the release of "version 4.0" in 03/2023 (which I mistakenly took to be "common knowledge".)
    – Fe2O3
    Commented Apr 4 at 21:25
  • the start of your post lines is to the effect of meta.stackexchange.com/a/384378/997587. but basically, you think that's a bad thing for SO and I think much less like that than I used to.
    – starball
    Commented Apr 5 at 1:57
  • 1
    @starball (Trying to avoid sliding back into "rant" mode...) The challenge of assessing the "quality" of a question, OP being new to the subject or new to this "forum", is judged by whoever happens to read a question and choose to (or to not) vote on it. Hi-rep old-timers lose vanishingly little by DV'ing a low-rep "newbie's" stumbling Q or A. Alliances & rivalries (& apathy & sock puppets) exercising vendettas does not serve the lofty ideals of SE/SO becoming a repository of quality Q&A. People do not like to be criticised or corrected. Chill LLM's do not have that frailty. .../2
    – Fe2O3
    Commented Apr 5 at 2:46
  • @starball .../ It is my contention that unaddressed and detrimental ego-stoking behaviours have been tolerated on SO (the tiny window I frequent) in the absence of a plausible competitor. The recent arrival (and update of) ChatGPT is coincident with the data shown in the graph above. If it looks like a duck and it quacks like a duck... Being "out-evolved" is nothing new...
    – Fe2O3
    Commented Apr 5 at 2:46
  • @starball The over-used term "disruptive" does, on this occasion, justify its use (imo). What is it that experiences disruption? (my opinion only) Those institutions that seek stability, eschew change and wantonly disregard feedback. "Survivorship Bias." It's incredibly difficult to survey those who've already severed ties, taking their business elsewhere, to ask them "Why?".
    – Fe2O3
    Commented Apr 5 at 3:00

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .