16

There has been some turmoil lately, and many highly active users, moderators and community moderators have stepped down.

I'm wondering how/if this has affected the activity on various SE-sites (and SO in particular). What has happened to the question/day rate, answers/question, votes/post, number of users answering questions etc.

Is there any available information regarding this? Are there any clear trends? Does it at all affect the overall network activity?

4

3 Answers 3

13

Executing the data-explorer query originally made by Jon Ericson in 2017 (see How is question quality measured in A/B tests?), where he categorizes the questions as:

  • good (not-closed and with a positive score or with score zero but with answers)
  • neutral (not-closed, score 0, no answers)
  • negative (closed or negative score)

and looks at the relative evolution, i.e. the time dependent behavior of the number of new such question as percentage of total new questions.

The graph for the last four years for Stack Overflow looks like:

Enter image description here

The x-axis is time and the y-axis is percentage of good/neutral/bad questions asked at a certain time. I marked some features. Christmases (red vertical lines) show a relative drop in neutral questions and an increase in bad questions. Either people ask considerably worse questions around Christmas or reviewing activity is increased relatively around that time or both.

Furthermore, there is a first yellow box around August 2019, which marks the 3-close votes experiment on Stack Overflow (bad questions increased and neutral as well as good questions dropped) as well as the second yellow box since December 2019, which is the last months or so, which is influenced heavily by the now permanent 3-close votes condition as well as Christmas as well as the Pronoun/Monica/Shog developments. The last two months are difficult to interpret and I would prefer to wait a couple of months more and see how activity continues.

And there is the green box, which is September 2019 to December 2019, which indeed sees a significant but not very big change in these quantities. In that green box, I would say, the relative number of good questions has decreased even faster than the long term trend, the number of neutral question also increased to a four year high (somewhere in October/November). There might have been a dip in review activity of new questions, which however was more than compensated by the change to 3 close votes in December 2019.

There is the long term trend of fewer and fewer questions with positive scores or answers that score positively on Stack Overflow (see also Why is the rate of positively scoring questions and answers steadily declining?). This long term trend seems to be intact.

I did similar computations also for a couple of other Stack Exchange sites and I think I remember a similar behavior for Mathematics at the very least. Some people describe this as "mission accomplished", where most of the most important questions in a field that are suitable for a Q&A like here have been asked and answered.

The fluctuations in these graphs are large enough that one cannot say anything else with certainty, and the effect of the September 2019 turmoil may only be temporary and not lasting, but if anything it has accelerated the long term trend, I would say.

3
  • 2
    So what is on the y-axis and what is the unit? %? Commented Jan 24, 2020 at 0:40
  • 2
    @PeterMortensen Percentage of all questions asked on that day.
    – Selcuk
    Commented Jan 24, 2020 at 3:28
  • @PeterMortensen Selcuk is right. Edited the answer. Thanks for editing. Commented Jan 24, 2020 at 6:13
8

On SO (in particular) there was a drop in activity during the end-of-year holidays.

The number of posts now is the virtually the same as last year -- now 18000 posts per week compared to what looks like an average of about 19000 for the whole of last year, and identical to this month last year.

The ratio of questions-to-answers doesn't seem to have changed.

I don't know about page views, visitors, etc.

That number (i.e. posts per week on SO) has been trending downwards very gradually and smoothly, over the years -- for example it looks like it was about 23000 in 2015.

Which might be insignificant -- i.e. it's the same order of magnitude, and there are probably more important metrics than the number of posts per week.

Anyway I see no evident recent change at all to that metric -- and thus nothing attributable to "some turmoil lately" -- who knows, perhaps that story is different on other sites, or there's some other metric where it is visible.

10
  • 4
    Especially on stackoverflow, I never expected to see raw question/answer counts to change significantly. The number of "do my homework plz" requests is not going to change because people here on MSE or MSO grow grudges about that company running the servers. The real impacts should be on quality, and reduced efforts of user based moderation. Which could be identified with SEDE queries, too I guess ... assuming that one spends the time to think them up and write them down.
    – GhostCat
    Commented Jan 20, 2020 at 11:54
  • I think I've seen it suggested that some of the people who are the most active on SO aren't the people who are most active on MSE. I'd be slightly curious to see how the Pareto principle applies -- to what extent is beneficial activity attributable to a minority of users. In the 1990s at least (I don't know about now) MS tried to identify some of those users and designated them MVPs.
    – ChrisW
    Commented Jan 20, 2020 at 12:05
  • First of all, note that there is meta SE and meta SO. A lot of people who only worry about stackoverflow will rarely show up here, and vice versa, people who come from smaller communities on the SE network ... might rarely turn to MSO. And yes: there are plenty of different "user groups". And a lot of the "high profile" content creators on stackoverflow rarely turn to the meta sites, if at all, to talk about problems about "content creation". But I think it is safe to say: those who are active here and on MSO, those are the ones that do a lot of the "user moderation" work that gets done.
    – GhostCat
    Commented Jan 20, 2020 at 12:12
  • 1
    That sounds plausible -- that people who are into Meta are also into the review queues, and possibly closing questions and so on -- i.e. trying to edit for quality even if not (also, as well as) actually posting questions and answers -- and I guess that's what you meant by "user moderation" (given the theory that the site moderators, be they elected volunteers or SE employees, should do as little as possible).
    – ChrisW
    Commented Jan 20, 2020 at 12:17
  • @GhostCatsalutesMonicaC. It's a problem which may be slightly orthogonal to SE's income. If their revenue is from selling Teams to Enterprises, for example, I imagine that Enterprises moderate their private sites in a slightly different way, i.e. that it may be their salaried employees who are doing it all there. So what you're talking about -- i.e. people here on MSE or MSO grow grudges about that company -- is I guess a concern of the "public Q+A" aspect of the business, which, I see as being mostly or entirely pro bono.
    – ChrisW
    Commented Jan 20, 2020 at 12:30
  • @GhostCatsalutesMonicaC. So I wondered, what metric would measure "quality" (of a site)? And perhaps it's the percentage of not-"unanswered" topics -- i.e. of questions which aren't closed and which have some upvoted and/or accepted answer[s]. That's the point of having "communities", apparently -- i.e. it's so that when you post a question, it is answered.
    – ChrisW
    Commented Jan 20, 2020 at 12:42
  • @ChrisW One of the problems with lack of content curators is explained here meta.stackoverflow.com/a/393004/4267244 It is not just about having questions answered, but ability to find answers to your problems without having to ask another question - unless you are completely clueless, that works way faster than asking question and waiting for an answer. If you have to dig through lot of crap to find gems, then the place is of very little use. SO succeed because of the quality... you would Google and one of top results was usually some SO Q/A pair where you would find your answer 1/2 Commented Jan 20, 2020 at 12:48
  • Nowadays, SO Q/A pairs are still mostly on the top, but you will not find only one but many... and most of them will be rather poor... I am more and more relying on gluing up pieces from less than stellar official documentation (insert your favorite tool/language) than on SO. That is quite different from what I was doing just few years back when SO was the fastest way for finding proper solutions. 2/2 Commented Jan 20, 2020 at 12:51
  • 2
    Number of close votes have increased a lot (probably result of the increased weight of a single close vote). Votes and views for new posts are going down (that's actually not easy to estimate). Commented Jan 20, 2020 at 13:05
  • 1
    @ChrisW it is about difference when you have to Google (because documentation is lacking complete information, examples...) and find one or two links comparing to 100. And it is not even about having 100 links... it is about those links not having proper solution... This is SO, now... millions of questions and yet very few have good solutions and many questions overlap, but are not closed as duplicates. Commented Jan 20, 2020 at 13:35
2

Of the sites where I've got access to analytics:

  • Traffic on History is about half of peak, with most of the decline being in the past six months. Posts are about two-thirds of peak, with the decline being more gradual.
  • Traffic on Information Security is about two-thirds of peak, with the decline starting about ten months ago. Posts are about half of peak, with most of the decline having taken place in the 2016-2017 time frame.
  • Traffic and posts on Space Exploration both jumped with the first flight of Falcon Heavy and have remained consistently elevated since.
  • Traffic on Retrocomputing shows no long-term trend.
2
  • Do we know whether the pageviews are comparable across years? It could be that at some point the identification of bots changed, or whatever.
    – Nemo
    Commented Jan 24, 2020 at 15:49
  • 1
    @Nemo, I'd expect changes in tracking criteria to show up as jumps in the data, and I'm not seeing any.
    – Mark
    Commented Jan 24, 2020 at 20:52

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .