17

Recently, a new post summary design and greatest hits page design was rolled out (and is expected to be extended to all other such lists of questions everywhere). It was met with strong disapproval with a current score close to -600. In the days after the rollout, staff did tweak the design multiple times in order to react to user concerns with a current design that is closer to, but still with distinct differences from, the previous design.

Is there any change in usage patterns visible during these days? Did people ask, vote on, visit or answer less or more on new questions during this time? If there was a change, is the usage level now higher or lower than compared to before or simply back to normal?

Recently there were also some downtimes for the service due to DDOS attacks. Unfortunately they could bias the results of any analysis here. If possible the downtimes should be excluded from an analysis.

Improve this question
8
  • My guess is that there was a measurable dip in usage (for some days the landing page was nearly unusable) that however was only temporary and activity should now be back to normal. Whatever is the case, it might be possible to see it in SEDE.
    – NoDataDumpNoContribution
    Feb 8, 2022 at 9:44
  • 1
    The underlying assumption that lower usability leads to lower usage is not necessarily correct. As long as the contents and structure of SE sites is still better than for any of the competitors the sites will continue to see high visitor numbers and high participation. Especially since the new designs aren't that terrible - I very much dislike the recent changes but I am still able to find interesting questions to answer, perform reviews etc.
    – Marijn
    Feb 8, 2022 at 13:09
  • @Marijn In the question there is no assumption. I simply ask for an effect, whether it be positive, negative or not measurable at all. The assumption that lower usability does not influence usage at all is also not necessarily correct. The thing is: we rarely get such strongly disliked changes, so if even with a score of -600 we do not see any effect, how can we hope to ever see anything for smaller amplitudes of approval or disapproval? It's harder to argue that meta discussions are relevant if you cannot directly see a real life impact. There may still be one though but a much slower one.
    – NoDataDumpNoContribution
    Feb 8, 2022 at 13:22
  • 3
    If there is a dip in usage, I would expect it to be with power users. I would look for a decrease in the number of answers, flags, and closed questions, etc. People who arrive from Google search wouldn't even see the change, and people trying to ask a question wouldn't care.
    – Andrew Myers
    Feb 8, 2022 at 15:07
  • @AndrewMyers And the possible dip might be mostly related to handling of new questions. They got the least traffic from google and rely more on these question lists.
    – NoDataDumpNoContribution
    Feb 8, 2022 at 15:55
  • 1
    My point is that usage patterns may not be a good dimension to measure the success of design changes, or the relevance of a meta discussion. This goes in both directions, a lack of increase in visitor and post numbers does not mean that the design change has failed its goals. A design change is succesful if it reduces technical debt and if people like it, a meta discussion is relevant if the company engages in meaningful discussion beyond "this is the new design, deal with it".
    – Marijn
    Feb 8, 2022 at 16:12
  • In both cases I would say that the success is limited, but usage patterns do not factor into the amount of success.
    – Marijn
    Feb 8, 2022 at 16:14
  • @Marijn I know what you mean. You don't trust usage patterns. I also don't trust scores on meta questions. People might say one thing but do another. This question deals with what people do. For example I don't need to answer questions on SO. If I find the design of it so terrible I might simply use it less, maybe only for information retrieval and not creation anymore. In the end you have to take everything with a grain of salt.
    – NoDataDumpNoContribution
    Feb 8, 2022 at 19:26

1 Answer 1

Reset to default
11

I'm trying to give an answer using the site analytics - not those of Meta; they are notoriously skewed by events exactly like this design change. Here are votes and traffic data from Ask Different, and I don't see any change in the usual pattern (slow on weekends and sometimes a spike because of hot network questions). There is no doubt Stack Overflow is a better source, but I'm a little short of reputation there...

Enter image description here

The site analytics can't differentiate between old and new questions, and SEDE looks like the right tool for that job. Let's see when (not if) @rene finds some statistics to share...

Improve this answer
1
  • SO analytics look even more boring in shape of recent week compared with pre-change. There's a noticeable dip around 26 January not in other mid-weeks, but I've not done stats.
    – francescalus
    Feb 8, 2022 at 15:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .