Post-mortem or root cause analysis of the recent issues (late 2019-early 2020)

Question

Teresa Dietrich recently posted regarding her and the company's efforts to restore community trust and repair the recent unpleasant issues.

Now that the company is making at least some meaningful effort to apologize and move on from the late unpleasantness, I thought it was a good time to do a post-mortem or root cause analysis on why things went so spectacularly wrong.

From about September 2019 to early 2020, a few issues hit the network in short succession, including

• The pronouns controversy
• Ads attempting to start audio
• Controversial firings
• The increase in question upvote reputation
• Content licensing changes

The result of this was a lot of counterproductive behavior that escalated as time went on. Different camps circled their wagons and pointed fingers in every direction, and staff retreated from regular, open communication to post occasional legal disclaimers and/or similar low-content or not-the-most-kind messages.

In my experience, poorly-managed systems tend to work okay on a day-to-day basis when everything is awesome, but, like a bridge that stands in good weather but collapses in a thunderstorm, tend to go boom when placed under stress. This is what I think happened last fall - the systems we had in place were robust enough to handle one or two controversies, but not all of them.

At this point, I'm no longer convinced that the problems were the result of a single "problem" employee who could simply be fired to solve the problem, and I recognize that sitting around attempting to assign moral or legal culpability tends to hurt more than help. What I am interested in exploring is why the system failed - not why the underlying issues (e.g. pronouns, firings) happened, but why the "system" so spectacularly failed when those issues proved to be controversial. With this information, we can help prevent the problem in the future. As the old maxim states, those who don't understand history are doomed to repeat it.

So,

• Why did the system fail so spectacularly when placed under stress?
• What can the community and the company do now so that future controversies (which will happen) do not cause a similar collapse?

Answer 1

It didn't start in September 19 and it's not over yet.

At least for the public programming Q&A on stackoverflow.com there are unsolved problems for years. Users have bad experiences and quality of most of the questions and to some extent answers too is questionable. Huge constant cleanup efforts are necessary to keep the noise down. Less and less new questions achieve a positive score. A very stable, long-term downward trend of quality and lately also number of questions, answers, active users happened.

The company could have done more to improve for example in guiding new users but also the community didn't really find a good, easy solution. Maybe it's really a hard problem to maintain Q&A knowledge bases. And it may not be very profitable in terms of revenue if you do not own the knowledge.

The company owns the software though and sells it and that is good.

At some point, maybe around 2018, maybe earlier, the company stopped seeing the meta community as significant and equal partner and started to do their own thing. That has been a mistake, but well, technically we are on their premises and they have the final say on everything that happens here. They could simply dissolve meta if they wanted to and it even looked a bit like it last year. I guess they were quite surprised when lots of users who knew Monica only faintly declared their fervent support for her. And they were unlucky with the communication when for example they compared firing a moderator with shipping a product.

They had shown the cold shoulder to the meta community, so the community rebelled in form of downvote orgies, curation strikes, moderator resignations, money collections, open letters and talking about alternatives. That seems to finally have made an impact.

Now, but this is far from over yet, it looks like the company, at least that's what Teresa Dietrich said, goes back a bit and tries to work it out.

But that won't be easy. There is not much goodwill left, there are less employees in the community team than before, the community mistrusts the company, there are unsolved issues and the big problem of how to continue with the Q&A remains.

The company, which is in for the money, and the community, which is in for itself, need to find a way to profit from each other. Some kind of win-win is needed. Maybe the Stack Overflow mission can give some needed guidance...

... stops typing, starts crying.

Answer 2

The biggest issue I had was with transparency. I still feel in the dark, despite many meta posts and frequent discussions on the matter, many of which were addressed by staff members.

When Monica was unceremoniously fired, it took a lot of piecing together of meta posts to eventually get an idea of what happened, and even then, many of us still don't know the details. Since it went legal, we never truly will. To this day, many users still call for her reinstatement, despite her leaving the network.

When Shog9 and Robert Cartaino were released, we had no idea why. We still have no true idea why. We have conjecture, and a few tweets to give us an idea, but the whole thing is still very blurry.

I recognize that, for legal reasons, some information must remain confidential, but we built a bond with these people. That can't be swept under the rug, it has to at least be talked about.

This lack of transparency and series of extremely questionable actions threw meta into wild discussion, with many resigning as moderator and/or leaving SE altogether.

You ask:

What can the community and the company do now so that future controversies (which will happen) do not cause a similar collapse?

The community can continue to hold SE accountable and continue asking for as much information as we're allowed to have regarding future changes. They can also accept that some information must remain confidential, whether it be for legal reasons or otherwise.

The company can indulge our requests for more information where possible, and continue to update us on their progress.

From the recent announcement by Teresa Dietrich, one of the highlighted points was transparency:

I want to start by establishing transparency with the community, and I know that transparency is an easy word to say but harder to define and put into action.

I couldn't agree more. Transparency is a result of frequent, open communication with the community at large, and that's very time consuming - but it's very important that the involved community remains informed about what's happening to the site they contribute so much to.

Teresa doesn't dive too deep into what ways the community team is going to increase transparency (because, as she said before, it's difficult to define), though she does note:

By being increasingly transparent we aim to regain trust with everyone - from brand new users to you, our most dedicated community members.

I certainly hope she's right. I certainly hope that we're all on the same page when the next big thing happens.

Answer 3

Why did the system fail so spectacularly when placed under stress?

First of all, who says that there is a "system" designed to withstand specific sorts of stress? Meaning: what makes you think that in 2019 there was still a "requirements designed system in place"? And what would those requirements be?!

Excursus: I recently learned about antifragile architectures. One key element there: to differentiate between things that are complicated (like a car that consists of thousands of parts, but all of it can be dealt with scientifically, and you can predict what happens), compared to complexity (like: human beings, that can't be predicted/approached by science to a large degree).

Why I mention antifragile: one of its core elements is the idea that in a complex environment, you have no idea about the incoming stressors. Meaning: you can't prepare for everything. But: when you prepare for "enough" different stressors, you gain the ability to handle other, unplanned stressors, too. So: when enough people survive the black plague, that leaves the remaining ones stronger.

So maybe, just maybe, somehow "we" survived, and SE Inc. survived, too. And that experience alone (and the will to learn from it) is an essential outcome here.

What can the community and the company do now so that future controversies (which will happen) do not cause a similar collapse?

The company can walk the talk. Teresa set a very positive tune, now the next weeks and months have to show that SE Inc. is willing to follow up their bold claims. For me, it is mostly about transparency. Tell us what you intend to do, and why. And then be open for feedback.

The community on the other hand: should accept that this is a process, that will take time, and that a misstep here or there isn't necessarily malicious, but maybe just a human error. We also have to understand that this is always about aligning different interests. They want to make money from a business, we want a place to host high quality content. That is again, a complex setup, and not necessarily all conflicts can be resolved in a way that makes everybody happy.

Of course, it is on us to hold them accountable. They promised, now they better deliver. If not, then it is really time for the final step and walk out.

Finally, for me, the most important aspect is: the feeling of not being heard. That is what drives people mad. If users give feedback, and there is no visible reaction to that, that kills trust. That kills the community. As said: as long as the company is open and transparent about their actions, even a statement like "we heard you, but for this or that, we will do something different" can be acceptable. As long as your efforts to listen to us are sincere.

Answer 4

From my perspective, watching the entire thing erode and collapse the moment I started wanting to get involved, these problems stemmed from a single root: Lack of communication.

It is not an uncommon issue among places like this, especially when you add layers that separate communities and businesses.

Not having procedures and policies in place that seem like common sense in hindsight certainly didn't help, but when it comes down to the wire the issues look largely related to the company coming in and saying that these changes are and there was no room to debate. They stood fast and offered very shallow responses void of any semblance of willingness to work with the community. Then when the responses did come that showed willingness, they were hollow and without action.

When either side of the equation refuses to listen or even appears to not be listening, negotiations breakdown. The moment people realized that what they were expressing was falling on deaf ears is the moment when the trust began to erode. It seems that the company has recently found some evidence to listen, and that things might have started to move forward.

Preventing this from happening in the future will require communication, both ways. It will require that the company willingly work with the community and that the community do the same with the company. I can say that I am at least hopeful that appears to be where the company is headed, but it is very early in the process to be confident. It will also take time for the community to trust the company again.

And by working with the community, I don't mean sending out surveys to people to gauge how welcome they feel. I mean bringing changes to the tables of users who are active and who built the office the company works in. I have seen it happen here before, and would love to see it continue to happen. All sides need to be able to communicate - both talk and listen. It at least appears now that the company has started to see this. The coming months will lay the foundation of the future of SO.

It also must be realized that the company is a company that requires profits and that not everything they do will be wanted or accepted by the community. The difference between an explosion of drama and irritated huffs is most often honest communication.

Answer 5

The SE sites have no obvious revenue stream, but also they are not owned by the users nor by a non-profit organization like Wikimedia. (So it's not a social network site like Facebook, nor a social news site like reddit, which have more obvious ways of monetizing user data for advertising.)

So while, for the user benefits, SE should be run solely to improve the site for the user value, instead it is run by an organization struggling to find a profitable business strategy.

This necessarily leads to internal conflicts inside SE Inc. between individuals who see SE as a product for the common good and individuals who want to grow the business of SE this or that way.

As no power is given to SE users in the decision making, SE community opinion remains a burden and a nuisance in the internal conflicts inside SE Inc.

As long as the business, power and ownership structure of SE remains as it is, conflicts will happen again, with community will being considered last, after the fact, and users will have no other way to react than to turn their back to the site in a tantrum.

Ways out of this would be to

• Find an obvious single business strategy that leads to growth, and a user group that's okay with it
• Hand over SE to a non-profit organization similar to Wikimedia
• Sell SE to a much bigger company such as Google to be run e.g. like Google groups (and this is a nightmare of course)
• Find ways to give much more power to the user community in all business decisions. Not just "involvement", but power to veto and decide modifications and budgets

The last suggestion is unusual for internet businesses, and I cannot think of another example at this scale, other than maybe sites (homepages, forums) run for political/religious groups.

Answer 6

Why did the system fail so spectacularly when placed under stress?

In each case one or more persons had an idea, sometimes a bit of thought went into it and sometimes a lot - then the person or group tried out their idea, (put it forth to a different group of people not previously consulted), and waited to see how it would be received.

Sometimes it (the idea / change / statement / etc.) was well received, usually not.

What can the community and the company do now so that future controversies (which will happen) do not cause a similar collapse?

The answer put forward is the-loop, but oval or even tesseract might be a more apt description as there's a few layers and there has been some stops and starts, some bumps along the way.

While the "question": "Will the composition of the two work groups mentioned in The Loop blog post be public?" doesn't seem like a "duplicate" of this question the answer (Dec 6 '19, yes not so recent) offered by Catija does provide you an answer: "At this point, we don't know. ... We will be offering a foundation for them and making suggestions but selecting who sits in the group won't be dictated by staff. ...", basically it's the-loop.

Things are being developed, discussed, 'checked and checked again'™, etc.

This time, when there's a release, they want the idea to float.

There has been previous prodding for updates, see: Can we have the dataset for the loop #2?, Please consider gathering Meta feedback before the "ship" stage, the-loop. It's looking like 6-8 months, that's an enhancement over doing things without the proper thought and care.