Why we initiated the latest round of design changes and the role of meta

Question

Now seems like a good time to remind ourselves of why we started making dramatic changes to the design of every site on the network. From Joe's post on meta:

We're introducing a new product, Teams (née channels), and doing the requisite research and design thinking to get it right. In addition, early on we realized that we have a unique opportunity to pay off technical debt that prevented us from serving Stack Exchange communities as well as we should have been.

There are something like 60 similar, but not compatible themes plus ~100 sites that share the "beta blue" theme. In 2016, our designers pushed out one design a month in an effort to catch up with the rate sites were graduating. In 2017, we had zero new designs. We've painted ourselves into a corner and need to give our designers some space to work with. We never again want to utter the words:

It turns out that when you try to pick a color to match 40 different site designs, you quickly realize you only have one real choice: black.

To get an idea of how the standard theme is implemented, you can look at the CSS and pattern library documentation. (We've never had design documentation in the past, in part, because there is so much variation in site design.) Standard design frameworks have proved an effective way to squash long-standing design bugs in the past and I expect the standard theme will eliminate many existing problems and prevent many more in the future.

Giving sites customized designs is a big part in our ongoing effort to rationalize the site lifecycle. (Opening more sites to moderator elections is another piece to the puzzle.) Unique community designs (and, by extention, brands) is one way we can invest in the Stack Exchange network. Not so obvious right now, but a standard theme also allows us to port Stack Overflow-exclusive features to other sites; new features that target Stack Overflow, Enterprise or Teams will probably stay locked into those platforms as long as we don't have a common design to target.

I would have liked these changes to be rolled out with the custom question list feature. That would have given everyone something to like about the changes other than a promise of fewer design bugs in the future. I know this state of affairs is not very satisfying.

How we get feedback

Reading the responses to What does constructive criticism of a design change look like?, it occured to me that people might not understand where meta feedback is most valuable. Meta was started, in part, due to my suggestion on UserVoice:

I know this has been declined multiple times, but I really think it's time to consider the problem of meta-discussions on the site.

In those day, nearly a decade ago, Stack Overflow was growing quickly and active users were an important source of ideas and feature requests. But, as you can see from the quote, most suggestions were declined. Usually they were declined with no explanation or terse pronouncements. Meta, by contrast, has turned out to be incredibly useful for working out certain types of features like the review queue indicator. It gives developers and users space to discuss the workings of the site in detail. Reasoned arguments on meta provide feedback that can't be reproduced elsewhere.

Unfortunately, we haven't always used that space to its best advantage. Jeff has talked about this in Listen to Your Community, But Don't Let Them Tell You What to Do. In contrast to his habit of declining suggestions quickly, we've swung to the opposite extreme. Ideas languish on meta. Once in a while, we'll dig up an old idea, but more often your hard work convincing us to do something goes unanswered. That doesn't mean it lacks value, however. When I wrote that UserVoice suggestion, I just wanted to have a place for meta-discussions even if the company ignored them.

In the last couple of years, we've cultivated other types of feedback. For instance, the blog post that introduced the navigation changes incorporated feedback from user interviews. From these tests, we can observe what happens when people try to use the interface. Direct feedback as people learn about the changes can only be reproduced on meta if the user is very diligent to record their own experiences in detail. Even then, it'd be just one of the ~5 individuals you need for good results. Since we do user interviews more frequently than in the past, we often have a good idea of what does or does not work in the UI before presenting it to meta.

We also use A/B tests and usage statistics. In the case of the left-hand sidebar, it's easy to spot when we made the change on Stack Overflow:

Meta feedback such as "I have a hard time finding the questions page" is especially useful if the data shows this is a common problem. If your concern happens to be less common, it requires more information about what makes your use case notable. Even then, it might be impossible to fix without breaking something for others.

By the way, the other sites that have received the new theme show a pattern unlike Stack Overflow:

Since Stack Overflow is a critical part of many programmers' workflow, we get criticism from outside the community. Traditionally, we've ignored that input because people who are not active on the sites tend not to understand how the sausage is made. Recently, however, we've heard criticism that we've taken to heart and responded to. If you can find the truth buried within misunderstandings of outsiders it can be useful to hear from the largely-silent majority of people who use the site passively.

Finally, there are internal considerations such as input from other parts of the business. Adding navigation support for Teams was a key reason we embarked on this round of changes, if you recall. We strive for transparency, but don't always succeed. Then there are trivial issues like the way our code is structured or people going on vacation or disagreements about how to do things that can have outsized effects on what we are able to do. Sometimes we fail to predict what information is most relevant.

Working with you to improve the feedback cycle

The sense I get from reading answers and comments here is that the frustration many of you feel stems from a lack of agency over the features, design and operation of the site(s) you love. The express goal of the standard theme is to support Teams, a feature many people will never get to use and probably will never be deployed to most sites. I'd guess everyone who uses meta regularly has a pet feature or bug that they wish we'd spend time on.

In addition, while the change will help sites without designs get their own distinctive brand in time, the predominate effect so far has been to water down existing site designs. Since Stack Overflow was the base upon which the standard was built, sites like Math and TeX look a lot more like Stack Overflow than they used to. And let's not underestimate the very real cost changing a UI has on people. It should be no surprise that people feel angry and helpless.

Both the DAG and community teams are working on ways to make the transition less shocking and get important feedback sooner as we continue converting sites to the standard theme. To quote Tim:

We're still a growing and relatively small company with limited resources. While we don't struggle and thrash around in the name of profitability, we're still going to struggle and thrash around as we encounter even scarier problems that come with becoming more established. We're way more public and open than anyone else and as you can see, it has been a heck of a ride. We wish we could do everything, right now, and make everyone happy. We'll get there.

You know how companies that put you on hold always say "Your call is very important to us"? Yeah, I don't believe it either. Then again, you gotta feel for them since many companies do care about keeping customers happy, but can't afford to have a call center full of people waiting around to give individualized service. So they gotta say something while you wait. Fortunately, as a software company, we can do more than ask you to reboot your router (as my ISP invariably asks). Sometimes we can fix things twice and get at the root problem.

At any rate, we can do better and maybe you have some practical ideas. How can we serve you better while still meeting our goal to standardize all 174 site themes?

Answer 1

It seems to me that the timing of all of this is just deeply unfortunate.

In my some three year experience with Stack Exchange as a whole, new features had to be fought over. You had to have broad community support, a highly upvoted meta post, a sharp and unassailable argument and business case, and quite a bit of luck to get your feature implemented. It was a unforgiving process that frustrated many people because it seemed like nothing was getting done, but it felt fair, because everyone and their pet feature request had to go through it. Now

Since Stack Overflow is a critical part of many programmers' workflow, we get criticism from outside the community. Traditionally, we've ignored that input because people who are not active on the sites tend not to understand how the sausage is made. Recently, however, we've heard criticism that we've taken to heart and responded to. If you can find the truth buried within misunderstandings of outsiders it can be useful to hear from the largely-silent majority of people who use the site passively.

That timing is incredibly unfortunate. I don't assume any malice of intent, I think y'all heard criticism you've really taken to heart and I'm not about to try and discredit that criticism or its validity in any way, but after years of being utterly unapproachable in the way of letting public suggestions affect large changes, you start to roll out a bunch of them. Not because the community asked for it (actually despite large contingents of the community asking you not to do it), but because outsiders with seemingly no stake or contribution to the success of the site asked for it.

This has a really bad and demoralizing PR look for all the people who've been patiently chipping away, spending their time, making their cases, for their suggestions to go nowhere. Now suddenly someone makes a few tweets or a blog post and you're reacting? Regardless of what it actually is, it just feels unfair and contemptuous. You have to really stretch and force yourself to assume good intent to not be insulted by that.

I'm not sure what the solution can be, but in the three years since joining this site and all the meta activity I've read before then I've never seen this amount of outright cynicism and dislike towards SE the company. It seems like the majority of the default trust and goodwill y'all used to have has been spent and replaced by cynicism and negativity. It is really sad to see rational discourse go off a cliff so dramatically.

Answer 2

I think part of the problem is the disconnect between effect and availability.

For years, feature requests have had to have one of two properties to stand much of a chance of advancing:

• be an improvement on most sites, or
• be limited to a few sites, do no harm to others, and be easy to implement

Examples of this second case include:

• MathJax on selected sites
• the Hebrew keyboard on Mi Yodeya
• different rules for collapsing comments on The Workplace
• the photo of the month on Photography (ok, I don't know if that was easy to implement)
• the professional-services disclaimer on Mi Yodeya, Health, and Law
• runnable code snippets on Stack Overflow

As you noted, there are feature requests that have been languishing for years despite being well-supported and beneficial. You can't do everything and you have to prioritize; I get that. And it's not all about the users; you have a company to keep afloat too.

Against that backdrop, though, we have a miscalibrated cluster of features. The driver for the left nav is Teams. Teams are only available on Stack Overflow, but everybody has to take the Teams-motivated left nav. When Teams were first brought up there were requests to enable them on other sites, and SE said no.

So here's my request: when there's a major feature that affects everybody, that's driven by one site and either does harm or is unpopular to the rest of the network, step back and ask if you can offer the motivating good thing to those sites too. Why not have the Moderator team here on Meta.SE where it makes more sense for its members? Why not allow companies to have teams on DBA or Software Engineering or Writing or Quality Assurance? Why not allow other groups (that are willing to pay the modest fees) to have teams on RPG or GameDev or Seasoned Advice or Board Games or Music? I don't know if there's a market, but I hope that the cost of turning on the feature on other sites isn't high, so in the interests of sharing the bounty along with the costs, could we not start with "no"?

I'm using left nav as a recent example here, but that's not the only case where earlier thinking and communicating about the broader network would have been helpful. Too often, it feels like things sink or swim on the basis of what Stack Overflow gets out of it. Sure, SO is the flagship site and has more users and activity than the rest of us put together, but it's pretty easy to come away from those discussions with the sense that the rest of us don't matter. I don't think that's what you think or mean to communicate.

When people ask "can we have X?", please look for ways to say "yes". When people ask "must we have X?", please look for ways to say "no".

Answer 3

After writing out this post a lot of this feels like a people/communication problem.

I guess it's about balancing the things that are going to take a little work, and make an impact over picking the battles you're willing to lose to win.

Practically - the bigger aim, of a standardised platform for the design for the sites is a given. I think it is a goal which has common sense basis and would help on the longer term.

However - I think SE occasionally (maybe as a small company) suffers from a certain degree of tunnel vision over current projects. It is not just what you need to do - it is how you do it, and how you communicate. At this point, the first we see of a new theme is the mockup and well, at this point costs are sunk.

While it's impossible to keep everyone happy - I guess the core questions are:

1. What matters to users actually using the site and how do we actually manage to reconcile it with the new, standard site design plans.

The reasoning behind things like removing more unique design elements and voting arrows for example - makes things easier to do, but it's literally something that's part of the site design anyway.If it is something you do once, and it makes the site a LITTLE more unique - that means a lot to users. Every win we can get for both "uniqueness" and standardisation helps.

So.. if you guys realise site designs are watered down, and its having an adverse effect, and your community is less engaged erm... things are not great.

2. How do we do this right once.

Cause really, if someone has themes they really really hate and you guys go "hey, they're right!" you're essentially designing two themes at once. Especially with the mature sites some of the unhappiness seems to be around the designer not actually 'getting' the site culture and what's important to them.

I'd say, especially on the first rounds having a slightly longer design process, and 'borrowing' a few staff or mods familiar with a site to point out the stuff users love and trying to balance between "we need to keep standard" and "the stuff we can include to make each site unique"

Or better yet start by asking users which aspects of their current theme is important to them. In many cases these are smaller things that fit into the current framework. Getting them engaged to start probably saves work, and you only need to do it for the initial stages/caddy style sites. Once the older sites are done, I suspect things will be less rocky, especially with newer sites.

Answer 4

I assume you're talking about concerns related to content quality (which naturally appears to be primary focus of the site "core group"). This may sound a bit paradoxical but I think the best way here would be to figure why you would need to address these concerns and serve this group.

Let me explain...

• To start with, look at two efforts which appear to consume most of company resources: standardizing themes and engaging new users. For both, you have some underlying theory explaining what is for better and what is for worse for the company, you have certain expectations of what you want to achieve by making particular changes and ways to estimate progress in achieving your goals.
   
  And you have metrics to estimate impact of the changes you make (granted you weren't explicit about metrics for UI standardization but if you use a decent issue tracker you could easily have it - I did something like that in one of past projects and it wasn't much complicated).
   
  Simply put, you have reasons to invest efforts into making changes and ways to assess results of the changes you make. Your reasons and assessments may be inaccurate or even mistaken but that's manageable - you can review, revise, adjust your approach and keep making progress, knowing that you are indeed moving towards goals you consider necessary to achieve.

• Now, compare that to content quality / core group needs - there is nothing like that. You can't tell why investing effort in this could be useful for the company, and you have no idea of how to estimate if things get better or worse (that is, better / worse for the company, not for some abstract global mankind which apparently only wants content to be better and doesn't care how much effort it would take of the company).
   
  This puts quality related matters in an extremely weak position when negotiating about investing efforts. When it comes to justifying these, the only reason you can provide is probably to just decrease meta noise. This leaves them no chance to stand when it comes to deciding where to put limited company resources.

A while ago I even wrote a post explaining why it makes sense for company to abstain of addressing quality matters (and instead, just keep pretending that they care)...

^{they will face a bunch of uncomfortable questions: what can be done at their side to keep site sufficiently nice, what could be the root cause of the snark, what can help to really improve things etc.}

^{These would be difficult questions and I can understand why they would prefer to stay away from addressing these. So far, other options I've seen that could help were: giving more power to experienced users to handle an inappropriate content and / or improving guidance for new users. Both these options seem to be quite effort consuming and rather difficult to do right...}

You see, the way to better serve the quality / core group needs seems to be to prove above wrong and start by finding a compelling reason to invest company efforts in addressing these needs.

---

- This is probably difficult and I can't tell you any concrete recipe but there is a knowledge source that looks worth studying: Stack Exchange features change log.

It lists multiple examples of past implemented features that focus on content quality. You can closer study these examples and try to find out if these carried any benefits for the company and maybe there are some worthy patterns or, I dunno, some manageable things that can be estimated and reasoned about and further used to justify investing company efforts into changes of that kind.

- You can also try to learn more about site visitors. Traffic stats suggests that there are millions visits a day, this is many orders of magnitude more than new question askers - what do we know about them?

As of late, company studies appear to be mostly focused on new users asking questions, but from perspective of long-term content quality this is just a dead end. This category of users is (naturally) primarily interested in getting immediate answers to their immediate questions and they have no reasons to care about them being searchable nor understandable to others nor duplicating what is already easily available.

Now, what about those others who don't ask questions but make such an overwhelming majority with their millions visits. Could they be interested in content quality (and thus get connected to concerns of site core group)? Why do they visit the site, how do they use it? How can company leverage their visits (they probably improve search rank of visited pages and company brand recognition, how can this be made useful for the company). Etc etc.

Answer 5

A lot of the frustration that users have with the changes could be reduced by being a little more upfront with how the change fits in the bigger picture and giving some preview of the end goal.

Some of the recent changes carried out, such as the left navbar, the black hand of the welcoming bandwagon, and the change of site themes, have been purported to be the "first step" of a "major overhaul". However, this wasn't clearly communicated, and in some cases, not at all communicated. As a result, these changes seem to be an unnecessary gimmick which nobody asked for.

Instead, for example, if the idea of custom lists was mentioned when left navbar had been introduced, and it had been clearly stated that the left navbar is in preparation for the custom list, the community would have likely given it a better reception.

The same goes with the black hand of the welcoming bandwagon, which too, not surprisingly, has got mostly hostile reception, so much so that the announcement post had to be locked. While we have been told that "further steps will be taken", we don't know what those are and how this black hand of the welcoming bandwagon fits into that bigger picture.

Hence, to reiterate, I suggest that the announcement of these changes give some preview of the future steps and describe more concretely the bigger context into which this change fits (even if the exact details are not clear). Otherwise, it just looks like the company is constantly working on the "gimmick of the month" while ignoring feature requests pending for several years.

Answer 6

How can we serve you better while still meeting our goal to standardize all 174 site themes?

This is an interesting but somewhat contradictory question. It juxtapositions design changes with service. Design, in this case, is about a sense of identity, while service is... Well not really about a sense of identity. I don't think these two things really overlap all that much.

It turns out that when you try to pick a color to match 40 different site designs, you quickly realize you only have one real choice: black.

I truly love the idea of "any color as long as it is black" as a quandary. It is a fantastic encapsulation of the the dichotomy between trying to customize, while still trying to economize. Because, most will state that their preference is customization, but when given a choice in which customization costs are presented, will usually choose economization.

This quote from Henry Ford:

It is strange how, just as soon as an article becomes successful, somebody starts to think that it would be more successful if only it were different.

It kind of sums up that question... But... Is that the question here?

I don't think so. What you are really facing is more to the notion of:

resist all temptations to raise expectations

You (Stack Exchange) has tried hard to develop communities. This has allowed you to capture, otherwise very expensive, labor to help your cause. One of the things that helped here, was the branding (customized styling) of the communities involved. But you long ago raised those expectations.

So while this question somewhat addresses the need to lower expectations around custom designs, it is not clear to me that it does not do so explicitly...