46

We're all struggling with communication and community stress. Trust is low and people are concerned. There are lots of changes occurring and it's not always easy to follow along or keep up-to-date. This post is not about the change itself at all - design changes almost always take time to get used to, so I'm not going there right now. The focus of this post is on ways to communicate that avoid leading community members to feel they are being cut off.

That being said, I want to recognize how well this specific case started out and I don't want to see staff who are working to communicate get discouraged or burn out due to the responses to their posts - particularly people in non-CM roles. I understand that not all staff members are able to handle unfiltered community feedback and there are cases where they need a break. The platform doesn't allow anyone to "unfollow" a post they created, so those answers and comments will always show up in their inbox feed.

Unfortunately, if the follow-through isn't communicated as openly and transparently as the initial discussion, you can end up in a net-negative situation. Throughout this post, I'll be recognizing things that went well as well as conveying my experience on the staff side of situations like this - credit is due for good work and staff do need care and consideration. I hope this might help both staff and community members step into each other's shoes a bit.

NB: I don't know why the decisions were made in this situation. My post relies on what I hope is a charitable interpretation - wanting to protect the author of the post from negative feedback. I don't intend to assert this is the case or misrepresent the specific staff member's needs, which are unknown to me. To reduce the risk of that, I'm avoiding naming anyone specifically.

The situation so far

This post is prompted due to the feedback post about tag design being locked - Seeking feedback on tag colors update. Based on the timeline, a few things happened:

  • A designer created a discussion asking for feedback about an upcoming design change to tags. (Nov 2023)
  • A CM updated the post with the final decision and images of the forthcoming design. (Feb 2024)
  • The change was shipped (May 2024)
    • no apparent announcement was made
    • change wasn't tested on a subset of sites first
  • An answer criticizing the change was added
  • A minor edit to the tags was suggested and approved
  • Two critical comments were added. While both critical, neither was offensive. Both somewhat duplicated the new answer.
    • One was descriptive and specific about concerns
    • One was terse and somewhat unspecific
  • A bounty was added to "reward existing answer"
  • A CM steps in and takes a few actions:
    • adds a comment only lock to the post
    • removes the bounty
    • removes the comment lock
    • adds and removes the comments only lock
    • adds the content dispute lock
    • adds an update reading "This change is now live."
  • The following day (20 minutes ago) a new update was added by the CM
    • It cited an unexpected influx of responses as the reason for the lock
    • The lock was removed
    • The update gives no additional guidance on how to share feedback or bugs and doesn't indicate the responses received were problematic, only unexpected.

As a note, there may have been other comments that have since been deleted that were more problematic than the two that remain.

Impact of locks on Community Members

Before I get into alternative options, let me explain why locks are concerning from a community standpoint:

  • While comment locks only prevent new comments on the question and might be acceptable in this situation, particularly if they're timed;
  • Most other locks prevent almost all interactions with the question. In particular, I'll focus on these three:
    • New answers
    • Comments
    • Votes on the question

When the CM's focus is protecting the staff from negative community sentiment, I can understand why a lock makes sense. They want to stop all of the feedback going to the staff member's inbox, particularly when it's negative and the team feels like they'd bent over backwards to be communicative and seek out community input.

Unfortunately, for someone in the community, it may be difficult to recognize that a lock may be being used to protect a single person and they may focus on how locking the post impacts them. They want to be heard and have a place to share feedback but, if the only apparent place to give that feedback that's been sought out by the company is locked, they may be at a loss of where to share feedback and bug reports. It leads to community members feeling like the company is not only not listening, but that the company is purposefully preventing them from communicating with the people best placed to address their concerns.

In particular, locking means community members can't even retract or change their support for the staff's post based on outcomes. In particular, this may lead to a sense of injustice or unfairness if they no longer support the discussion.

Core moderation philosophy

One reason that using locks can be concerning is that there's a core concept in the moderation philosophy on Stack Exchange to minimize the impact of moderation decisions as much as possible while keeping as many people as safe as possible without infringing on general access and use of the site by uninvolved users. This can be a difficult balancing act and moderators and CMs struggle with it every day.

Even as the author, that's worded somewhat confusingly, so here's a hypothetical example: a site is being inundated with offensive questions from unregistered users.

[redacted due to describing an internal enforcement process that isn't public]

This practice of looking for the most light-handed solution should help prevent unwanted behaviors without overly burdening people who are participating respectfully.

The stress of design changes and communication

This project started out really well.

The designer made an intentional effort to communicate and seek community insight before determining how to change the design. She outlined several options and explained the reasoning for the changes and goals. In February, a CM updated the feedback post with their planned changes and more screenshots of the final design that included additional tag types that community members had asked about.

I'll admit that I didn't see the update from February until today, so I didn't notice that an announcement had been made with an explanation of the decision. I'm guessing many others may have missed it, too. Now - full disclosure - I wrote an answer actually in favor of version 3 (or 4) with a long-winded breakdown of the various aspects of the change. Others voiced their own answers and reasonings and the decision was to go along with the design that seemed to have the most support - 3.

Thank you for providing the requested feedback on the original post. From the discussion, it seems like Option 3 and Option 4 were more favorable. Therefore we have taken that into account and settled on Option 3 as the base style due to its cleaner aesthetic.

Considering this work y'all had done and lack of responses to the decision update, it may have led to an assumption that the design had support from community members. This seems to have led to a decision that no MSE post announcement or testing phase (e.g. shipping on MSE only) were necessary. If that February update had been posted last week and been featured, I might have agreed but this change likely felt like it came out of the blue since the last communication was three months ago. Even more so for people who don't use MSE and may never have seen the November post.

Remember - while MSE may be where many of the most avid users come for information and to keep updated about the platform, many people don't use it regularly, so when you're shipping a big change - everyone will notice and need a reminder. It's not necessarily about the MSE crowd being unaware of the changes.

Toolkit options that keep communication lines open

There are two sets of options I'll be covering - ones that involve locking the original post and ones that do not. If at all possible, staff should collaborate with the site mods so they understand the goals and can support in a way that aligns with them.

When locking, how do you keep communication open?

There are valid reasons and cases where locking makes sense and is reasonable - particularly if the lock is temporary. This is most common in cases where you want to start fresh or change where the feedback is going but people are still participating on the original question.

  • If at all possible, lean on the least restrictive lock type - a comment-only lock - to focus discussion in answers rather than comments.
  • Explain why the post is locked.
    • For comment-only locks, the explanation can be in the comments section - usually some indication that comments were getting out-of-hand and that users should vote or write answers instead.
    • For other locks, the reason will hopefully be to move discussion elsewhere, more focused on collecting feedback.
  • Expressly state where users should raise concerns and that staff will be reviewing them. Examples could be:
    • A new, staff-authored meta post specifically designed for new feedback.
    • A new, user-authored meta post the staff have committed to participate in and respond to feedback and bug reports.
    • New questions on meta about specific bugs or concerns.

If the goal is protecting staff from feedback, ensure that the person posting on meta is able to interpret the feedback thoughtfully, even if it's not phrased very kindly. The benefit of using a new staff-authored post is being able to give the full context and explanation for users to see.

Options to avoid locking a question

Solutions here depend on next steps and goals.

  • If you want to collect new feedback on the same post but the author is struggling with the responses, the best option may be to change the owner of the post temporarily to alleviate that input. While not ideal, it protects the staff member while allowing feedback to continue. Consider swapping it back once feedback has quieted down.
  • If you want to collect new feedback on a new post, you can likely do so without locking the old one, particularly if the new post is featured. Be certain to include all of the appropriate background in the new post and cross-link the two.
  • If you don't wish to collect feedback in one place, ensure your update message tells users where to give feedback now. Review new answers and comments on the post and remind users of your request if they miss it using comments.

Conclusions

In any of these situations, keep in mind the core moderation philosophy and avoid choices that prevent valid and respectful participation. Locking is a big tool to pull out right away. Take time to assess the situation fully before jumping into action in such a heavy-handed way, particularly without any communication. It's worth leaving a post for 24 hours without locking while you discuss how to respond because it avoids giving community members the sense that they've been silenced.

Feedback is inevitable - regardless of how well a decision has been communicated at the outset, plan ahead and give community members guidance on where feedback belongs throughout the process. Locking an entire post due to an influx of unexpected feedback - particularly on a post specifically asking for feedback - isn't a choice that fits into the moderation philosophy.

No one - staff or user - is a punching bag. Offensive responses are offensive. Remove them and address the user directly or reach out to moderators for support.

I understand things are difficult and y'all are likely having to handle a lot more than you can. I'm sorry. I wish I could help.

18
  • 17
    It's been floated a couple of times that it might be preferable to have an official account post these feedback posts, which would then mitigate the issue of having a specific staff member have to handle these situations. Commented May 22 at 18:16
  • 18
    "When the CM's focus is protecting the staff from negative community sentiment, I can understand why a lock makes sense." - I have said this before, others have joined me in the present and I will continue to say it in the future. Either we stop claiming that the feedback are personal attacks to whoever got the short straw and had to post the new "announcement" (I am personally quite feed up to have to always add the formula "When I say you I mean the company") and suspend the few real bad actors or we make that "Company account" to hide names
    – SPArcheon
    Commented May 22 at 18:19
  • 8
    Thank you for sharing this perspective, I appreciate keeping it in mind. Having grown very wary of the recent SE ways, I'm certainly one of the people would instantly jump to "feeling like the company is not only not listening, but that the company is purposefully preventing them from communicating".
    – GSerg
    Commented May 22 at 18:36
  • 9
    @SPArcheon It's not really about whether feedback is a personal attack or not (assuming it's not an overt personal attack). Not being personal doesn't immediately alleviate all of someone's issues with the feedback. It can still be stressful and exhausting and anyone - even a CM - can need a break. This particularly occurs when someone feels like they did everything right at the outset and are still hitting walls and getting accused of failing to communicate. It's just hard.
    – Catija
    Commented May 22 at 19:30
  • 8
    Honestly I'm a bit surprised to read this. While I'm of course not going to argue this was handled flawlessly (nothing ever is), it's completely normal and even desirable for CMs to say "further feedback in this place at this time is not helpful." And... yeah, "unexpected" is frankly a reasonable descriptor. The post was featured for 30 days network-wide and the consensus broke towards option #3; the community's preference was implemented.
    – Slate StaffMod
    Commented May 23 at 1:08
  • 8
    Even still, I can assure you that while I'm not involved in the decision itself, I've seen folks thinking very carefully through the original rationale and current feedback. So feedback - elsewhere - is being read and reviewed. If anything, the lock was probably placed so that old feedback could be processed and reviewed for any discrepancies.
    – Slate StaffMod
    Commented May 23 at 1:08
  • 10
    The actual change also happened much after the feedback period, and caught quite a few people, myself included by surprise. And generally the process included suggesting an alternate place to comment first. It was also networkwide all at once. While there's mitigating factors as far as staff goes, a long term/time project generally means people are updating periodically, not rolling it out with zero warning, not even a meta post. Commented May 23 at 5:55
  • 13
    @Slate I think I mentioned most of what you're saying in the post but I also think you're not really responding to or considering the key points. I have bent over backwards to give credit where things went well and explicitly said that the lack of community feedback after the February update was likely why staff didn't expected to hear an outcry. But I also explain why that is not the sort of assumption that should be made, particularly when shipping a change six months after the original discussion. That said, my focus is on the locking of the post and why that impacts community sentiment.
    – Catija
    Commented May 23 at 6:55
  • 12
    The company is not required to seek out feedback at all times. If the company believes that no feedback is necessary and is, in fact unwelcome, that's... a choice - but it shouldn't prevent you from announcing the changes because you recognize that they are highly visible and users may want to understand the context. Regardless of whether you are open to feedback or not, though, it will be given. You don't have to read it, respond to it, or act on it but actively preventing it sends a clear signal. Why is that an appropriate response, considering the points in my question?
    – Catija
    Commented May 23 at 7:06
  • 9
    @Slate One of the problems in this case was not that company didn't listen to the original feedback (it seems that community was split between options 3 and 4), but the fact that change went live without immediate announcement (Meta post which would also allow new feedback) and that it accidentally happened to be a bad change after all (bold fonts turned out to be more disruptive than anticipated from the original pictures). It would probably fare better if there were no major problems with the change. But because it is hard to predict how it will go, all the more reason to announce. Commented May 23 at 12:31
  • 9
    @Catija, the thing is, as far as locking, you're making assumptions about why the post was locked, and those assumptions are wrong. It's not that Charlotte tried to silence feedback because folks aren't "open to feedback". She did it because she wanted to make sure she had the correct sentiment about the feedback that was sent months ago - she, in fact, re-run the feedback analysis and count to double check that the choice that was made was in fact the choice that was made months ago. This would've been much harder to do had the post not been locked (due to how the tools for that work).
    – Cesar M StaffMod
    Commented May 23 at 14:11
  • 7
    So she locked the post for a bit while she was doing that. Should she have left a comment saying so? Sure. It'd have been easier to figure out why it was done. But after the analysis was re-done and she double checked she had the right answer and her memory from 6 months ago was correct, she unlocked it. It's as simple as that. Not leaving a comment was an oversight, yes. But a small one in the grand scheme of things, so assuming that it was locked to stop feedback is blowing things way out of proportion.
    – Cesar M StaffMod
    Commented May 23 at 14:12
  • 4
    @CesarM Half a day is not a "bit". Both of those sets of changes there were reactive, done long after people pointed it out. I don't see what you hope to achieve by being so defensive. The point is not to blame someone. The point is to identify the sub-optimal bits, figure out how not to repeat those mistakes, and set up a process to ensure that happens. Catija described many of them quite well. It would be nice to see some staff post say "Yeah, we messed up on some aspects of this release, here is how we're prevent that from recurring." instead of bickering.
    – Dan Mašek
    Commented May 23 at 14:37
  • 9
    @DanMašek half a day is actually pretty fast to complete a re-analysis. It usually takes longer.
    – Cesar M StaffMod
    Commented May 23 at 15:20
  • 4
    @CesarM I also agree that an explanatory comment would have made things a whole lot clearer, and have gotten rid of any false impressions that dissent is being suppressed. However, if she didn't have time to do so, a better option would have been to make the lock temporary, so that the system's lock notice would indicate that it's a temporary lock. That she didn't do so made the system lock notice indicate an indefinite lock. Commented May 24 at 0:02

7 Answers 7

24

I would very much welcome having a new post when rolling out some changes to receive feedback once a change is live.

First, the proposed changes could look fine when proposed, while in real life people can determine that what they thought would work, does not work at all or has some problems.

Second, sometimes the proposed changes are not 100% the same once they are released and there might be bugs, so having the new post focusing on what was actually released is better that having to post feedback on an old post.

A new post with an announcement of a change and new feedback also allows people to be less emotional and have more patience in their response to the change, because there is an open communication channel and we all know that it takes some time to take the feedback into an account.


I posted the bounty on the old announcement as the new answer posted which asked for a rollback seemed like a good option, considering there was no official response on the new post which noticed the change.

It is possible that my description in the bounty notice was a bit over the top. I sad something in line that "This change literally brought tears to my eyes." and that rollback of the feature looks like a good idea, but the thing is that for my eyes this change had such an effect. After half an hour of going back and forth, squinting, through Stack Overflow and Meta (especially home pages where tags are extremely dominant feature), I couldn't look at it any more because my eyes were full of tears from too much strain.

Even today, I have been avoiding main pages and trying to look only superficially going where I want to go avoiding looking at anything that has new tags.

If my bounty description was a problem, I sincerely apologize, but the overwhelming effect this change had on my ability to use the sites was just too much for me to handle.

1
  • Reading all the posts about design changes and the difficulties of communication on SE the fact that this is a public website seems to me to be missed. There are trolls out there whose aim in life is to disrupt communication and belittle any attempt at development in good faith. The Open Source movement manages this very well, maybe there is a lesson there.
    – MT1
    Commented May 27 at 5:34
22
+500

Sharing the perspective from a staff member

I’m going to preface this answer by letting everyone know that I’m a designer and I was involved in the recent design change to Tags. In fact, I signed off on the final design, reviewed and approved the related PRs before the release. We’ve had more than one designer work on this change — such as LSee — but I’ve been involved on this project since the beginning.

There were a lot of comments on that new post (that have since been deleted) that, in my opinion, could and have been delivered in a much more productive and respectful format. Although this post is not about what should be considered offensive or not (as I’m sure we all have different opinions on the degree here), I do want to call out that those types of comments do have an impact on everyone seeing them. There are hard working people (including designers, engineers, and other staff members) that work on these changes who see and read your responses on meta.

What happened?

There’s a certain amount of time and effort that goes with posting on Meta. Most of us are working on multiple projects going on at the same time, which means we need to plan and schedule when we can allocate the time for crafting, reviewing and responding to a Meta post.

Specifically around asking for feedback on designs, we also need to plan for the amount of time a designer will get to gather feedback and make changes. Whether it’s a single round or multiple rounds of feedback, each time needs to be accounted for and coordinated amongst everything else that’s going on.

For the Tags, our development, design and CM efforts were limited, only allowing us to allocate Meta time and resources for the initial rounds of feedback during the design phase. This is often the case and the reality is that we don’t have the resources to do a testing phase after it’s been developed. Keeping all this in mind, our goal was to get community feedback and make any changes necessary during the design phase. Changing this component was already a follow-up initiative after receiving feedback from the community that the new blue used for tags was causing issues (1, 2, 3, 4, 5).

I won’t repeat the events outlined by Catjia above since they’re pretty accurate up until the release of the change. We had made a decision not to post an additional Meta post before the release or to gather additional feedback after the release, because we currently do not have time to execute on a second round of feedback.

I also agree that the amount of time in between the initial post, the update and the final release was too long. We did not intend this change and process to take so long. I believe this is one of the causes for the not-so-great effect from this comment:

“I just love how SE loves rolling out features that were possibly announced months ago, so that we all could completely forgot about them, until they smack us right into face (eyes).”

Unfortunately, as much as we tried to coordinate all the people and series of events that needed to happen for this release, things don’t always go according to plan. We had a lot of unexpected delays that led to this project spanning 6+ months instead of our initial 3 month plan. Can I promise this won’t happen again? No, but maybe there’s a better way we can manage communications when something takes longer than expected.

In all honesty we thought we had done the process correctly here (excluding the amount of time in between these events). We made design decisions after gathering and listening to the community’s feedback, and ultimately released the version the community preferred. We did not expect this change to have the reaction it did once released. Our selection of what we understood to be a relatively non-controversial option is part of the reason why we chose not to create a new announcement before launching. A visual change on one component (albeit a very common component) didn’t feel that big to us at the time.

Why lock the post?

I’ll take responsibility for the locking of the post as well. As responses and comments were going up on the new post, we started the process of analyzing the original post to triple check the original responses were in fact for Option 3. This takes time, we need to read through all comments, answers and tally votes on everything. At the same time, we were seeing new activity coming into the original post and a comment suggesting people should change their vote. Although it is possible to figure out which votes and responses would have come in after the launch vs when we were actively seeking feedback, it requires more time and effort.

All of this paired with the amount of unnecessarily creative comments resulted in myself suggesting to our CMs that we should lock the original post in the hopes of saving ourselves additional effort and getting caught up with all the responses coming in from all the posts related to the Tag change.

This was not an attempt to silence the community. The lock was in fact placed so “that old feedback could be processed and reviewed for any discrepancies” — Slate. The lock was up for less than 24 hours and I assumed the community would continue to leave any new feedback on the new post. I now see that the lock itself had a larger impact on the community than I anticipated and I will be much more hesitant to suggest such an action in the future and will consider alternate options to avoid locking first.

Alternate course of actions

We plan to discuss these series of events as a team to review and understand what we can do differently in the future. The following are my own thoughts that I’ll bring back to the team to for review:

  • We could have added an update to the original post sooner, including directing the community to leave new feedback on a new post.
  • We could have changed the title of the original post, making it clear that the ‘feedback seeking’ round had been completed. My sense is that we usually keep original titles for historical accuracy but maybe this can change for this type of post.
  • We could have specified a limit for the timeframe of when we were seeking feedback in the original update, again to make more clear that the ‘feedback seeking’ round had been completed
  • We could have chosen to only lock votes (I actually forgot this was even an option) paired with an update that we were conducting an analysis.

I and the rest of the team appreciate your passion and hope that you can help us all move forward together.

22
  • 5
    Speaking of the lock, again, keep in mind that the system notice indicates if a lock is temporary or indefinite, and if the former, what its duration is. The lock was only there for 24 hours, but it was not specified as a temporary lock when applied, which falsely indicated that it was an indefinite lock during the time it was applied. I'm certain that specifying it as temporary would have greatly reduced the impression that voices were being suppressed. Commented May 28 at 16:15
  • 7
    I certainly appreciate the response and the level of detail you provided. I will be thinking about further responses but I wanted to make a quick note about locks. Unless it's been added recently, there's no vote-only lock type. Second, locking the question doesn't prevent voting or commenting on the answers. If you're only concerned about the question, that's fine, I just wanted to clarify these aspects.
    – Catija
    Commented May 28 at 17:49
  • 4
    "Alternate course of actions" these are all things that seem to miss the mark for me. Let's be very honest here - releases of new features and/or designs have often enough been very jarring for a variety of reasons. And the communication of when something is being released has also often been rather lackluster. The usual course of action is that something is released, then users scramble to figure out if that is a bug, or a feature, was it announced. Often they also have feedback. None of your options solve this initial confusion, nor the outlet for responses. What happened now? Well,
    – VLAZ
    Commented May 28 at 18:02
  • 3
    the same thing - nobody knew the tags would be released. There was no (recent) announcement or anything. There is no way to leave a comment about it. The only official channel is the months old thread that even asked for feedback. You seem annoyed that people left disparaging comments. But what other alternative were they given? None of the options are along the lines of "We could have announced a day before and offer people chance to respond to the change". Or anything that would be less jarring and more receptive to feedback. The options proposed are, rather wearisomely, revolving around
    – VLAZ
    Commented May 28 at 18:03
  • 7
    "We could have tried more things to limit users from trying to engage with us". You should be thinking about how to involve the community more and make them feel heard. Because there is already the perception that SE isn't listening (or "selectively listening" if we want to be generous). The optics of locking a thread when people want to respond (and again have no other official alternative) are so poor you had to write this post. Literally all of the issues here (unexpected delays, unexpected community reaction to what the team thought was a small change, deliberate decisions to not
    – VLAZ
    Commented May 28 at 18:03
  • 3
    announce anything, having to lock posts, mismanaging locks, etc.) could have been mitigated with communication. There is evidently a level of disconnect between the team and the community. But going silent and then trying to scold people for being mean isn't what is going to solve it. More open information would have been. That doesn't need to be a commitment to multiple rounds of feedback. That takes at least some information going out. "Remember X thing - it's been delayed".
    – VLAZ
    Commented May 28 at 18:03
  • 9
    "we don’t have the resources to do a testing phase after it’s been developed" - I'm sorry to hear that, it's a situation that any developer (including me) hates, but well, that's something the company should fix, right? Specially because it can affect your users in a negative way, as you might have noticed. Anyway, if you're so understaffed that a "simple change" like that takes months, maybe you should review your priorities and postpone/cancel those changes (and maybe focus on the bigger/most important ones). Just saying...
    – hkotsubo
    Commented May 28 at 18:04
  • 3
    That something takes longer than expected is totally understandable. It would be useful if there was more transparency and communication on the progress of the effort. Assuming there's some sensible project management happening, it shouldn't really be too difficult to get input into a monthly high-level progress report. The company seems to be well capable of producing countless blog posts, that seem to be mostly irrelevant to the site -- perhaps some of that effort could be redirected to keeping the community up-to-date regarding what you're doing to really improve the site.
    – Dan Mašek
    Commented May 28 at 21:21
  • 3
    "As responses and comments were going up on the new post, we started the process of analyzing the original post" -- Since you seem to have been aware of my post tagged as a bug report quite soon after it was posted.... why didn't you post an answer there, or even a comment? A simple "Yeah, we did a release, see this post." would have been great, and I'd have gladly accepted as an authoritative answer. It's still not there... I'd even have accepted a dupe close. We were just left guessing and then half a day later someone made a small edit acknowledging the release and locked the feedback post.
    – Dan Mašek
    Commented May 28 at 21:52
  • 4
    just to say - if the process took too long cause y'all were understaffed, We're very willing to apply pressure. Just blink twice. Also, as a former MSE mod, locks are fine - the problem generally tended to be lack of co-ordination with the mod teams, + the lack of someone from the company realising during that period/acknowledging the follow up post. The 'correct' thing would have been to edit the post saying that you'd prefer a new post for feedback /posts (per bug), right on top, and linking it. The community also appreciates updates - even quick ones. Commented May 28 at 23:42
  • 3
    There's multiple teams that lack the time and bandwidth to do their job it seems - I'd already brought up this for the community team. I'm quite willing to formally ask about it for y'all too. The time and effort put into communication initially is likely useful in saving time later, and a lot of stress. We don't want to have staff quit due to community induced stress or company induced stress, but as the end uses and practically the 'client' for these designs, it would be nice if our needs and wants are taken into consideration with these things. Commented May 28 at 23:44
  • 4
    "[We] ultimately released the version the community preferred" - ahh here's a pitfall: Meta is a subset of the wider userbase (generally the most involved users). You can ask the 20% of us what we think, and we can even agree with the changes, but when you LAUNCH, it's going out to 100% of the user base: you're gonna have confused users wondering why someone moved their cheese. This is why we need a launch meta (or even the old meta with an update!) featured across all sites so we have something to point to that explains the reason for the change.
    – Robotnik
    Commented May 29 at 7:01
  • 2
    I appreciate your answer. I would say that for very visible changes that will be noticed by literally everyone, you will want to have a new Meta post, even if it is just two sentences and link to the original announcement as plenty of people will miss the update on the old announcement. New post always has greater visibility and also allows users to give you the feedback regardless of whether you will be able to act on it or not. Also the amount of passed time is only a problem because there was no new announcement so even people who had seen original could forget about it. Commented May 29 at 14:43
  • 2
    From the design aspect, you have light theme and dark theme, what works for one does not necessarily work for other. Blue was not a problem on the light theme, while it definitely doesn't look good on the dark. Also font weight will have different effects on both themes. I had to use custom style for light theme to revert the weight change as I literally cannot read the sites for more that 5 minutes without my eyes objecting with new style. But, after a while I also reverted to original blue coloring (bg and font) as gray was too dull and depressing (I don't have a proper word). Commented May 29 at 14:50
  • 2
    On the other hand, on SOfT where color remained, but font got new weight (which is still too heavy) I actually can use the sites as the color softens it a bit. So far I didn't had to revert the style change there, but I also don't use Teams that much (maybe a few minutes every few days comparing to literally living on the Stack Overflow. Commented May 29 at 14:55
18

This answer is in response to comments from CMs on the question. It's somewhat difficult for me to respond to comments briefly that are stating so many things without risking a huge comment thread. While this is non-standard, I feel it's important to contextualize my comments in a way that the single long comment thread can not.

This is an issue with the meta discussion format, so we do what we can with what we have.

Firstly, I want to share that I appreciate that both CMs indicate that it would have been better to give an explanation when locking the post:

Cesar:

Should she have left a comment saying so? Sure. It'd have been easier to figure out why it was done. [...] Not leaving a comment was an oversight, yes.

Slate:

It could have come with a better explanatory comment [...].

Unfortunately, I'm disappointed that both of them opted to minimize this error by emphasizing the temporary nature of the lock:

Cesar #1:

So she locked the post for a bit while she was doing that.

Cesar #2:

half a day is actually pretty fast to complete a re-analysis.

Slate:

(temporarily!)

... and making it out to be a very minor error that's not worth a reaction:

Cesar:

But a small one in the grand scheme of things [...].

Slate:

[..] but I feel that's at best a nitpick.

The quotes above are excerpts but I've linked to the comments because I've made a particular effort to avoid quoting staff in a deceptive way.

Sure, in the grand scheme of things this whole issue (tag design) is pretty minor and y'all likely have bigger things to be focused on. Maybe it's reasonable that the lock was only active for 17 hours. But by focusing on minimizing these aspects rather than discussing how to improve, you're risking further alienating people. Dan Mašek said it really nicely:

[...] I don't see what you hope to achieve by being so defensive. The point is not to blame someone. The point is to identify the sub-optimal bits, figure out how not to repeat those mistakes, and set up a process to ensure that happens. Catija described many of them quite well. It would be nice to see some staff post say "Yeah, we messed up on some aspects of this release, here is how we're prevent that from recurring." instead of bickering.

A sizable section of my question is designed to outline an interpretation of the actions by community members. But the interpretation is valid, regardless of the intention of the staff or how anyone interprets the situation. CMs are completely allowed to stand up for their team and explain the situation and try to respond to the concerns, whether you feel they're valid or not. The thing is, there's a difference between stating your own position, and trying to invalidate the feelings of community members.

I posted this question because I saw people who were confused by and frustrated at the decision to lock the post with no explanation and I wanted to give a voice to that while also recognizing there's a very kind and plausible (if incorrect) interpretation - which has helped some community members see there are explanations other than "we're being silenced".

The choice to lock a feedback post - for any reason - is confusing. I can't remember a time where this has happened in the past and it certainly wasn't something I would have ever considered or encouraged without carefully communicating the reasoning - as stated in my question.

The lock was added at 23:00 UTC on May 21st and removed at 16:32 on May 22nd. Yes, that's less than a day and yes, CMs need to sleep. It may have been a very fast re-assessment of the feedback - I don't disagree with that. If there'd been an explanation, this question would have never been asked.

Feedback analysis tools? Tell me more!

Let's look at what's been said about why locking the question was necessary.

Slate:

If anything, the lock was probably placed so that old feedback could be processed and reviewed for any discrepancies.

Cesar:

she, in fact, re-run the feedback analysis and count to double check that the choice that was made was in fact the choice that was made months ago. This would've been much harder to do had the post not been locked (due to how the tools for that work).

Since I 100% believe that CMs can look at post dates before deciding to review user feedback, these statements seem to indicate that the CMs are using some sort of tool to analyze post content. While I can understand using a tool for this, it seems like the tool has some limitations, which led to CMs taking unexpected actions to control the data fed to the tool:

  • Data import has limited functionality:
    • It can't use post or comment date to restrict or exclude data.
    • It's not possible to manually exclude data from the analysis.
  • Report history or consistency may be limited. While unclear from the information I have, it seems one of the following may be true:
    • It doesn't save reports of prior analyses.
    • Running the analysis multiple times gives different results.

I may be completely wrong, so I'd love to know more about the tool you're using to analyze community feedback, how it works, and how that might impact how staff interact with the site due to the limitations of the tool. It would also be fascinating to see an example of the tool's output. I'm also curious how much CMs do to spot check the tool's results to ensure it's accurately conveying the data it's analyzing.

Being transparent about this can really help us work better together. If we know you're using a tool to understand feedback, that may impact how we give it or how we vote on answers. Essentially, how much does context matter and what's likely to be interpreted by the tool in what way? Does the tool give context for its output i.e. citations or does it just summarize the data?

As an example, I wrote a long answer to the post in question breaking down all aspects of the examples with the conclusion that version 3 seemed like the most consistent option while listing specific concerns about the design and stating that I personally preferred version 4. That garnered a lot of upvotes (48/-1) but how did the tool interpret the answer and weight it based on votes? Did it call out the specific failings about version 3?

Regarding assumptions I made in the question

This may be somewhat tangential because I think it was caused by a comment I wrote responding to something Slate said in a comment rather than by the content of the question itself but I feel that it's worth pointing out.

Cesar says:

@Catija, the thing is, as far as locking, you're making assumptions about why the post was locked, and those assumptions are wrong. It's not that Charlotte tried to silence feedback because folks aren't "open to feedback".

While I did state in a comment:

The company is not required to seek out feedback at all times. If the company believes that no feedback is necessary and is, in fact unwelcome, that's... a choice - but it shouldn't prevent you from announcing the changes because you recognize that they are highly visible and users may want to understand the context. Regardless of whether you are open to feedback or not, though, it will be given.

I can understand how the hypothetical above could have been interpreted as me indicating I believed they weren't open to feedback. That was not my intention. If that statement has lead to derailing this conversation, I'm sincerely sorry for failing to phrase it more clearly since my goal is to share my point of view and experience.

In the question I attempted to avoid indicating any negative assumptions about the behavior of the staff here - particularly, I did not ever indicate that I thought they were not open to feedback. I actually expressly avoided ever even implying that was the case and purposely excluded examples of solutions for situations where the staff were not open to feedback, which I did consider.

My feelings about and interpretation of the actions are not assumptions of their decisions. I wanted to share my point of view in an honest way in an effort to give insight into the impact of the actions taken in this situation. I believe I clearly show that I'm considering alternative, fair explanations to the best of my ability with the limited information I have.

A year ago, I would have shared this guidance privately with the team, hoping to avoid even small missteps that might erode what trust community members have in the team. Failing that opportunity, I wanted to contextualize and explain why choices can have negative consequences, even if everyone has the best of intentions and how important it is to carefully plan community engagement, even for what may seem like minor interactions.

I have attempted to engage fairly and empathetically in my interactions with staff on MSE or MSO because I know that my former teammates are struggling and I miss them and being able to help. I also recognize that there are likely cases where they wish I would zip it. This is likely one such case. I'm sorry for that.

2
  • 9
    I don't see any reason for you to be sorry. If we can't have this sort of discussion, we might as well call it a day.
    – Dan Mašek
    Commented May 24 at 18:49
  • 7
    I don't think we would be better off without your insight, and I don't think this has made things worse for them. (The discussion you kicked off here has made me more sympathetic to the staff involved, because it's revealed context that makes their mistakes look more reasonable. Even if it doesn't help them do better next time, unless I'm a weird outlier that still helps them.)
    – wizzwizz4
    Commented May 26 at 20:06
17

In my opinion, the key problem here came from a lack of communication, rather than the act of pushing the change. This comment from the later bug report summarizes this well:

I just love how SE loves rolling out features that were possibly announced months ago, so that we all could completely forgot about them, until they smack us right into face (eyes).

There's a way that could have made this rollout a whole lot less jarring, and at the same time not have given off any impressions that there were no considerations made: provide a date when the change will be rolled out that's at least two weeks in the future, and re-feature the post to make sure everyone across the network knows.

The 2-week advance notice will give people time to adjust to the change, as well as time for the user script/style community to write scripts/styles to undo the change, so those who want to work around it can install them beforehand and thus have no time where they'll see the change at all. This will also provide for a feedback period where users can express concerns with the change before it gets rolled out.

8
  • 4
    Two weeks would be generous, but even few minutes before pushing it out would have been better than nothing.
    – Dan Mašek
    Commented May 22 at 22:03
  • 7
    before it gets rolled out to be fair, most of the negative impact that stirs complaints are not obvious until after the rollout. A few curated screenshots in an announcement post aren’t all that touchy-feels. It’s almost like there should be a canary site where changes are deployed first, then an announcement “hey we did this, let us know what you think” … but of course if feedback is ignored anyway then the timing of the announcement is kind of irrelevant. Commented May 22 at 22:44
  • 3
    @Dan minutes only make a difference to users who are on the site all day long. For me, several hours notice is the same as a few minutes. Commented May 22 at 22:46
  • 6
    @testing-ma-lady My point was that when I noticed that (which just happened to be at the time they pushed it out -- I walked away for a few minutes and upon reload it looked different) I would at least see some obvious explanation of what's going on. Anything beyond that would be nice, sure, but I that bare minimum seems like a reasonable expectation.
    – Dan Mašek
    Commented May 22 at 22:52
  • 1
    I always find it interesting that the timings here are expressed in minutes when the sites are dealing with a global audience in many different time zones. 24 hours is a minimum notice period. If there are some users that are on the site all day long what day is that? West coast USA? East coast? Europe? India? Australia?
    – MT1
    Commented May 23 at 5:21
  • 4
    It would be fine if the change was announced a day or two before rollout, or even just before the change. There are many people who will go searching the Meta when some change happens and will not be aware of it before. Having fresh post helps in such cases as it is easier to find (and being featured also helps) and bugs and how well change works can only be seen after the rollout, so announcing it two weeks before would be too early. Commented May 23 at 6:19
  • 1
    @MT1 Huh? The only thing I've expressed in minutes was how long I walked away for. Not sure what other unit would have been appropriate, given that my bathroom lies in the same timezone.As for my first comment, I'd have hoped that the emphasized "before" made it sufficiently clear what message that first sentence is trying to convey. You can't built an argument on single word ripped out of context.
    – Dan Mašek
    Commented May 23 at 10:39
  • 1
    @testing-ma-lady: To that point; the images are all four unreadable to me; I could not have known the effect until they went live, and I am adversely affected by it.
    – Joshua
    Commented May 23 at 21:09
16

I think its worth looking at the bigger picture, and contrasting this with a recent, better managed rollout.

While I'm not a developer - I've a background in computer forensics academically, and the software release cycle was one of the things they'd covered in school.

With the new image host project we had clear, early communication that it was coming, and clear, open venues for feedback on each step. I entirely understand that a 4-5 month old post can get a little confusing when its suddenly got new replies.

I think the process here should have been

  • not wait as long between initial announcement and release
  • limited user acceptance testing with real users (the new image uploader did a limited release first, and this worked very well)
  • announcing maybe a week or two before that this was the option picked, the reasons and opening up a new post, and the floor for feedback
  • release with clear responses to feedback not just including bugfixes.

Ideally somewhere here would be "let the relevant mods know" I literally found out about this cause the page changed midway. We have mechanisms for this, meta posts, announcement and featured tags. Any change must be announced and telegraphed ahead of time.

The new image uploader did all this, even a limited roll out. Folks have been really responsive and generally its a textbook example of how to do a rollout.

Design changes feel like "We're doing this, now, BYEEEEEE".

As for criticism - trust the system. More importantly make it your friend. The company has done a poor job at winning trust, and trust is what you need for something like this to succeed and more importantly to find a design that works for everyone. Locking the post is going to result in more, not less anger from people, and the outlet's for that are not always as healthy as one would like. While I'm not a member of the moderation team any more - if someone's being needlessly mean, or doing personal attacks, I think the commitment to dealing with flagged, unkind content is still the same. It goes.

At the same time, when you have moderators and regular users of the site saying a change isn't good, its worth revisiting the changes rather than digging your heels in. We might be a little precious about certain things, but we're also the end users you are designing for.

14

Regarding the "staffling John Doe is getting stressed because of criticism on meta after announcing feature x", we had that discussion a hundred times before. There are two simple, obvious solutions:

  • Either create an official account "Stack Exchange"/"Stack Overflow" under which all change/new feature announcements are made.
  • Or have the CEO announce them, as done in in pretty much every other company. Normally the CEO is aware of changes to the flagship product and approve of those changes. Normally the CEO is proud of the flagship product and willing to promote it in person. This was previously the case at SO as well.

Or at least have some product manager/CTO announce them. Sending forth the rank & file to act as "cannon fodder" to announce the latest feature in a discussion post is cowardly and bad leadership. If you are in a leadership position, you should already know this - why do I have to post this answer.

5
  • Basically "Define a [sensible] release process and stick to it." Like many other issues we keep running into over and over, this seems like rather low-hanging fruit to fix.
    – Dan Mašek
    Commented May 23 at 11:01
  • 1
    SE is completely capable of doing a decent job of it. I've made a case for not doing 1 recently. You also assume that Q&A is the flagship product in the eyes of leadership and it hasn't been in years. I'm not sure the competencies in writing and dealing with meta are strong company-wide but that's another issue. There's people good at it, but good communication takes practice and a certain mindset, which seems to have been generally neglected :D Commented May 23 at 11:02
  • 5
    @JourneymanGeek Yeah. What I don't get is these endless cycles. The figure out how to do something, do it right for a while, but inevitably in 6-8 it falls apart and we're back to where were. There's something fundamentally broken in the organization, but not being an insider, it's difficult to tell what exactly it is.
    – Dan Mašek
    Commented May 23 at 11:20
  • 5
    @JourneymanGeek Specifically the site Stack Overflow is the flagship product, simply because it's by far the most famous one and without it the company wouldn't be worth much. Next to nobody out there have heard of any other products from the company. Nobody in their right mind would pay $1.8 billion for "Teams" or the numerous low traffic Q&A sites. Though admittedly the value of the company ought to be significantly lower now than during the Prosus sale, given the repeated layoffs and competence losses that came with them.
    – Lundin
    Commented May 23 at 11:29
  • 1
    Eh, I'd disagree- while the SO name is used, SO inc's flagship product has tended to be a series of SAAS products. We'd be in much better shape if Q&A was treated as a core product Commented May 23 at 11:32
7

I believe I might be the one who added an answer to the official feedback thread here. I find the new graphic changes so horrible that I'm admittedly having a hard time keeping the criticism constructive. Posting below the official feedback thread rather than making a separate post throwing a tantrum about the changes seemed more constructive though.

The locking and unlocking was weird, but what's even more weird is that I posted that answer then thought nothing would come out of it. And yet when I return the post has a 100% consensus in terms of voting, currently +38 for, 0 against. Similarly, the separate post about the changes All tag labels suddenly appear in bold font has the question up-voted 160 times and down-votes 1 time. The top-ranked answer criticizing the new layout has been voted 93 up-votes and 1 down-vote.

There is never a 100% consensus among users on meta about anything. Not just majority vote but 100% consensus. This is the first time in 10+ years I have seen it happen.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .