The Q1 2020 Community Roadmap is on the Blog

Question

I just posted to our blog about our First Quarter Community Roadmap for 2020. Here's a preview of what the roadmap looks for this quarter:

Please see the post for details on the different themes of our upcoming work as well as on some of the different projects that our Public Q&A and Community Management teams have been and will be working on this quarter. We'll be releasing more information to you about each project as it nears release. We are about to kick off our Q2 Roadmap planning so please share if you have feedback we should consider.

I also wanted to let you know that we shared the Internal Community Engagement Guidelines with the company today as promised.

Answer 1

This is a really encouraging blog post, and I appreciate the roadmap being shared with us. We need to talk about part of it though.

We are making fundamental changes to how our close question system works so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators. Questions that get voted to be closed will be hidden, giving authors the opportunity to improve their questions in private. Our hope is to make it easier on everyone by providing clearer guidance and encouraging better questions. [...]

Could you clarify what this means implementation-wise?

It sounds like it means a closed question will be hidden from everyone except the author (or maybe high-rep users). This would be very, very bad for everyone involved, even the author of the question!

Here's how the close-edit-reopen process works on many large sites:

1. Close: The question is closed ASAP, because serious work needs to be done on the question and we need to put answers on hold for now. (Otherwise as the question changes, answers will get invalidated and we'll have a huge mess to clean up, including people upset their answers are now not answering the question—because they were fine originally.)
2. Edit: Community members actively engage with the closed question and its author, asking questions, making edits, and providing direction that bring the question into a reopenable form.
   • If the question is not going to be reopenable (e.g. it's just a poll for peoples' opinions on their favorite soda) then that gets conveyed during this engagement. The querent may be directed to other places they can ask about this instead.
3. Reopen: People come across the question as revisions bump it, and reopen it once it's reopen-worthy. Alternately, maybe the question never should have been closed at all: people who discover it reopen it right away. Invested community members may open meta Qs requesting assistance reopening the question at any stage.

Hiding a closed question impedes or makes impossible all three steps of this process. As one of my fellow mods put it: newbies already believe closing to be a death sentence for their question. This would mean it actually is a death sentence.

Let's walk backwards through this and talk about the problem hiding a closed question represents at each step.

3. Reopen

This relies on users discovering a question as revisions bump it, evaluating it, and casting reopen votes. The meta Qs that might arise around the question's closure rely on invested community members discovering the question while it's still closed.

Who's going to do any of this if nobody can discover the question?

2. Edit

Getting a question into a reopenable shape always involves engagement from the broader community. Authors do not usually independently know what to do with their question to get it into an open-worthy state without feedback, or they would have asked it that way to begin with! Continuous engagement with the broader community is critical here.

That helpful engagement that gets questions revised can come from the entire reputation spectrum, both in theory and in practice. (Provided you have the 50-rep comment anywhere privilege, usually.) Even users who are too new to contribute can learn a lot from seeing our process in action, including learning that if/when their question is closed they can expect similar assistance.

1. Close early

Closing questions fast and early is an extremely important quality management tool. We need to hit the brakes to give the question room to be worked on without also dealing with answers arriving.

But people want to help. They really, really want to help. It's the whole reason they're here on the site. If a question is hidden when it's closed, that means they will not be able to help while it's closed. That means people will not want to vote to close a question if it's in a possibly-recoverable state, because then they're blocking the community from helping it.

This means people will not use the close tool early and often, or at all if they can help it, leading to really bad quality issues and headaches. Many of the people who currently vote to close fast and early will instead begin to abstain from close voting—and for the exact same motivation!

In summary

... So, with all that said, I hope you can understand why I hope “questions that are closed get hidden” does not actually mean what it sounds like. But I don't know what else it would be.

Could we talk about that thing? Can you tell us more about it? If it's what it sounds like, can we not do this thing?

Answer 2

Where have you been all these months? (Hint: I'm only kidding, I know she's just joined the company).

Just the mere fact you take things to the blog, and put up a summary here is basically priceless! Simple things can make such a big difference! Thank you very much, and please continue with that approach!

Yet, let me say this:

We are making fundamental changes to how our close question system works so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators.

Please consider to not only announce fundamental changes to be coming. Instead outline/draft them here before spending a lot of resources developing them. An agile mindset and development model should include user feedback as early as possible. If you consider the users to be true stakeholders, then act accordingly.

Your proposal sounds like a large waterfall, as if you design all these features behind closed doors, to then be surprised in case they miss the point, and users are unhappy about the result, and more importantly: the fact that they weren't consulted upfront.

So: I very much appreciate that you keep us updated with a relatively detailed timeline. But as said: announcements alone aren't what will make the difference in the long run. It is also about working together. And changing essential elements of the Q/A "technology" is something that many users here will consider to be "at the heart of everything". So please include us on that path.

That doesn't mean that each single detail needs to go through a lengthy MSE/MSO discussion. But writing up an outline for each major change (that could for example include the work that you already spent on evaluating options) and have that up for feedback for a few days would surely benefit the product that you are working on and the relationship between company and community.

Answer 3

If engagement is not growing with the overall user base, a lower percentage of you are getting value from these interactions each year.

This is a premise many of us more dedicated users reject. In fact, one of SO's founders explicitly explained why he believed it was wrong: Optimizing for Pearls.

We believe asking questions on our site is a privilege, not a right. ... If we don’t do our part to cull the bad questions, then we risk alienating the true experts who provide what really matters: the answers!

...

Perhaps you’ve noticed a theme here. Incoming questions are a universal constant, all around us in countless billions. But answers — truly brilliant, amazing, correct answers — are as rare as pearls. Thus, questions are merely the sand that produces the pearl. If we have learned anything in the last three years, it is that you optimize for pearls, not sand.

...

That’s why we’re determined to keep question quality high, even at the cost of refusing a little sand. It’s true that you can’t have Q&A; without questions, but having the wrong sorts of questions is far more dangerous. The fastest way to kill any Q&A; site is to flood it with low-quality questions. I think Mark Trapp summed it up best in this meta answer.

Please go read that linked answer, as well.

SO's founding philosophy was that most users would derive value by learning something from existing high quality content, and we credit SO's success as a site to that approach. So please answer my question about why SO has so abruptly changed its philosophy on "increased engagement."

Answer 4

Become part of the community again, don't administer it from outside

What I am completely missing on your road map is that you actually try to be part of the community.
"You" as in: the entire SO company.

There is the famous concept of eating your own dog food.

SO employees need to frequently ask questions, answer questions, and moderate. Right now, I don't see this happening, I don't see you feel the same pain points as we do, but rather you stare at some KPIs like engagement or question closing rates as a proxy for quality. Every employee at SO should be answering at least one good question per week somewhere, for example.

Answer 5

User engagement - you are solving the wrong problems...

This answer may be skewed with Stack Overflow perspective, what I am saying here is not necessarily true for each and every site in the network.

You don't need more new users, nor people asking more questions

Stack Exchange sites work the best when people don't have to ask questions, because questions they might have are already answered and easy to find. For that you don't even need an account.

You don't need to make closing process more friendly

Problem with closing questions is that at the point when question requires closing it is already too late. No matter how much you improve and polish that process (besides the ability to close questions faster) you will never make that process "friendly" enough.

Improving question asking process also has its limitations. Main problem is that new users (and fair number of existing users) are not familiar with the rules. And even less new users is aware that posting poor question or answers can lead to post ban.

Make all rules clear before posting

When I say that, I mean not leaving to the user to go around and look for rules - make them blatantly obvious. Especially, the fact about post ban. I had reputation well over 3000 when I first realized that post ban exists.

Also, give people access to all their deleted content. Current process where people have to go to Meta and beg moderators to dig up their deleted posts is far from ideal. And most people don't even know that their deleted posts also count towards ban.

Answer 6

Thank you, this sounds extremely promising.

• A clear roadmap is a welcome turnaround from the "6–8 weeks" meme.

• Some of the items seem to show a growing understanding that helping curating users help the question askers helps Stack Exchange as a whole.

• The announcement of network-wide question asking improvements, better close voting UX, and hiding closed questions feels like the long-awaited rain after a drought.

  Especially the last one: back when I was active in curation, we'd often downvote closed questions just to get them hidden from the site's front page. That sucked for everyone involved. This new approach has the potential to be much better. And I remember the years of lobbying that went into the question wizard. It's great that it will finally become usable more widely.

While I'm still very sceptical of these new noises coming out of the Stack Overflow offices, I'm quite glad for the general direction of these announcements and hope everything continues along these paths!

Answer 7

We are making fundamental changes to how our close question system works so that it’s a friendlier experience and more educational for post authors, while making it easier to edit and reopen closed questions, and reduce the burden on curators.

When can we expect to see some more information on this? I'm really interested to see what your mock-ups are, and would prefer to be informed every step of the way as you release these changes, and would love to be able to contribute to its design.

Questions that get voted to be closed will be hidden, giving authors the opportunity to improve their questions in private.

I have a few questions about this...

1. Will users with the cast close and reopen votes privilege be able to see these hidden posts?
2. How will you relay to the author of a closed question what they can do to get their post reopened?
3. How can other users assist authors of a closed question to get their question reopened, especially if it's hidden?

I feel as though this change, once refined, could definitely increase the amount of questions that get refined and reopened. One of my concerns, however, is that hiding these posts will eliminate the possibility of helpful users stepping in and instructing the user how to better their question, because most of the time, helpful edits are made by other contributors rather than the question author to get questions reopened.

General questions:

4. Will you be open to discussion on these changes as you release them incrementally?
5. Would you be willing to roll back some of these fundamental changes if the community has widespread concerns over it?

Thank you for keeping us updated on your plans for the future. Please continue to keep us informed.

Additionally, try to open up channels of communication on the changes you plan to make, and ensure that everyone has their chance to contribute to these (very large!) changes.

Answer 8

Very promising, especially that Meta is included in the discussion.

One question springs to mind. You say

Though our active user base continues to grow, our engagement has remained the same. What this means is that while more users are coming to the site every month, the number of users who engage meaningfully in the site does not increase proportionally.

To change this dynamic, we will balance investing in improving our tools and features that benefit our long-term users with initiatives that convert new users into engaged ones.

The dynamic suggests that as engaged users leave, new ones rise to take their place.

I'll ask the stupid question "Why do you want more engaged members?" The idea that you'd like more engaged members suggest that there's something about the stacks that isn't getting done that somebody want to see get done. Is it that questions are going unanswered? Is it community moderation tasks aren't getting done? Is it that non-engaged users are waiting too long for their answers? The language you use seems to suggest that we need more engaged members to adequately deal with the throughput, but it can be a valuable exercise to make sure that everybody understands the real goals.

Of course, it's entirely possible that that goal really is simply to have a higher number entered in a spreadsheet box, but it doesn't feel like this is the answer.

Answer 9

Since you have mentioned the "Developer Survey" as a way to understand the community in that fancy image, I just wanted to point at something that might not be a big deal, but it was disturbing enough to completely push me away from such things.

I have been participating in the developer survey since four or five years ago. I liked the way the questions were designed and taking that survey gave me a feeling of inclusiveness. Last year, when I was going to take it again, I faced a geo-blocking message. I complained about it on meta and was somehow amazed about the warm response I got that gave me a sense of faith in this great community. This gave me some real hopes that at least there are some places on the internet where we won't be dealing with such discrimination.

But to my further amazement, when I tried to take the survey this year, I faced the same message! Though I shouldn't have been surprised regarding the other incidents and messes which gave the impression that nothing is impossible/unexpected anymore. But I really hoped that since the problem was immediately fixed last year, it won't happen again. Well, this was kind of offensive and I decided not to participate in any kind of such activities anymore, even if it is fixed in the future.

Now you might be wondering what this has to do with the things I said? To be honest, I don't know exactly. But I just wanted to point at the fact that the damage to the community's trust (well at least, my trust as a droplet in the ocean) is accumulating and the need to take some serious actions is becoming more evident everyday.

Answer 10

"Though our active user base continues to grow, our engagement has remained the same. What this means is that while more users are coming to the site every month, the number of users who engage meaningfully in the site does not increase proportionally.

To change this dynamic,..."

Yikes. This kind of approach really worries me. I know this isn't a popular opinion (been down-voted before badly, I mean we're talking meta-effect badly), but I truly believe the most valuable asset SE has are those users who fly in, get an answer, and fly away. These are the ones that cement into the broader culture the idea that StackExchange (StackOverflow) is the place to go to for your answer, even if they never post a question, post an answer, or even register. This makes it all worthwhile for those of us who do take part in the sustenance of this site. I myself like participating, and will probably continue to do so at my varying levels for years to come.

But if you start to chase off the very casual users, this site will eventually become forgotten in the haze of near-infinite internet possibilities for places to find answers. I do understand the corporate philosophy that constant growth, however implausible or impractical, is always a good thing. I just think you ought take a hard look at how you define 'growth', and how you can profit from that, before putting into place things that in anyway attempt to force participation in exchange for information.

Answer 11

Welcome to StackExchange, Ms. Dietrich! (This answer was written before I noticed your second initial post.)

About your post, in general and in context of recent events

Yours is an interesting post, with some promising and some worrying aspects. But most of all - it is disconnected from what has been happening here on the network over the past few years, and specifically over the past several months.

I suppose that this is in part only natural, with you being new on the network - but if SE inc. as a company wants to have a "clean slate", this needs to be stated explicitly, and also - answers need to be given and amends must be made before this can actually happen. This is particularly worth emphasizing because we have already noticed how more veteran company employees occasionally make "How things are going to work from now on"-type posts which completely ignore what they had done before, said before and been told or asked before. Again - this is not personal criticism of you.

Also, your post describes planned unilateral action. While you mention trying to understand the community, engagement etc. - the underlying framework is of the company making decisions about the network and then enforcing them. The tradition - recently more honored in the breach than in the keeping - is that many policy decisions are reached through a process on meta sites and in chat-rooms, by the active users; and while moderators and company employees have a privileged position in that decision-making process, it still a community process. This applies specifically to some of what you're considering: Changing the question-asking UI, changing the closing process and so on.

Remember the network's raison d'etre

I concur with most of the criticism and apprehension voiced in other answers. Please bear in mind that the network exists to build up a repository of useful answers to questions, not to maximize people's active interaction.

Thus, for example, if you're thinking about an email newsletter, you need to ask yourself: "How will this help the production or improvement of answers to questions?" If the answer is "it won't", then it's probably not a good idea (and will also be poorly received on account of that fact.)

The above is a more nuanced point then one might first assume. Specifically, encouraging useful/meaningful questions to be asked is a valid goal w.r.t. the criterion above, because such questions beget good answers. But artificially encouraging "engagement" of users means more work for the active users / editors and more noise in the body of questions.

The problems with statistics

Your "understanding the community" rubrique lists the "Site Satisfaction Survey" and "Developer Survey" as actions. I believe these are not likely to significantly help your understanding of the community. This is for at least 4 reasons:

1. Sampling issues - especially with the first one. The population of survey-responders is not representative of the population you need to understand.
2. You only get answers to what you already know you want to ask.
3. A lot of insights can not be obtained by asking a straight-up question of individuals. Perhaps one might even say the more important insights about a group or community cannot be thus obtained.
4. It's not a single community! There are some commonalities; and some of the more active users live in more than one network site - but some of the SE sites are literally worlds apart. From what I gather, interpersonal skills SE is almost nothing like Stackoverflow. And while the latter is larger than all others, they can't just be lumped together. In fact, a "Developer Survey" is by definition irrelevant to most network sites: Biking, religion, cooking, politics, history...

Answer 12

From the blog post:

Questions that get voted to be closed will be hidden, giving authors the opportunity to improve their questions in private. Our hope is to make it easier on everyone by providing clearer guidance and encouraging better questions.

This does not encourage better questions. It just hides bad content.

Also, this seems like something that will reduce the community's power to close questions. In other words, more bad content is allowed into the sites for the sake of pretending to be a friendlier place for people who have a hard time with the stack Q&A format.

One thing that I like about Stack Overflow, for example, is that when I have a problem I don't have to search through s... Erm, sand to find what I want or need.

I really love that post from 2011, specially this:

We believe asking questions on our site is a privilege, not a right. If, after a few fair attempts, you haven’t been able to prove that your contributions to a particular Stack Exchange make it at least … not-worse … then we reserve the right to refuse your questions. If we don’t do our part to cull the bad questions, then we risk alienating the true experts who provide what really matters: the answers!

You already took a step away from that philosophy when you made upvotes to questions worth as much as upvotes to answers. Now it seems like tolerance to sand will increase too.

If you want more users in lieu of quality, you can save a lot of effort by just changing format. Rebuild the network as a series of forums. Just saying.

Answer 13

On the topic of roadmaps...

I'm an avid fan of games, especially AAA PC games, but also smaller indies. Lately, the game industry, and especially the AAA industry, has gotten to the worrying trend of releasing games in an unfinished state with tons of missing features, a huge amount of bugs and a core gameplay loop that's generally considered to be unsatisfactory by the target audience (don't worry, I have a point to make). Then, on the day of release, the developers release a roadmap, similar to what you did, with a list of intended feature updates and changes to the game, in hopes of retaining the customers while they spend another year or 2 finishing the game. (see also Early Access)

There is a problem that has consistently appeared in this situation, especially with the most egregiously unfinished game releases, like Anthem, Fallout 76, the Epic Games Store,..., namely the problem of the inflexibility of these roadmaps. through whatever reason, the roadmap does not get fulfilled, planned features get delayed or even canceled, and if they do get released they're poorly implemented. This is often because there are pressing issues with the game that take up development time, either bugs from release that need to be fixed or sudden appearances of problems that weren't noticed before and need to urgently be fixed. In most cases, roadmaps miss their targets and even get canned after a couple months of struggling.

Obviously Stack Overflow is generally stable and not like an unfinished AAA game in terms of feature completeness and amount of bugs, but after being faced with tons of AAA roadmaps that inevitably miss their mark, the concept of a roadmap, in my eyes and those of other gamers, has become synonymous with mismanagement, missed goals and slow reactions.

What are the plans of Stack Overflow to ensure that the roadmap it is setting out will in fact succeed, both in terms of completing the goals on it and ensuring the goals on the roadmaps will have the intended effects?

Answer 14

I already tried to leave this as a comment on the blog post itself, but of course it didn't get approved (can't have dissenting opinion on the front page, can we?). Well, let's see if this gets deleted as well.

---

I would like to point out that all of the changes that the community has been criticizing over the past months (if not years) are still in place. Every single one of them. Nothing has been repealed or amended.

In light of this the statement

Increase user engagement

We are launching a series of educational email campaigns for new users, non-active users, question askers, and question answerers. These series are designed to take a proactive approach in providing guidance and support for all users.

reads to me like the company is still operating under the impression that they "didn't explain things well enough to us."

Lack of explanation on your part or lack of understanding on our part is not the problem here.

Answer 15

There's one piece of this blog post that strikes me as non-sequitur and a wrong handling:

Improve feedback metrics

Our users have shared that they don’t feel heard by Stack Overflow. We will define a dashboard for the various types and methods of feedback and dedicate time to processes and outreach that identify ways we can be serving our users better and facilitating better two-way communications.

Uhhh...what?

The situation of having a community that doesn't feel heard is NOT solved by "setting up a dashboard." That's what you do if you have a bunch of machines that you need to monitor. NOT real people.

What you need to do is look at what is being said by the people who say they don't feel heard, and seek to understand them. And then respond to them appropriately in such a way that they actually know that you have understood them (which may also imply that you are doing something about whatever it is they brought up).

This is entirely and only a human process and cannot ever be done by machines. And certainly cannot be done by a dashboard or by numerical information.

---

Overall, it's nice to see what looks like a greater intention to engage with the community. Hope it will pan out. I just had to say something about that one paragraph because it seemed so entirely off the wall.

Answer 16

In January, we launched our monthly blog series “The Loop” to share research and feedback insights and methodology.

Please, fix the blog first before using it for things which need feedback. Currently blog is very poor: it lacks any kind of intergration with stack exchange, it doesn't looks like it's desinged for community (it has very "stinky" promoting design), it will not let us to give feedback on feedback, etc.

My suggestion would be: if you plan to post something for us (stack exchange members), do it on meta please.