Community and Moderator guidelines for escalating issues via new response process (March-April 2020)

Question

Last week I posted the company’s commitment to responding to Meta and the Moderators here. If you haven’t read it yet, doing so before reading this one is recommended, since it gives a higher-level perspective of what the company is committing to:

• it introduces the new process for managing community feedback the CM team has created;
• it presents a timeline for testing that process, between March 16, 2020 and April 30, 2020;
• it outlines the planned communications around that test.

As promised in that post, this one contains the guidance to ensure moderators understand when to escalate issues that they feel need to be addressed by staff, and when not to. It also has some guidance for the overall community of what posts should be brought to the moderators’ attention as candidates for escalation. This guidance is applicable for the duration of the test, and will be revisited and updated once it concludes.

There is a sister post on the Moderator Team that includes guidance for moderators on how to escalate issues from there too. That post can also be used by moderators from the whole network to discuss how to handle requests for escalation, or whether or not particular posts make for good candidates.

What makes a good candidate for escalation?

New questions:

For any new question posted during the test period (anything posted from March 16, 2020 on), consider the following questions:

• Is the question a feature request that looks like it has community support?
• Is it a bug report that others have been able to reproduce?
• Is the question only fully answerable by an employee?

If you replied “yes” to at least one of the above, then that question is a good candidate for escalation.

Older questions:

We realize that there are a lot of outstanding posts all over the various Meta sites on the network that have not been addressed by staff — some of them posted a long time ago too. To strike a balance between having the CM Team be flooded by all of these and completely leaving them out for this test, we propose instead that you focus on resurfacing old questions that relate to either something only a Community Manager would be able to respond to, or to things currently being worked on by the various product teams (which allows us to easily find these older discussions so that we can use them as part of our research).

So, if a post meets the guidance for new questions above, but it is was posted before the test began (before March 16, 2020) it also needs to fall in one of the below buckets to be a good candidate for escalation:

• Community-specific concerns: anything, from policies to community guidance, that only a Community Manager could reasonably respond to. This includes tweaks to help center pages, for instance, as well as any such modifications that do not require dev time.
• Close UX: any posts having to do with the general experience and mechanics of question closing; the problems users face as asker, answerer, curator, or moderator when it comes to the current system; any feature requests or suggestions, etc. (mentioned in our Q1 roadmap, and with more details to come when Q2’s plan is shared.
• User guidance on how the site works: any posts around educating users about how Stack Overflow works, how to ask good questions, how to give good answers, best practices for curators etc. Posts that are specific to particular user segments (new askers, new answerers, new curators, etc.) are also good candidates (we’re currently working on a series for emails to educate users on various parts of the system, but this feedback will also be useful for future efforts such as better onsite user onboarding).
• Question following: there is a dedicated meta post to solicit feedback for the upcoming work on this, but any previous feature requests or general feedback/frustrations around question favoriting, ability to follow, ability to get notified on questions you're interested in, ability to turn off notifications/noise, etc., would be good candidates.
• Oneboxing: the oneboxing feature for the GitHub/Stack Overflow for Teams integration was built with other integrations in mind, and with the intent of rolling it out to the public Q&A sites too. Anything around general oneboxing in Q&A is a good candidate, along with things like including code snippets in questions, integrating with other dev tools, issues with maintaining question accuracy as the code or question evolves, etc.
• Jobs application management: we are doing some work on our application history page, so any posts around better management of Jobs applications make for good candidates.

Some of these are SO-specific, but most will eventually apply to all of the network sites, even if with some simplifications (just like what happened with the new “ask question” page). In addition to those larger buckets, for each non-SO (or MSE) site in the network, the CM Team also wants to see that special feature your community has been asking to be enabled for a while now (Mathjax, syntax highlighting, etc.), or that particular warning that would help your new askers — so give us a top 5 of outstanding site-specific customizations from your site, and just make sure their score is positive. Note that features that need dev time and that are only applicable to your site are unlikely to be given a high priority.

Oh, and hopefully it is redundant to say this, but just for the sake of clarity: don’t repost old posts just so they can get attention as a way to game this system, please.

How do regular users nominate a post for staff attention?

If the question meets the criteria above, flag the post for moderator attention using the “in need of moderator intervention” option, making sure to provide a link to this post for context. Be as clear as you can about why you think the post is a good candidate.

To prevent overloading moderators with flags, please avoid going on a flagging spree. While the CM Team is (mentally…?) prepared for an initial wave that’s possibly (hopefully?) gonna subside in the weeks following the start of this test, we want to remind you that your moderators will be the first line of people dealing with these. The CMs will work with the mods to make sure this doesn’t generate a huge increase in their workload, and try to alleviate it as much as possible — but we are also relying on you not bombarding them with tagging requests.

How do moderators escalate a post for staff attention?

How do moderators handle flags nominating a post for escalation?

Refer to the guidance above on how to tell if a post is a good candidate for escalation. Other than that, just use your judgement as you would for handling any other flag. There may be cases where you want to mark the flag as helpful but don’t feel like adding the tag is necessary - that’s fine: again, use your best judgement. If you’re not adding the tag, try to use the flag response field to explain why, so the flagger also gets some information about the decision.

As mentioned above, it’s likely that there will be a large initial wave of post being flagged as candidates for escalation. With that in mind, it is also likely that there’ll be a corresponding inflated number of posts being escalated by moderators — that is fine.

If you’re unsure, talk to your fellow mods or feel free to ping a CM in the Teachers’ Lounge about what they think. It’s OK if moderators err on the side of over-escalating issues, rather than under-escalating them: if the CM Team determines something could have been answered without having to elevate to staff, it presents a good opportunity to point moderators to where that information could have been found, as well as to tweak this guidance.

If you escalated an issue by adding the status-review tag, but something caused it to be “solved” — maybe someone from the community could actually answer it and did so; maybe a bug was really an issue on the user’s side; etc. — please ping a CM in the Teachers’ Lounge explaining the situation. We will then figure out with you what to do about that particular issue (which will likely mean removing the tag, at least).

Ok, how do moderators actually escalate a post, then?

Escalating a post is as easy as adding the status-review tag. Doing so ensures that post is picked up by a feed that puts the question on our internal tracking system.

If you are a moderator, refer to the section above describing what makes for good candidates for escalation — if a post fits, add the tag (there’s no need to go through the flagging process for regular users).

If the post already has the status-review tag, ping a CM in the Teachers’ Lounge, and we’ll add it to our system manually.

What happens once a post is escalated?

The CM Team will categorize and prioritize the post, and will then pass it along to the relevant team, which can be the CMs or any relevant product team. It will then be worked into that team’s existing weekly workflow, with the intent of being replied to as soon as the team can manage to.

Note that the commitment being made is to respond to as many posts as possible — that could mean answering or leaving a comment, or adding a different status tag. This doesn’t necessarily mean implementing feature requests or fixing bugs — hopefully that will sometimes happen, though.

Some of the posts will likely still be unreplied to: we’ll keep track of all of those numbers, and report them once the test is over. This will help us define targets for how many posts we can respond to, and how quickly we can do so. We will then review these targets quarterly to make sure we’re setting realistic targets and meeting them.

---

We hope this guidance is clear, and believe it’ll help us achieve the goals outlined in the new process we are testing. It should also answer most of the outstanding questions in that original post. Since this is as much of a test for us, internally, as for all the communities and moderators, feel free to request clarification on anything that’s unclear or confusing in an answer below. Feedback to be taken into consideration for tweaking the process and/or guidelines for the future is also welcome.

Answer 1

I'd like to surface one aspect that I hope folks will be sensitive to.

Allowing MSE moderators to be the arbiters of which feature requests receive this tag, and thus receive attention from SE, is giving power to three users who were selected by Stack Exchange as moderators. They weren't democratically elected by MSE users; they were appointed. Moreover, they were selected to moderate MSE, not to prioritize what features are most important to work on; but this policy also makes them gatekeepers for feature requests. (This is a sense in which MSE mods differ a bit from MSO mods.)

This post states eligibility criteria for a post to be considered for the tag and a process for nominating posts for consideration by the mods, but it leaves a lot of discretionary power to them about which they will and won't feature with the tag.

This has the potential to create friction in the future. It could be setting MSE mods up to receive some unhappiness from users who are unhappy that the post they nominated was not selected to receive the tag, and might force MSE mods to take a position on debates where they don't or shouldn't get involved (by forcing them to accept or reject a proposed tag).

I'm not saying that you shouldn't do this. It's great you are looking at ways to filter which posts will receive attention and to make it feasible for the company to be engaged on Meta. It seems reasonable to test this process for now. However, I hope that if this becomes a longer-term process, some thought is given to a process that involves the community to a greater extent.

Answer 2

and we’ll add it to our system manually

Any chance that you will make this list of tasks with priorities public? Would be nice to see it.

Answer 3

We hope this guidance is clear

On a first glance, it feels like that. If it is really sufficient, that experiment will tell. That is the whole point of being agile: don't assume you know everything up front. Provide something to start with, then start trying. Adapt, repeat.

Exactly as outlined here. At least me, I feel comfortable enough to enter that phase, given the information you provided to us here.

It is obvious that quite some thought went into this proposal. And the level of commitment behind it is amazing.

From that point of view, my biggest concern right now: that the initial wave is really large, and overburdening the people tasked to make the decisions.

So: we users should be very conscientious here, and those doing the work should let us know when "too much" is coming in. There is no shame in admitting that available resources are limited. So everyone manage expectations here!

Finally, the question for the community is: do we need some hard rules to determine that "community support" aspect?! Like a minimum number of upvotes within a certain period? (And I think: that aspect is worth its own discussion, but that one was closed as DUP to How is consensus determined on Meta sites? )

Answer 4

If you escalated an issue by adding the status-review tag, but something caused it to be “solved” — maybe someone from the community could actually answer it and did so; maybe a bug was really an issue on the user’s side; etc. — please ping a CM in the Teachers’ Lounge explaining the situation.

My instinct for handling cases like this would be to simply remove the status-review tag. Would that not be sufficient? Is the communication with your internal tracking system only one-directional, such that adding the tag will add the question to your tracker, but removing the tag will not remove the question without manual intervention? Is it really required to ping a CM over this?

To prevent overloading moderators with flags, please avoid going on a flagging spree.

Thank you for calling that out explicitly. That was my biggest concern with this system when it was first proposed to moderators in private channels. It remains my biggest concern, as a moderator of a site whose associated Meta is itself larger and more active than pretty much any other site in the network. The GIF is what we look like on a good day (note that the head stays generally above the water). I really don't want to be inundated with bazillions of flags on a giant backlog of ignored requests that someone wants a CM to follow up on. There are a lot of someones.