Last week I posted the company’s commitment to responding to Meta and the Moderators here. If you haven’t read it yet, doing so before reading this one is recommended, since it gives a higher-level perspective of what the company is committing to:

  • it introduces the new process for managing community feedback the CM team has created;
  • it presents a timeline for testing that process, between March 16, 2020 and April 30, 2020;
  • it outlines the planned communications around that test.

As promised in that post, this one contains the guidance to ensure moderators understand when to escalate issues that they feel need to be addressed by staff, and when not to. It also has some guidance for the overall community of what posts should be brought to the moderators’ attention as candidates for escalation. This guidance is applicable for the duration of the test, and will be revisited and updated once it concludes.

There is a sister post on the Moderator Team that includes guidance for moderators on how to escalate issues from there too. That post can also be used by moderators from the whole network to discuss how to handle requests for escalation, or whether or not particular posts make for good candidates.

What makes a good candidate for escalation?

New questions:

For any new question posted during the test period (anything posted from March 16, 2020 on), consider the following questions:

  • Is the question a feature request that looks like it has community support?
  • Is it a bug report that others have been able to reproduce?
  • Is the question only fully answerable by an employee?

If you replied “yes” to at least one of the above, then that question is a good candidate for escalation.

Older questions:

We realize that there are a lot of outstanding posts all over the various Meta sites on the network that have not been addressed by staff — some of them posted a long time ago too. To strike a balance between having the CM Team be flooded by all of these and completely leaving them out for this test, we propose instead that you focus on resurfacing old questions that relate to either something only a Community Manager would be able to respond to, or to things currently being worked on by the various product teams (which allows us to easily find these older discussions so that we can use them as part of our research).

So, if a post meets the guidance for new questions above, but it is was posted before the test began (before March 16, 2020) it also needs to fall in one of the below buckets to be a good candidate for escalation:

  • Community-specific concerns: anything, from policies to community guidance, that only a Community Manager could reasonably respond to. This includes tweaks to help center pages, for instance, as well as any such modifications that do not require dev time.
  • Close UX: any posts having to do with the general experience and mechanics of question closing; the problems users face as asker, answerer, curator, or moderator when it comes to the current system; any feature requests or suggestions, etc. (mentioned in our Q1 roadmap, and with more details to come when Q2’s plan is shared.
  • User guidance on how the site works: any posts around educating users about how Stack Overflow works, how to ask good questions, how to give good answers, best practices for curators etc. Posts that are specific to particular user segments (new askers, new answerers, new curators, etc.) are also good candidates (we’re currently working on a series for emails to educate users on various parts of the system, but this feedback will also be useful for future efforts such as better onsite user onboarding).
  • Question following: there is a dedicated meta post to solicit feedback for the upcoming work on this, but any previous feature requests or general feedback/frustrations around question favoriting, ability to follow, ability to get notified on questions you're interested in, ability to turn off notifications/noise, etc., would be good candidates.
  • Oneboxing: the oneboxing feature for the GitHub/Stack Overflow for Teams integration was built with other integrations in mind, and with the intent of rolling it out to the public Q&A sites too. Anything around general oneboxing in Q&A is a good candidate, along with things like including code snippets in questions, integrating with other dev tools, issues with maintaining question accuracy as the code or question evolves, etc.
  • Jobs application management: we are doing some work on our application history page, so any posts around better management of Jobs applications make for good candidates.

Some of these are SO-specific, but most will eventually apply to all of the network sites, even if with some simplifications (just like what happened with the new “ask question” page). In addition to those larger buckets, for each non-SO (or MSE) site in the network, the CM Team also wants to see that special feature your community has been asking to be enabled for a while now (Mathjax, syntax highlighting, etc.), or that particular warning that would help your new askers — so give us a top 5 of outstanding site-specific customizations from your site, and just make sure their score is positive. Note that features that need dev time and that are only applicable to your site are unlikely to be given a high priority.

Oh, and hopefully it is redundant to say this, but just for the sake of clarity: don’t repost old posts just so they can get attention as a way to game this system, please.

How do regular users nominate a post for staff attention?

If the question meets the criteria above, flag the post for moderator attention using the “in need of moderator intervention” option, making sure to provide a link to this post for context. Be as clear as you can about why you think the post is a good candidate.

To prevent overloading moderators with flags, please avoid going on a flagging spree. While the CM Team is (mentally…?) prepared for an initial wave that’s possibly (hopefully?) gonna subside in the weeks following the start of this test, we want to remind you that your moderators will be the first line of people dealing with these. The CMs will work with the mods to make sure this doesn’t generate a huge increase in their workload, and try to alleviate it as much as possible — but we are also relying on you not bombarding them with tagging requests.

How do moderators escalate a post for staff attention?

How do moderators handle flags nominating a post for escalation?

Refer to the guidance above on how to tell if a post is a good candidate for escalation. Other than that, just use your judgement as you would for handling any other flag. There may be cases where you want to mark the flag as helpful but don’t feel like adding the tag is necessary - that’s fine: again, use your best judgement. If you’re not adding the tag, try to use the flag response field to explain why, so the flagger also gets some information about the decision.

As mentioned above, it’s likely that there will be a large initial wave of post being flagged as candidates for escalation. With that in mind, it is also likely that there’ll be a corresponding inflated number of posts being escalated by moderators — that is fine.

If you’re unsure, talk to your fellow mods or feel free to ping a CM in the Teachers’ Lounge about what they think. It’s OK if moderators err on the side of over-escalating issues, rather than under-escalating them: if the CM Team determines something could have been answered without having to elevate to staff, it presents a good opportunity to point moderators to where that information could have been found, as well as to tweak this guidance.

If you escalated an issue by adding the tag, but something caused it to be “solved” — maybe someone from the community could actually answer it and did so; maybe a bug was really an issue on the user’s side; etc. — please ping a CM in the Teachers’ Lounge explaining the situation. We will then figure out with you what to do about that particular issue (which will likely mean removing the tag, at least).

Ok, how do moderators actually escalate a post, then?

Escalating a post is as easy as adding the tag. Doing so ensures that post is picked up by a feed that puts the question on our internal tracking system.

If you are a moderator, refer to the section above describing what makes for good candidates for escalation — if a post fits, add the tag (there’s no need to go through the flagging process for regular users).

If the post already has the tag, ping a CM in the Teachers’ Lounge, and we’ll add it to our system manually.

What happens once a post is escalated?

The CM Team will categorize and prioritize the post, and will then pass it along to the relevant team, which can be the CMs or any relevant product team. It will then be worked into that team’s existing weekly workflow, with the intent of being replied to as soon as the team can manage to.

Note that the commitment being made is to respond to as many posts as possible — that could mean answering or leaving a comment, or adding a different status tag. This doesn’t necessarily mean implementing feature requests or fixing bugs — hopefully that will sometimes happen, though.

Some of the posts will likely still be unreplied to: we’ll keep track of all of those numbers, and report them once the test is over. This will help us define targets for how many posts we can respond to, and how quickly we can do so. We will then review these targets quarterly to make sure we’re setting realistic targets and meeting them.

We hope this guidance is clear, and believe it’ll help us achieve the goals outlined in the new process we are testing. It should also answer most of the outstanding questions in that original post. Since this is as much of a test for us, internally, as for all the communities and moderators, feel free to request clarification on anything that’s unclear or confusing in an answer below. Feedback to be taken into consideration for tweaking the process and/or guidelines for the future is also welcome.

  • 7
    Is it permitted to flag old posts about things that should be able to fixed in a relatively short period of time? I didn't know about the code: search term for searching inside code blocks until today, and it should be as easy as adding 1-2 sentences to /help/searching. – S.S. Anne Mar 12 '20 at 21:44
  • 14
    What happens to all these questions, the ones currently tagged status-review? Do they get dumped into your feed at the start? Will you remove the tag from all those automatically and then add it back to some of them manually? – Rand al'Thor Mar 12 '20 at 21:46
  • 5
    @S.S.Anne yes, please feel free to flag those - just don't mass do it. – Cesar M Mar 12 '20 at 22:11
  • 6
    @Randal'Thor they are not going to be dumped into this test, no. What we do with them (either retag, use a different tag or what) will be determined after the test – Cesar M Mar 12 '20 at 22:11
  • 5
    I should add, @Randal'Thor, that many of those are already being tracked by the Community Dev Team. – JNat Mar 12 '20 at 22:42
  • 4
    @JNat I think it will be really important to address those, likely by removing the tag for posts not currently being tracked and keeping it for those that are (hopefully via script rather than manually :) ). Certainly reasonable in the interim to let them hang in limbo, though. – Bryan Krause Mar 12 '20 at 23:26
  • 2
    Out of those questions that are already posted, what to do with those with tags other than status-review that aren't implemented yet and aren't declined or deferred? (e.g. status-planned, status-reproduced, etc.)? – Sonic the Curiouser Hedgehog Mar 12 '20 at 23:40
  • 4
    One problem with "Is it a bug report that others have been able to reproduce?" is that some people are quick to not be able to see and reproduce a bug and downvote or close no-repro; not realizing that there is more than one server at more than one location, and that the software on the user's end is also sometimes relevant. In some cases of deployment the site's revision number or API build number causes the commenter's misunderstanding. – Rob Mar 13 '20 at 0:45
  • 2
    Do you have a (simplified) flow chart with this? There is a lot of information in your post but I suspect the core of it is much simpler than it looks, would prefer to avoid the confusion. – Mast Mar 13 '20 at 7:23
  • 1
    Let's leave those as is for the test, @SonictheAnonymousHedgehog: those are likely on someone's radar, given they have those tags. Once the test is concluded, we'll reassess and decide what to do going forward. – JNat Mar 13 '20 at 11:11
  • 4
    The "What makes a good candidate for escalation" part is the most relevant one, @Mast. But I'll see if I can make some time today to put this in a flowchart (if not, I'll try early next week). – JNat Mar 13 '20 at 11:13
  • 2
    Edited the question slightly to address that question, @S.S.Anne – JNat Mar 13 '20 at 11:41
  • 1
    "to things currently being worked on by the various product teams" Maybe I skimmed over it, or is the ul list of topics, but how moderators are supposed to know what the product teams are working on? – Braiam Mar 13 '20 at 22:30
  • 1
    It is the list that follows, yes, @Braiam :) – JNat Mar 13 '20 at 23:40
  • 2
    They aren't likely to get less attention if not escalated, @Marijn, since staff are still looking at MSO and MSE all the same, and can add the tag ourselves — but you can also flag it to be escalated, yes, as long as it is well received and others have been able to repro (that way we make sure it is accounted for). – JNat Mar 19 '20 at 13:31

I'd like to surface one aspect that I hope folks will be sensitive to.

Allowing MSE moderators to be the arbiters of which feature requests receive this tag, and thus receive attention from SE, is giving power to three users who were selected by Stack Exchange as moderators. They weren't democratically elected by MSE users; they were appointed. Moreover, they were selected to moderate MSE, not to prioritize what features are most important to work on; but this policy also makes them gatekeepers for feature requests. (This is a sense in which MSE mods differ a bit from MSO mods.)

This post states eligibility criteria for a post to be considered for the tag and a process for nominating posts for consideration by the mods, but it leaves a lot of discretionary power to them about which they will and won't feature with the tag.

This has the potential to create friction in the future. It could be setting MSE mods up to receive some unhappiness from users who are unhappy that the post they nominated was not selected to receive the tag, and might force MSE mods to take a position on debates where they don't or shouldn't get involved (by forcing them to accept or reject a proposed tag).

I'm not saying that you shouldn't do this. It's great you are looking at ways to filter which posts will receive attention and to make it feasible for the company to be engaged on Meta. It seems reasonable to test this process for now. However, I hope that if this becomes a longer-term process, some thought is given to a process that involves the community to a greater extent.

  • 10
    Suggestion: for MSE in particular, maybe we could create one big meta thread where all the answers are suggestions for which posts to nominate as status-review. After an answer reaches a certain score, the moderators add the tag to the linked post (and delete that answer to make room for others to rise up). This would pass the decision-making responsibility from unelected moderators to the wider community. – Rand al'Thor Mar 12 '20 at 23:13
  • 1
    @Randal'Thor I reckon that would swing things far too much in the opposite direction. We definitely still want some gatekeeping to prevent "importance" being determined by sheer popularity alone. Otherwise we might as well just ignore MSE and child meta issues entirely and only focus on MSO problems. – goldPseudo Mar 12 '20 at 23:18
  • 40
    I'm less concerned with allowing MSE moderators this responsibility, not only because I trust them individually but also because moderation is not free from community review, whether moderators are appointed or elected. I really doubt there will be much daylight between what MSE mods and the community as a whole agrees should be tagged. On the other hand, I am definitely concerned with burdening MSE moderators with this responsibility, for the reasons stated here. – Bryan Krause Mar 12 '20 at 23:24
  • 3
    It's worth noting that the primary intent of MSE moderators is mainly to reduce workload on SE staff moderating the site. Prior to November 2018, this site was moderated entirely by staff members. (As an example of where MSE mods don't have a say, it's in the featured tag here since that's network-wide rather than per-site.) – Sonic the Curiouser Hedgehog Mar 12 '20 at 23:42
  • 10
    We actually can set those tags. We just typically often don't because we don't have the level of insight into the actual status of the flags in question. – Journeyman Geek Mar 13 '20 at 0:27
  • 4
    I can't make my mind up if this post is preemptively accusing MSE mods being incompetent or trying to shield them from mishaps. I opted for the first explanation as that seems to be the strongest message. This post has my down vote for that. If it can be edit to make its intent more obvious I'm happy to revisit. – rene Mar 13 '20 at 7:35
  • 3
    @rene My reading of this post is that it's nothing about the MSE mods themselves, their competence or otherwise, and just about the principle of giving unelected officials all the power to decide which feature requests from the whole network of users are worth putting forward to implemented. Even being sure the current trio would do a good job at it, can't you see why that might not be a good look? – Rand al'Thor Mar 13 '20 at 8:42
  • @Randal'Thor no, even with your extended explanation, which is appreciated, I still don't understand why that might not be a good look. It still feels as taking a stab at mods, which in general and on MSE specifically will always face opposition from me. – rene Mar 13 '20 at 9:04
  • 3
    @rene OK, assuming you're one of those who's lost a lot of trust in SE recently, thought experiment: let's say the company makes some more bad decisions, Meta mods object, SE summarily fires them all and replaces them with yesmen who'll support the company unquestioningly on everything. But there's already an established policy to leave status-review tagging here entirely up to the Meta mods. Would you think that's a bad thing? – Rand al'Thor Mar 13 '20 at 9:07
  • 2
    I get the concerns being raised in this answer, with regards to the possibility of overloading the moderators, and to the fact that they are appointed and not elected. Re overloading: as the post notes, the CMs will work closely with the moderators so that we can adjust our approach if needed and assist the mods as much as possible. – JNat Mar 13 '20 at 11:20
  • 5
    (cont'd) Re not elected: I want to highlight the fact that this is a test, and that after it's concluded we can reevaluate how we'll operate going forward. Also, the sister post on the Mod Team linked to in the question is a place where all mods from the network can talk about how to handle certain flags, or whether posts make good candidates, etc. So there's room there for the MSE mods to not have it all on them :) (will make an edit to make that more evident) – JNat Mar 13 '20 at 11:20
  • 4
    @rene, I'm definitely not trying to accuse the MSE mods or imply anything about them. I have a lot of respect for them. Rather, I wanted to raise a point about the principle. – D.W. Mar 15 '20 at 22:46
  • 1
    @JNat The test of appointed moderators began in November 2018. How long is that test supposed to take? – Sonic the Curiouser Hedgehog Mar 16 '20 at 20:07
  • 2
    Dunno, @SonictheAnonymousHedgehog. I said "this is a test" above as a reference to the escalation procedure, not that. – JNat Mar 16 '20 at 20:13
  • 2
    @FedericoPoloni The vast majority of moderator actions are non-controversial (unless people want to make controversy out of nothing as a troll). Comments are always ephemeral, so removing them gets less oversight. Closed questions can be reopened by vote. When there are concerns about moderation, users can (and do) raise the issue on Meta. – Bryan Krause Mar 27 '20 at 17:45

and we’ll add it to our system manually

Any chance that you will make this list of tasks with priorities public? Would be nice to see it.

  • 2
    Yep, if only to avoid people flagging the same stuff over and over again. – Jenayah Mar 13 '20 at 7:18
  • 13
    There are no plans for that during the test period. I agree that there would be some benefit to this, but dunno if it's up to me. We'll see what the test brings, and go from there. – JNat Mar 13 '20 at 11:29
  • @JNat thanks for the response. Will see (: – Suvitruf - Andrei Apanasik Mar 13 '20 at 11:38

We hope this guidance is clear

On a first glance, it feels like that. If it is really sufficient, that experiment will tell. That is the whole point of being agile: don't assume you know everything up front. Provide something to start with, then start trying. Adapt, repeat.

Exactly as outlined here. At least me, I feel comfortable enough to enter that phase, given the information you provided to us here.

It is obvious that quite some thought went into this proposal. And the level of commitment behind it is amazing.

From that point of view, my biggest concern right now: that the initial wave is really large, and overburdening the people tasked to make the decisions.

So: we users should be very conscientious here, and those doing the work should let us know when "too much" is coming in. There is no shame in admitting that available resources are limited. So everyone manage expectations here!

Finally, the question for the community is: do we need some hard rules to determine that "community support" aspect?! Like a minimum number of upvotes within a certain period? (And I think: that aspect is worth its own discussion, but that one was closed as DUP to How is consensus determined on Meta sites? )

  • 5
    I think it will be ...hard... to set any particular hard rules. Some issues pull a lot of attention and votes but really need a lot more iterating from the community before they become useful feedback (another way to think about this is that it is a question that gets tagged, but it may be a good answer that actually needs review). Others are very simple and maybe even a little boring, like an unintended consequence of a CSS tweak. For those I'd think it's best to escalate ASAP because it can likely be traced to a recent commit and resolved quickly, not really needing any more feedback. – Bryan Krause Mar 13 '20 at 1:38
  • 3
    @BryanKrause: And to add onto that, "hard rules" across the network don't make sense when community sizes (and thus votes/scores) can vary drastically. – V2Blast Mar 13 '20 at 3:27
  • Yeah, sorry. With "we" I specifically meant this place here. – GhostCat Mar 13 '20 at 4:19
  • 1
    As you say, I am hugely +1 on this post for putting forth something that a) feels like it has had significant thought put into it but b) doesn't assume that thought means the design is perfect and the work is done. – Xiong Chiamiov Mar 13 '20 at 16:06

If you escalated an issue by adding the tag, but something caused it to be “solved” — maybe someone from the community could actually answer it and did so; maybe a bug was really an issue on the user’s side; etc. — please ping a CM in the Teachers’ Lounge explaining the situation.

My instinct for handling cases like this would be to simply remove the tag. Would that not be sufficient? Is the communication with your internal tracking system only one-directional, such that adding the tag will add the question to your tracker, but removing the tag will not remove the question without manual intervention? Is it really required to ping a CM over this?

To prevent overloading moderators with flags, please avoid going on a flagging spree.

Thank you for calling that out explicitly. That was my biggest concern with this system when it was first proposed to moderators in private channels. It remains my biggest concern, as a moderator of a site whose associated Meta is itself larger and more active than pretty much any other site in the network. The GIF is what we look like on a good day (note that the head stays generally above the water). I really don't want to be inundated with bazillions of flags on a giant backlog of ignored requests that someone wants a CM to follow up on. There are a lot of someones.

  • 5
    Removing the tag won’t change the post in our internal tracker so we won’t have an accurate count of questions that still need attention. – Catija Mar 13 '20 at 2:50
  • @Catija: Yep, I figured as much: the ping is more to tell the CMs "you can remove this from your queue" than anything to do with getting permission to remove the tag (or whatever). – V2Blast Mar 13 '20 at 3:26
  • 3
    I'm still concerned about that flagging spree. While it's possible to discourage 1 user of raising 20 flags a day, how do you discourage 100 users from raising 2 flags a day? Not much of a problem if those happen to be 100x the same 2 posts, but we all know that's not how it will go. – Mast Mar 13 '20 at 7:26
  • 2
    The possibility of overloading moderators is frankly also a concern for me: CMs will be in the SO mod chat room, so we can keep in constant contact and assist y'all if the water starts to go dangerously above the neck. Let's keep communication lines open, and try to adjust as the circumstances demand :) – JNat Mar 13 '20 at 11:25

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .