Join us for the second MSE Town Hall on Wednesday June 22nd

Question

Last month we got together with many of you in the Tavern for our first Meta Stack Exchange Town Hall. Like we shared in my original announcement, amazing as meta is, we need to do more to keep lines of communication open with the users who work alongside us making our software great in the longer haul to shipping. So, we're doing a three-event trial run to see if this fits the bill, and this marks the second in the series. During last month's chat event things got hectic and disorganized...yet it was still some of the best time we've spent with you all in a while, and we're excited to do it again.

When and where is the next MSE Town Hall?

We'll be meeting in the Tavern for an hour on Wednesday, June 22nd, at 1pm EDT/17:00 UTC. Here's a handy tool for figuring out what time that is for you, wherever you're based.

What's the topic?

Let's discuss chat moderation, as suggested by rene.

Will the format be the same as last time?

We're making a few tweaks in hopes of improving the signal to noise ratio.

At last month's town hall, we all clamored to speak and it was a little hard to dig into the meat of the subject at hand. This time, you already know the topic will be chat moderation, so we have the chance to get some thoughts out in the open prior to us all tumbling into the Tavern. Getting an early look and reading on your perspective beforehand will help us make the most of the conversation. Feel free to use the answers below to flesh out any thoughts you have on chat moderation. For example, you could share some thoughts on:

• What disruptions are difficult to handle in chat currently?
• How do Room Owners make sure rooms stays within the community limits?
• How can chat be used effectively for a specific goal/purpose on the main site, while minimizing friction?

Weigh in via answers below, and we'll be grabbing a handful and using them to focus the conversation during the town hall.

Can I have a chat transcript for last month's MSE Town Hall?

You bet.

See you in the Tavern next Wednesday!

---

Update: Thanks to all of you for the insightful discussion earlier this week. You can find the chat transcript here (Part 1) and here (Part 2). Feel free to post a few words on how you think it went in the comments. Otherwise, keep an eye out for info in the coming weeks about the third installment in our test run of these events!

Answer 1

Do we have to have three separate chat servers?

Specifically thinking about chat.SO here. Main meta is so different from the rest of the network that it makes sense for it to have its own chat system, where the only 'moderators' are SE employees and where users who are suspended in chat can come to air their grievances (until they get suspended there as well), just like the meta main site.

But SO, different as it is from the rest of the network, doesn't really need its own chat server. Its having one makes moderation more difficult. The only mods in SO chat are the 20-odd SO moderators, whereas in every other chatroom all 500-odd moderators have mod powers.¹ I've never ventured there myself, but I've heard tell that some of the rooms on chat.SO have language to make the roughest soldier cringe and flags to put the most hardened Bridge or Mos Eisley veteran to shame. Those overworked SO mods could do with some moderation help over there, and the chat.SO network could do with being unified into chat.SE and held to similar standards.

The answer is probably that this would be too difficult to implement, but meh - I'd like to get the idea out there at least, see how people vote on it and why.

¹ _{Whereas the SO mods do get moderation powers on chat.SE. It's not fair! :-P}

Answer 2

How to deal with chat room "culture" that conflicts with the "be nice" policy?

I've been involved with at least two chat room explosions that involved (supposed) "outsiders" of a chat room flagging chat messages that contained content they found offensive.

Depending on the severity of the statement, this tends to bring lots of mods in along with lots of high-rep users who saw the flags and inevitably leads to a long conversation/argument/complaint session full of:

• You took it out of context
• it was a joke
• no one actually believes that
• we say stuff like that in here all the time and no one gets offended
• You don't understand because you're not one of us
• The people flagging are just trolls
• If people don't like it they can just leave

Personally, I don't think that "culture" should be allowed to invalidate the "be nice" policy but that doesn't seem to be the general consensus... particularly for the people who feel "wronged" or their supporters.

Where, as a mod or room owner or even just a user of one of the chat spaces should you feel the line should be drawn?

---

Note, I am not talking about people who intentionally go searching for content to flag just so they can get someone "in trouble" by flagging them months later.

Answer 3

Chat is real-time, and that by itself makes moderating it hard.

I'll be honest; when I see that "flags pending" indicator show up, I tend to ignore it unless the number is growing rapidly. Why? Because of the real-time nature -- it's probably in a room I don't know, on a subject I don't know, with people I don't know, and by the time I can figure out enough context to act, the damage has been done already. Besides, I know that every moderator currently in chat (up to about 500 people) is seeing it at the same time I am, so it doesn't feel critical for me to go handle it. (Tragedy of the commons, you say?)

It would be nice if we could come up with some way of directing chat flags to the people most likely to be equipped to handle them, while still maintaining checks and balances against certain "room culture" pathologies. There's been talk of showing flags to room owners first and only everybody else after a few minutes, but maybe there are other approaches.

Some ideas for exploration, off the top of my head:

• Maybe indicate somehow if a flag is from one of "my" rooms, where "my" means room owner or a room I frequent?

• Prioritize flags from people with a good flagging record? Indicate that flags are coming from people with a good record? Specifically in that room or on that site, maybe?

• Somehow alert a site's mods (and room owners?) if flag rate in one of their rooms spikes? I might not usually sit in my well-behaved room that gets maybe three flags a month, but if it's gotten ten today mods should probably know that. Ideally this is a real-time ping that isn't just buried in the "flags pending" bubble.

• Can we do anything to provide better context in the case where people are flagging each others' messages in the midst of an argument, or a user is hostile to another user and spite-flags that person? This is a hard problem. Maybe, to start, flags from people who also have pending flags against themselves could be presented together somehow? Instead of treating flags in isolation, can we present related ones together to get the bigger picture?

• Handle flags on older messages differently from flags on active chats? We don't necessarily want to prevent flags on hours-old content, but anything not in the current room "buffer" (however many messages that is) can probably wait a bit longer.

Answer 4

Should flaggers be able to (or required to) provide a reason for their flags?

When flags do come up, it's fairly common for me to look at the message and have absolutely no idea what's wrong with it. In these cases, unless there's an argument under way in the chatroom it came from, there's generally no way to find out what motivated the flagger, much less do anything to solve it or prevent it from happening again. I don't like assuming these flags are "frivolous" or "abusive", but there's simply no alternative in the current system.

One possible implementation would be to provide a freeform textbox with each flag, just like custom flags on the main site do. I have no idea if this would help, because I have no idea if any flaggers would use it...precisely because I have no idea what motivates "most" flaggers to flag. So it might be worth trying solely for the purpose of getting some hard data.

Another implementation would be a list of flag reasons. My personal opinion is that we could dramatically improve the practical effectiveness of the flagging system just by distinguishing between these two reasons:

• This message is rude or offensive, and should be deleted.
• This user is being rude, hostile or argumentative, and a moderator is needed to de-escalate the situation.

The current way flags are handled where 10k users click on valid/invalid buttons for the message without any context makes some degree of sense for the first reason, where only that one chat message is at fault. It doesn't make any sense at all for the second reason, which I feel is one of the most common legitimate flag reasons (and arguably the most important one to handle well). If this ever did get implemented, I imagine the second reason would do something completely different such as notifying a bunch of actual moderators/CMs rather than every random 10k user. And perhaps the first reason would be far less likely to get you chatbanned than the second.

In my ideal fantasy world we'd do both of these, so that when you flag someone for saying they ate a sandwich you can pick "This message..." rather than "This user..." and type something about why this particular kind of sandwich is considered offensive in your culture. Seriously, I would love to know.

Answer 5

What about foreign-language rooms? Specifically thinking about chat.SO here.

My opinion is that they should just be shunted over to chat.SE, and disallowed completely on c.SO - but it'd be interesting to hear other folks' opinions (and staff opinions) about the subject.

Should we allow these rooms at all outside of sites in that language?

Answer 6

Should flagged chat messages be reviewed in isolation or in context?

Personally, I almost never click the "valid" button on a flagged message because whether or not something is offensive, in my opinion, usually depends very heavily on the rest of the conversation. So my response is always to go to the other chatroom and see if there really is a situation getting out of control or if it's just a completely frivolous flag.

I have no idea whether this is the majority or the minority viewpoint on chat flag review, but more importantly, I suspect this lack of context encourages people like me to flood into the flagged chatroom, and that flood alone sometimes creates its own meta-drama, leading into many of the more obvious problems chat moderation currently has.

Answer 7

When can the flagger, as opposed to the flaggee, be considered the problem?

This is always an issue to consider when matters arise which concern one person offending another.

On the one hand:

• encouraging the attitude that flaggers are the problem will lead to some people feeling freer to post offensive content and others feeling afraid to speak up when they're offended
• the flagging system is there for a reason, and discouraging people from using it rather defeats the purpose, whatever that purpose may be.

On the other hand:

• we can't have a policy that the flagger is always right - this would be abused by trolls daily, and people would be getting suspended for nothing left and right
• flags cause drama - issues that could be resolved peacefully by people simply talking to each other become escalated affairs involving heightened emotions when flags are raised.

At first glance, it seems that the issue comes down to "where should we draw the line between content that could reasonably cause offence and content which isn't really flaggable except by trolls?" And indeed, that question is what the 'peer review' system for chat flags exists to answer: when you get flagged, a jury of six 10k+ chat users can decide whether or not the flag is valid (unless a mod gets there first).

My own solution is slightly different from this ... come along on Wednesday to find out more! ;-)

Answer 8

Are we showing chat flags to the right people?

• Why can someone with zero chat experience handle flags?
• Should there be more restrictions on who flags are shown to?
  • Should these restrictions demonstrate knowledge of how the chat system works?

---

• Maybe require some number of chat messages (1000? More?) in addition to Q&A reputation?
  • Room owners should be able to see all flags from the room they own regardless of other qualifications.

Answer 9

What is offensive?

Or: we need better guidance on what's considered offensive.

• This inevitably devolves to 'room culture', i.e. what the regulars think is acceptable - they won't flag the rest. Then someone new joins, with different perceptions of offensive, and starts flagging away. That never ends well.
• The majority of Be Nice obviously deals with how we interact with other users. This is good - as far as I know, there are very few if any 'room cultures' that promote or allow rudeness directed at other users.

• We end up on the topic of vulgar language. Be Nice suggests avoiding vulgar terms.

  • Most 'borderline' flags fall into this category. Most rooms I've seen have a fairly relaxed attitude towards a swear word now and then, but once a flag is raised it typically ends in deletion (of an otherwise-useful message!), suspension and a very annoyed user (or even entire room).

  • Should this be somewhat relaxed for chat? (My opinion in comments.)

    • This has come up many, many times. General consensus seems to be "some vulgar language is alright, but don't overdo it", but "it depends if someone feels like flagging it and enough [~6, out of how many?] others agree", leaving us where we are now. Continue doing this, or change? See (desc by date):

      • Toward a philosophy of Chat
      • The "Be Nice Policy" - and Chat
      • What is our policy on strong language in chat used in good humor, and flags on said language?
      • Are expletives allowed in SE chat?
  • If the rule is relaxed¹, should individual rooms be allowed to choose whether they want to allow a bit of vulgar language (in a non-abusive fashion)? How can that be handled with the global flag system?²
    • Currently, many rooms will implicitly allow some, while other rooms would probably prefer to keep it completely clean. A global "always allowed" or "no way" policy will probably make a fairly large group unhappy either way.

---

¹ I'm trying to avoid the room culture vs be nice issue here - if the decision is to not relax the rule, then of course rooms should not be able to choose differently.

² Or, preferably, without the flag system?

Answer 10

What disruptions are difficult to handle in chat currently?

As a moderator, not much. If something's disrupting chat, and it won't stop if I ask it to, I can suspend it. (There's one exception - keep reading.) So perhaps this question should be what disruptions are difficult to handle for regular users/room owners?

As far as I can see, the the point where a moderator gets called in is (broadly) one of two situations:

• offensive/abusive ('regular') flags that a mod just happens to handle
• moderator custom flags cast by users or owners when things are spiralling beyond their control

Those, then, are the situations that have regular users or owners tied up in knots, and those are the situations we should focus on. In particular, some questions to think about:

• Why are there so many more invalid offensive flags in chat than on main sites? Is more guidance about when to flag required?
• What can be done to reduce the number of times a room owner has to call in a moderator because they feel powerless to control the room?

I'm a big fan of letting rooms moderate themselves as far as possible - it's better for the room than having a possibly 'foreign' moderator turn up and be stricter on the situation than is totally necessary. But for self-moderation to happen, users and owners need to feel that they can control the conversation, and if necessary, room owners need to feel that their powers support them.

In an ideal world, we'd also have it so that room owners feel that they will be supported by the moderators, if they have to call one in. From what I can tell now, for a room owner, calling in a moderator is a bit of an unknown quantity - I'd like to see how we can make sure that if a room owner is doing the right thing, they know they'll be supported by the mods.

[The one thing that I find hard to deal with, as a moderator, is foreign-language flags or chatrooms. Without knowing the language, I just can't moderate - it's either Google Translate, or wait for a mod who does know the language. Neither of those are brilliant options, but this situation isn't terrible left as-is.]

How do Room Owners make sure rooms stays within the community limits?

This is the crux of it. If room owners can be encouraged to moderate effectively, then we have an ideal situation where moderators only need to be called in very rarely.

Perhaps controversially, I'd like to point to Mos Eisley as an awesome example of room self-moderation. This is the room that, late last year, was frozen for a week by the community team because the attitudes in it were toxic and it was almost un-moderatable without being an 'insider'.

Shortly after that incident, impromptu elections for room owners were held on meta, two new (non-mod) room owners were installed, and we've almost not heard a peep from there since. I think the key elements here were:

• RO's came from the chat regular community
• The chat community trusted the RO's (they were elected, after a fashion)
• The RO's actively moderate conversation

The moderation the owners do isn't capital-M Moderation - it's more like catching potential issues early, steering away from potentially dangerous topics, and gently guiding chatters on the right path.

How can chat be used effectively for a specific goal/purpose on the main site, while minimizing friction?

Let's be clear: chat has become something it was never intended to be. It was intended, as far as I can tell, as a venue for less structured site discussions, with some socialising alongside.

What it's become is more like socialising, with occasional site discussion.

That's not a bad thing, but it's important to recognise that we can't moderate it for what it was intended to be; we have to moderate it as it is. It's a problem child, but in the same sort of way that dumping your child on the street if you don't like it is considered bad, we can't just give it up as a lost cause.

I think the major cause of friction in chat is differing opinions and differing styles of moderation. Another answer here makes a good point about what's offensive?; that's a big friction point sometimes. However, I honestly think the best way to bring friction down in chat would be to standardise moderation strategies as far as possible.

This difference in moderation style is, I think, partially why moderation gets a lot of resistance in chat. This is Stack Exchange, we're all used to even, consistent moderation - but in chat, that changes. It's perhaps not totally the fault of the 'great unwashed', but also potentially quite a bit the fault of the moderators as a collective (not individually, I'm sure we all do our best to moderate as an individual).

I recognise that doing that isn't totally possible, but we can do better than what we have right now, which is each moderator moderates how they think is best. We should tell moderators what's best (or at least give some good guidance - more than is currently available) to avoid massive conflicts of style.

I hate to blast my own horn, but a number of moderators have noted that I seem to be good at chat moderation - I get on well with users, while keeping my ability to moderate efficiently. Now while standardising policy on what I, alone do is a terrible idea, the same is true of a number of moderators (a few names spring to mind). If we used the expertise of the moderators who are widely recognised as being good at chat moderation to create a set of strong guidelines on how to handle various things that crop up, then I think we'd go a long way to standardising moderation style and reducing friction.

---

That was a fair ole' essay. Now to find a way to compress it for chat...

Answer 11

I would like a warning mentioning the automatic suspension, should the flag be validated, on the flag tooltip.

I think that middle ground lies in the perception of the flag system. It isn't apparent what the flags actually do, and having an automatic punishment tied to them results in the "someone's upset but won't talk about it, and someone else will have to pick up the pieces" situations we've seen in the SFF chat room(s). Add the fact that we have conflicting use cases for the flag on the forum and on the popup message, and you're left with the perception that they're harmless and to be used liberally, when in fact they have very real consequences -- a short suspension seems like nothing until it snowballs into something else.

Answer 12

How do we deal with new people coming into a developed chatroom with a culture?

Similar to Catija's answer, new users can often join a chatroom and become confused due to the culture that has been developed there. This can cause many problems for the new users and those who have been around for a while. Some problems could be:

• Confusion between what is acceptable

• New users posting messages that don't "fit" the culture

• Familiar users posting messages based on the culture of the room, confusing the newer users

• Familiar users getting frustrated on the new users' inability to comprehend the new culture

• Or vice versa, where new users are being frustrated by the culture of the room

• etc.

This seems kind of like what to do with newcomers, treat them nicely or the hard way and stuff like that but this is a big issue. Lots of times new users get their chat privileges, join a chatroom, and become totally surprised about how everything works in that room. What gets even more frustrating is adapting to each and every developed chatroom. So how exactly to we treat and deal with new users that join a developed chatroom with a culture they're unfamiliar with?

Answer 13

Is flag handling by 10k+ users part of the “room culture” solution or part of the problem?

The only aspect of “room culture” that I see as a problem is the part that reinforces and promotes behaviours that SE as a whole is currently trying to eliminate from chat. There is a momentum to resist any reforms around acceptable behaviour and how chat is moderated. Unfortunately, given community self-moderation principles, this introduces a rift and conflict into the very moderation we're attempting to use to enact these reforms.

I've started moderating chat more in the past few months, and something I see over and over again is users reacting to a message flag as if it's a weapon being used in conflicts between room regulars, or a weapon in an attack from “outsiders”. This starts with a 10k+ user objecting to the flag that they can see (sometimes even attempting to shame the unknown flagger into owning up to it), and other users jumping on the bandwagon of flag-shaming. The idea that flagging is often invalid seems to be, if not a majority view, a very loud minority view that is disproportionately represented in actual messages posted, and hence in room cultures across many rooms. I've also seen mods' handling of flags deemed invalid merely because the room's high-rep regulars would have handled it the opposite way if that darn mod hadn't dropped in to interfere, which leads to the paradoxical assertion (and room culture belief) that mods should not be moderating chat.

Normally I'd say that distributing flag-handling to high-rep users is a good way to balance the workload. But currently the network is trying to turn around the momentum that chat has as the place where Be Nice doesn't apply, because “room culture.” During a time like this, the chat body-politic seems to contain more user who are vocally opposed to reforms than users who vocally support them. And these users have flag-handling powers.

Moderating chat often feels like telling the tide to stop coming in.

In this period of transition, are the very users whose behaviours we're hoping to redirect into more productive cultural customs the people who should be handling flags?

Is a temporary moratorium on 10k+ flag handling a good idea? Is there some other adjustment to chat flagging possible that would give staff and moderators the ability to better promote acceptable behaviour/content policies? The ability to review recent flags is a great new tool to have, but it's still limited: reviewing the recent flags to find badly-handled flags (and then do what?) seems like shutting the barn door after the horses have already left.

As a point of data, we have 500+ chat moderators and the record of recently flagged messages seems to show that we're getting fewer than a half-dozen a day on average, with the rare peak around 10 a day. It would seem that we have few enough flags that it would be feasible for only moderators to handle chat flags at the moment. Average flag handling time currently appears to be a matter of fractions of a second, and I don't think slowing that down by having fewer handlers will hurt, and may improve flag handling.