Code of Conduct: enforcement and sanctioning

Question

The new Code of Conduct outlines and clarifies a number of our rules. But it also includes a section on how violators of those rules will be punished. I don't want to discuss particulars of the CoC itself. I want to investigate the structure of sanctioning for such rules.

These rules apply to everyone. That's their purpose; to outline how we should and should not interact with other people. No one should be exempt or immune, and all should be bound by these rules equally.

But sanctions for rule violations inherently matter to established users more than newer ones.

There are several reasons for this. Newer users are less engaged on the site than frequent users. As such, they will simply have fewer chances to break the rules. A one-and-done account may only make 5 comments, and even then only on their single question. A sub-50 rep account is limited solely to comments on their questions.

By contrast, established users see many questions. They seek them out. They comment on them, as a part of trying to keep the site clean, to rescue bad questions from the scrap heap. Per unit time on the site, an established user will make more comments, and therefore have more chances to fall afoul of the rules.

Additionally, regulars and veterans spend a significant amount of time on the site. So not only will they have more comments per-unit-time than a newer user, they will have a longer time to make those comments.

Coupled with that is the fact that regulars and veterans are tasked with keeping the sites clean and functional. Whether through review queues or just from reading the site, these diligent people will see the absolute worst the site has to offer. While this is no excuse for being rude, it's hardly surprising that, after dupe-closing the same question for the tenth time, one's patience has worn rather thin.

But the big reason why rule violations matter more to established users than newer ones is because the accounts of regular users matter more to them than the accounts of new users matter to those users. If a new user gets banned for saying any of these things, meh, no big deal. Even if they don't just create a new account (which may or may not go undiscovered), not being on SO does not matter to them. They don't care about the ban status of their account that much. They may complain about it on Twitter or some random blog post, but their interaction with their SO account is over.

By contrast, regular users are heavily invested in their accounts. We attach reputation and so forth to it. We associate your questions and answers to it. To abandon your own account would be painful for a lot of users.

The ability to comment or post is really important to such people. Take that away, even temporarily, and you impact them far more than you impact newer users. Give a new user a 3-day ban, and they probably won't come back. Give an established user a 3-day ban, and odds are good they probably will.

Not only that, there's the whack-a-mole factor. Every day, hundreds of new users show up. A non-trivial number of them will have a bad attitude and quickly run afoul of our CoC. And they will be sanctioned. But how you sanction them is irrelevant, since most of them won't be coming back. Even if you perma-ban them for a single infraction, just as many such users will show up tomorrow. And the next day. No matter what sanctions you impose on an individual bad-acting new user, there will be another to replace him tomorrow.

So the number of rude comments from new users will not be significantly addressed by sanctioning them.

By contrast, regular users of SO are a finite resource. Ban one, and there won't be a replacement. Or at least, not for quite some time; it takes a while to create a quality user of the site.

Now, I'm not saying that established users should be exempt from the CoC, or that they should be given leniency. What I'm saying is that whatever sanctioning procedures exist need to take the above factors into account. Not merely that the punishment fit the crime, but that there should be some recognition for how many good interactions such a user had before their bad one. New users have few interactions either way. Established users have many.

That is, a blanket policy of instituting a ban for making X number of CoC-violating comments would not be reasonable. Such a policy doesn't take into account the difference between a user with only 10 comments and a user with 1000.

The concern I have is that a sanctioning policy will be either mechanical (X number of infractions = Y punishment) or too subjective (always chosen by community moderators, who naturally have their own biases and blind spots). The mechanical one will catch far more of our established users while doing little to nothing about bad behavior from new ones. And while bringing CMs in on the process can certainly ensure that punishments fit the crime, that opens up an entirely new issue: how much you trust your CMs.

It's important that the CoC is genuinely enforced. That "slipping up" is not OK and won't just be brushed aside (thus dealing with people who "slip up" much more than acceptable). But it's just as important to realize that "slipping up" is a lot easier when you make 3,000 comments every year or so than when you only make 3.

How exactly do we create that balance going forward?

Answer 1

Moderator response is going to vary by moderator and moderation team so take what I talk about below with that in mind. This outlines some of the options that moderators have and how I've personally responded to situations in the past.

---

You're completely correct that it's more likely for someone who uses the sites constantly to get frustrated/upset/annoyed... due to the sheer volume of cruft some sites attract and dealing with it day after day. Everyone gets frustrated from time to time and that's something that we all struggle with and have to control.

These sorts of one-off outbursts (regardless of who makes them) are pretty unlikely to have any sort of repercussions other than the comment being deleted. The new Code of Conduct doesn't necessarily make that clear. In many cases, there's a step before the "warning" phase that's just "removal". It's important to note, though, that the moderator response to a one-off outburst depends on a few things:

1. The severity. - How rude is the comment? Something snide or snarky will probably just get deleted. Something laden with profanity or other offensive commentary will be more likely to get an actual mod message warning or an annotation.
2. The flagging. - Is it flagged? How much?
   • Comments will generally be flagged as either "rude" or "no longer needed". Three (non-custom) flags will delete a non-upvoted comment without moderator involvement and comments with certain keywords require only one flag. When either of these happens, the moderators may not even be aware of the comment/s unless they review deleted comments on a post flagged for some other reason. Custom flags don't come with this concern but the flags may be handled if the comment gets sufficient other flags.
   • If the comment doesn't get flagged at all, the moderators aren't going to be notified and the comment will stick around. This is why flagging is such an important part of the process and why making users aware of their ability to flag comments and allowing low-repuation users to flag comments on their own posts is important.
3. The frequency. - Is this really a one-off situation? The way the system mitigates the fact that comments can be removed without moderators ever knowing about them is that users who have three or more comments deleted through rude flagging within seven days will trigger an auto flag for moderators.

So, there are both subjective and objective (system-implemented) controls in place to guide moderator teams in deciding how to react to content that breaks the Code of Conduct. In addition, every user has a moderator-only comment history page that allows moderators to view either all of their comments, all deleted comments, or all flagged comments (though it's not possible to view by flag type - which I would really appreciate).

For users who do pick up a three rude comment auto-flag, this is usually the first place I look to see their comment history and decide what might be the best avenue for addressing the behavior. There are several options in addition to deletion, mod messaging, or suspension:

• If there's a lot of vitriol being passed back and forth, it may make sense to start by deleting anything problematic and leaving a comment on the post encouraging them to take a break or focus on the post rather than arguing or being rude.
• In cases where it's one-sided but seems like unusual behavior, the best course of action may be to annotate their account to note the incident but otherwise do nothing.
• In (rare) cases where it may be beneficial, starting a dialogue with that user through a private chat room about their behavior may be an option.
• Moderator teams may talk to each other (within a site or outside it) to see if any have specific experience with the user and talk through what should be done, if anything.

These are all options in the moderator tool kit - and there are probably others that I'm forgetting at the moment.

In addition to all of this, higher-volume sites are also more likely to be distributing the workload of flags across several different moderators... as many as 25 on Stack Overflow... and there are many, many more users to keep track of (or not). This means that on the bigger sites, it's less likely that moderators will be able to even remember that a specific user (high-reputation or otherwise) has a history of bad behavior outside of using what tools are available and polling fellow mods' memories.

---

When have I moved to actually message/suspend users?

Pretty infrequently, actually. Admittedly, I probably should have taken that step sooner in some cases but I think that's a common ... for this purpose, let's call it a "failing"... in many moderators. We generally want to give people the benefit of the doubt, give them more chances than they may deserve. Often times, deletion is the best solution for all.

Let's say a new user gets a snarky response to their question. If I see that comment - either through a flag or in my normal perusal of the site - I'm probably just going to delete the comment unless there's a gem in it worth polishing. Moderators are generally discouraged from editing comments but, as with the items above, sometimes it's the better choice.

If I have the time, I may poke around that user's comment history to see if there's anything concerning. If there is, I may annotate or say something in the site's moderator room to see if it's worth escalating to a moderator message. If they already have an annotation or a long history of problematic behavior (but no warning), I'll send out a moderator message warning them to watch their behavior. If they do have a warning already for the same behavior, it's time to suspend...

I hope this shows that, with very minor infractions, it takes a good amount of repeated ill-behavior to actually lose access to the site.

Now, when we're talking about more than snark - abusive, offensive content - the escalation gets a bit faster. Warnings happen sooner, after single offenses. Sometimes, even an immediate one-day suspension if it seems like the user is unable to disengage from the situation after a comment requesting them to do so.

---

Why is comment deletion so strong of a choice?

With the exception of the banner on a user's account profile when suspended, we generally address these sorts of things in private. Short of a suspension, a targeted user will likely never know what action was taken in response. Deleting the comment is our way of saying "we don't want this here". If the target never saw the comment, they've been saved seeing it; if they did see it, this sends a signal that the comment was judged and found wanting.

It's not perfect. I have personally been troubled by finding blatantly rude comments gone without knowing whether they were removed by the OP or a moderator. This is addressed to some degree by another auto flag - it's raised when a user frequently deletes their own comments - but the trigger for it is high enough that it's pretty rare to ever see it as a moderator.

In cases where the user flags their own post asking about deleted rude/snarky comments, I'd generally mark those flags helpful and offer an explanation for the removal and emphasize that the comments were unwelcoming and removed as such.

Answer 2

To borrow, or steal a phrase from my other answer...

A code of conduct is a shield, not a sword or bludgeoning stick

One of the nice things in being an SE moderator is that except in extreme cases, we're trusted to do what's needed.

A fundamental part of the current "be nice FAQ" is to assume good intent. And that also means well, we're fallible sacks of meat, water and electricity that sometimes don't quite do the perfectly sensible things.

If someone clearly slips up, whether it's in terms of their prejudices, sometimes throwing the book at them isn't the right answer. I've at least once told off a core member of my community about something he said that was insensitive and it worked out.

Ideally, the community watches out for each other. Folks go "hey dude, this is not cool". And the other guy goes "sorry! I messed up!"

And all's good in the world.

It's when people keep at that behavior that, well, we need to do something. We warn. We ask politely. We suspend you.... it takes a lot for folks to go "erm, we don't want you anymore".

It's not about punishment. It's not about kicking people in the rear every time there's a microaggression. It's not about any sort of political or ideological purity.

I think there are a few flaws with your reasoning.

An established user should know better. An established user has spent time in SE and should understand what's expected of them. He should know there's no reason calling out someone on their race, gender identity, or inseam size - only their posts. In fact, he has tools at his disposal - downvotes and such to deal with low quality posts.

As a user who has been on SE a while... there's churn. Surprisingly people lose interest and wander off, then come back. On SU's reputation tables the user above me has not been seen in over half a decade. And... he wasn't irreplaceable. If you cannot attract new, engaged users, your community is diseased and is going to die slowly and painfully of attrition.

I'd happily suspend Jon Skeet if I needed to. It would be just a little awkward since he's a really nice guy by all accounts, and I cannot see him doing anything suspension worthy but really, you do not get a pass for being an established user. As an established user - you need to be setting an example.

In addition, in practice a CM will only really stick their nose in if asked (by anyone) and is needed. For the most part, your mods can deal with almost anything that comes their way.

I don't like the "new user" vs "experienced user" split. I was new once. I had someone give me a hand quite a few times. My spelling for one was terrible (still is a little ;) ). Folks helped me be the person I am now. As an experienced user, you can gently guide new users to the right way of doing things. You can edit, vote, and comment _{appropriately, suggesting improvements}. In a sense, you have the full suite tools to deal with these things in addition to experience in the SE way of doing things.

It's worth remembering September never ended, and everyone and everywhere has new users who aren't totally familiar with each community's norms. Your approach is essentially sticking our heads in the sand, or turning the community into a good old boys club.

So no, fundamentally speaking, the COC won't really be anything more than an attempt to formalise rules. I don't think enforcement will or should fundamentally change.