Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 153 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

In response to waffles's request for feature enhancements to the moderator dashboard coloring system:

The context

We have a new moderator dashboard that displays a variety of statistics about moderator activity: number of closures, comments, votes, deletions, and so on. In an apparent effort to call attention to the inequality of moderator activity in any one of those areas, the moderator with the lowest activity for a given metric is colored red, and the moderator with the second lowest activity is colored orange.

The problem

The coloring logic currently doesn't give the benefit of the doubt to moderators, and errs on the side of coloring a statistic. For example:

  • When all moderators are tied for a statistic (like at the start of a new day when everyone is at zero), everyone is colored red.
  • As a corollary to this, shame coloring is immediate, even when it would be unreasonable to presume a moderator should've had activity (i.e. when the day resets)
  • If there are four moderators, and two of them are tied for second place, the two second place people are colored orange.
  • If a moderator lags behind the pack by even one point (one closure, one vote, one comment, etc.), they get colored.

The proposal

Make the coloring less sensitive: give moderators the benefit of the doubt when determining if they need to be shamed. Possible alternatives:

  • Ties are good, not bad. If two or more moderators are tied for first or second place, they should not be colored.
  • Ties shouldn't be exact. If I'm only 10-20% behind the other moderators, my statistic shouldn't get shame coloring.
  • Give time for moderators to react. Shame coloring shouldn't be on the daily dashboard, at all. I would hope nobody expects moderating to be a 24/7 job.
share|improve this question
@Anna I don't want to remove it, I want to make it less of a nuisance. – user149432 Jun 26 '11 at 4:15
@Mark Tim's proposal is to shift the focus of the metrics and make it less of a nuisance that way. I figured it was close enough for a vote. – Adam Lear Jun 26 '11 at 4:18
@Anna @Mark Trapp I don't think this is an exact duplicate of my proposal. It states a similar problem, but different solutions and that might be helpful in the interim. As @waffles said, nothing is carved in stone. We can close / merge / faq it all once a decision has been made. – Tim Post Jun 26 '11 at 4:19
@Tim Fair enough. :) – Adam Lear Jun 26 '11 at 4:27
How else do we know we're losing? – random Jun 26 '11 at 14:52
up vote 27 down vote accepted

Before I start, there is one bit of highlighting logic I am not changing.

Moderators who have not been seen for a week on the site, get the "hot" coloring. Moderators who have not been seen for a month get the "supernova" coloring. This allows you to quickly pick up on who is around and who is not.

My initial design had quite a few flaws. The new design is as following:

Stat coloring will only appear on the monthly filter.

It is unfair to highlight users where we do not have enough data collected to even make a call. Nor is it fair to highlight users that have no chance of ever "catching up" to the old established mods.

If a stat has less than 30, it is excluded from highlighting.

In small SEs often mods only need to delete 5 comments a month. In these cases the load is handled just fine without needing to call anyone out.

Posts created and Mod messages are excluded from coloring

(unchanged) There is no reason to highlight activities that are too rare like banning and mod messages. There is no reason to highlight the fact someone posted lots of questions OR answers on the site as it does not relate directly to mod activity.

Stats that are 50% or less of the median value for that column get the "hot" coloring

Stats that are 25% or less of the median value for that column get the "supernova" coloring.

Moderation on SE sites is a volunteer job. We expect that sometimes people are too busy to perform the role. People are on holidays, busy at work, and so on. That said, in a hypothetical world, it is unfair for one mod to take on the whole burden of moderation and for the rest of the moderators to slack off, or even worse be too busy racking up rep.

The concept around the dashboard is to allow both SE staff and moderators to see the gaps in moderation and work together to create a team of moderators who spread the load more-or-less evenly.

share|improve this answer
Thanks @waffles, that sounds like a big improvement. – C. Ross Jun 27 '11 at 12:25
I would look at using something like the top and bottom quartile (or more divisions for large sites) as the distribution is unlikely to be normal. (A good mod is not bottom quartile in lots of measures for lots of months.) – maybe ask the people on the stats site about this… – Ian Ringrose Jun 29 '11 at 12:22

Concur. I also don't see why the statistics used are absolute.

If mod1 left 30 comments, mod2 left 32 comments, and mod3 left 28 comments, do we really want to shame mod3? I think that's silly.

I think a more useful measure would be to highlight moderator activities that are some percentage off the mean. I'm going to suggest 20% as a first guess. In the above example no one would be highlighted.

In another example you have mod1 with 300 comments, mod2 with 20 comments, and mod3 with 15 comments. Currently only mod3 would get "shamed", when really both mod2 and mod3 deserve it.

FWIW I agree that any shaming on a daily scale is not helpful or useful.

share|improve this answer

You must log in to answer this question.