What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 122 Stack Exchange communities.

Stack Overflow, due to its size, has some unique problems. One of those is flags. For example, in the last 30 days as of the time I am writing this, there have been 26,710 flags. That is 890 flags per day on average.

One of the primary duties of a community moderator is to look at and process flags, as noted in the theory of moderation blog post:

Even with active community self-regulation, moderators occasionally need to intervene. Moderators are human exception handlers, there to deal with those (hopefully rare) exceptional conditions that should not normally happen, but when they do, they can bring your entire community to a screaming halt — if you don’t have human exception handling in place.

The most common moderator task is to follow up on flagged posts. Every post contains a small flag link, which anyone with 15 reputation can use.

Flags are good. Flags help us maintain Stack Overflow and improve it. But there are so many of them. If you do the math, and assume each flag takes about one minute to look at, load the page, and decide (probably an optimistic number), that is 890 minutes of work per day -- almost fifteen hours in total.

In addition to using every clever trick in the book we can think of to expedite and automate flag handling, we've continually been expanding the number of community moderators on Stack Overflow to help share the work of handling this tide of flags:

Note: This list only reflects the results of the moderator election at that time, they are not indicative of who is currently a moderator of the site

As of now we have sixteen community moderators, visible on the /users page of Stack Overflow.

Of course, serving as an elected community moderator is, and always will be, a completely voluntary at-will activity. But I believe the sheer scale of Stack Overflow now demands some additional policies specific to Stack Overflow community moderators.

We are not asking anyone to make special accommodations, but it is increasingly untenable for us to have elected moderators who spend substantial amounts of time on Stack Overflow, but do not participate in the primary duty of a moderator -- that is, helping us follow up on and resolve flags from Stack Overflow users. It's unfair to the other elected moderators, and it is unfair to the community.

Just as in open source, "many hands make light work", but this requires each of the moderators to be truly pitching in and contributing by resolving flags by clicking that bright orange number periodically throughout the day.

We are of course open to adding more community moderators. But that is avoiding the issue.

I believe we regrettably must have a new, specific policy for community moderators on Stack Overflow, due to its size and scale: I propose that on Stack Overflow, all elected community moderators must close a 'reasonable' number of flags while they are on the site. If they do not, they cannot continue to hold the position of Stack Overflow community moderator.

"Reasonable" is of course To Be Determined, and the point of this is not to demand specific numbers -- all we're looking for is a fair and equitable balance of flag handling among community moderators over a period of 2-3 months.

Comments? Opinions? Thoughts?

share|improve this question
6  
Wouldn't it make sense, instead, to increase ever so slightly the cost of flagging on SO, given that scarcity of flags currently isn't an issue? Of course, it depends on how many posts on the site only get 1 flag before they're handled on, and thus how many of those risk never getting moderator attention as a result of the change. –  badp Aug 9 '11 at 6:40
45  
almost all the flags are good and valid flags that need handling, and each one makes Stack Overflow a little better. The problem is not invalid or superfluous flags. –  Jeff Atwood Aug 9 '11 at 6:42
5  
This sounds reasonable. Do the moderators have anything to say? Are all flags similar in how much time they take to process? Are there any cases of moderators spending a very long period of time just to sort out one flag? –  Pëkka Aug 9 '11 at 7:04
3  
@waffles ... or discharge the soldiers that don't fight and hire new one's that do? –  DMA57361 Aug 9 '11 at 7:35
5  
@DMA which is the exact problem ... 1) do you discharge them? 2) what is a good soldier? –  waffles Aug 9 '11 at 7:36
1  
@waffles I was reading the If they do not, they cannot continue to hold the position of Stack Overflow community moderator. in the question above - which implies discharging is being considered. And I can only assume 2 will be determined by the "performance chart" on the history page, but I think that is the real point being asked here. –  DMA57361 Aug 9 '11 at 7:39
6  
@Will 2) is something only you moderators can really comment on. Is the number of processed flags a reliable metric for whether you're a good moderator? Are there cases of moderators processing very few flags, but still doing a lot of mod work? Those would have to speak up –  Pëkka Aug 9 '11 at 7:39
20  
@waffles: Us un-diamonded masses don't know the details of how many moderators are "never participating in fighting" (nor do I think we should), so it's pretty hard to comment on that aspect other than to shrug and give a general "That seems reasonable". Which makes it strange to me that this is a Meta post -- it seems like this would be better handled by emails among the devs and mods. –  Josh Caswell Aug 9 '11 at 7:50
3  
@waffles In your example there are infinite rations -- what finite resource are slacker mods consuming? –  Michael Mrozek Aug 9 '11 at 14:22
7  
What is the cost of none active moderator? (e.g. eat the rations, does not seem to relate to moderators) –  Ian Ringrose Aug 9 '11 at 15:22
6  
@Ian: During elections, there are a certain number of moderator posts available. These are then taken. If one of them is filled by somebody who then does not do any moderating, then something is wrong. Seems pretty obvious to me. –  Lightness Races in Orbit Aug 9 '11 at 18:40
8  
I feel this question deserves more time than I can give it, but my post would revolve around the thought "Moderation, rather than being an act of guiding the community, has largely become an act of cleaning the toilets." I'm suggesting we take a hard look at some of the flags the mods clear, and assign them to 40k users, which would immediately add another 100+ people to the flag handling queue: meta.stackexchange.com/questions/101772/need-new-30k-abilities –  Adam Davis Aug 10 '11 at 22:26
1  
@Jeff 1) True, assuming all 890 daily flags require that level of leadership. 2) The community might also be able to provide some of that leadership. –  Adam Davis Aug 10 '11 at 22:47
1  
@Adam we have spent the last three months exhausting every angle on that. Again, I am not at liberty to disclose the moderator activity data but it is not a subtle difference. We are talking 2 orders of magnitude PLUS. –  Jeff Atwood Aug 11 '11 at 1:12
6  
lol, I think all you need to do is ask a question like this any time mod activity starts slacking. The queue hasn't been over a hundred since you posted this. –  Won't Aug 11 '11 at 11:57
show 14 more comments

14 Answers

Automate and offload to the community.

High rep users can already vote to close, migrate or delete bad or off-topic questions. They can not do much for equally bad answers. Perhaps we could trust the system to delete answers identically flagged by a group of high rep and/or high flag weight users?

Similar to the close reasons for questions, we could have standard reasons like for example:

  1. the post just says "Thank you"
  2. the answer is really a follow up question
  3. the post just contains a link to the guys blog
  4. the post contains mostly four letter words
  5. the post is just unintelligible

and leave the more difficult cases to the moderators.

share|improve this answer
2  
Items 1,2,5 in your list can be flagged not an answer; 3 could be edited, spam or not an answer (depends on how blatant the advertising is); and 4 could be edited, offensive or not an answer (depending on if it can be cleaned up in to a real answer). Maybe the point is not an answer flags need automating? But apparently some users do try to use flags to say "I don't agree with this"... so caution is required. –  DMA57361 Aug 9 '11 at 7:30
2  
Yes, that was kind of my idea. If the flags were more precise than "not an answer" perhaps some of them could be automated and relieve the moderators to work on the harder cases. Also high flag weight might be part of this, trusting people who have a history of correct flags. –  Bo Persson Aug 9 '11 at 7:42
2  
I think, as per the answer that's been deleted and other comments, this answer misses the point that Jeff wishes to discuss - not about reducing the workload of mods, but making sure that all the mods are at least making a suitable effort to clean the site. Otherwise, would it be better to then replace the "less good" mod with another member of the community who will put more effort in? –  DMA57361 Aug 9 '11 at 7:45
8  
Well, if there is less work to do wouldn't the work actually done then be sufficient? –  Bo Persson Aug 9 '11 at 7:48
    
Probably, but even with a reduced workload is it acceptable to have the majority of moderation work done by a minority of moderators? –  DMA57361 Aug 9 '11 at 7:57
    
@DMA57361, yes, the less flags that have to be handled by the moderators, means more time that they can spend on Meta, or in SO answering questions. –  jzd Aug 9 '11 at 11:25
    
And what, @jzd, if again only that same minority of moderators are then on meta helping? What about the "other" moderators who aren't helping out anywhere? Less work won't solve that problem - sure, it's not a bad thing to reduce the workload, but it's not the point of this question. Incidentally, please do note that I'm a SU mod, not a SO mod, so my comments here RE minorities doing the work are conjecture based off of the implications in the question. –  DMA57361 Aug 9 '11 at 11:29
add comment

We can't simply keep adding moderators in order to compensate. This doesn't just side step the issue, it creates more work for the existing moderators as well. New moderators require training and help initially, which means more must be asked from the people who are willing to spend time focusing on moderating.

If you are able to spend time asking or answering three to five questions per day, I don't see why it would be unreasonable to ask that you also spend twenty minutes helping to keep the pressure in the flag queue down to a reasonable level. While none of us committed to a certain level of participation when accepting our agreement, I think that we all understood that:

  • Moderators should want the responsibility
  • The main part of a moderator's job is to attend to flags

I can't conceive of someone wanting the responsibility unless they visit and use the site on a regular basis with plans to continue doing so.

Things do happen in our lives and I don't think anyone should feel bad about not being able to spend much time in their capacity as a moderator due to personal issues. Additionally, some days are simply more exhausting than others. I think we should set some expectations, as long as they remain 'on the average'.

With that being said, I don't think that it's unreasonable to ask that each moderator:

  • Services an average of 60 - 100 flags per week. Note that it's extremely common to clear ten flags within a minute, depending on the nature of the flag. As said above, of course there will be periods where a moderator simply can't give that much time, but on average I don't think it would be difficult.

  • Participate here on meta. Answer a few questions each month asking for support, help the community manage the frequent duplicates, help with the (few) flags that we get here from time to time.

  • Participate in chat on occasion. A lot of ideas regarding moderation are discussed in the moderator room, like "what the heck do we do with these old and famous questions that aren't on topic any longer?"

I realize that some may view any help that someone can give as welcome, but I think that outlook sours when it becomes obvious that a slot that could otherwise be much more productive is being blocked.

While moderating, we come in contact with everything that the community objects to, which is frequently very smelly stuff. It really is not fair for some to spend the bulk of their time earning esteem, reputation and badges while the rest of us labor in the proverbial salt mines that double as an adult day care center.

It comes down to 'responsibility' in the term 'want the responsibility'. While it is a completely volunteer effort, I don't think it's unreasonable to imply a bit of obligation when mentioning the responsibility.

share|improve this answer
    
I agree that participation on meta and in chat should be encouraged, but we're not particularly lacking in those areas. I do hope that in future elections participation on meta will influence voting, but I think to address the issue at hand only a baseline of flags cleared should be established as an expectation. –  Bill the Lizard Aug 9 '11 at 13:03
    
@Bill - I was speaking in the over all sense. Still, per 1k flags serviced participation on meta is almost compulsory, so I guess the two would go hand in hand. I agree that setting some kind of bar for flags is the most important thing. –  Tim Post Aug 9 '11 at 13:22
4  
@BilltheLizard: Meta participation is not always thought of positively: twitter.com/#!/cky944/status/29549656941797376. Maybe it's the unicorns and silly stuff, or maybe it's the feeling that there's a clique of high-rep users (I've heard both reasons). –  mmyers Aug 9 '11 at 13:35
    
@mmyers: I hadn't considered that. We've reduced the amount of unicorns and silly stuff considerably of late, but I don't think there's any way around the perception that there's a clique of high-rep users here. I think that's a natural consequence of having a meta site. (Edit: Of course, I went to the front page and saw an "Eeek!" question posted by a cartoon unicorn right after saying that...) –  Bill the Lizard Aug 9 '11 at 13:45
2  
New moderators require training and help initially -- True, but Kev and Lasse stepped into the fray with a minimal amount of hand-holding. –  Robert Harvey Aug 9 '11 at 16:53
    
@Robert Harvey Yes, they did. I mentioned it because many (describing those who might be interested in moderating) are often intimidated by 300+ flags, especially if called into question on MSO during their first month or so. The trepidation surrounding making a single mistake can be and often is something that people need a bit of help with to overcome. –  Tim Post Aug 9 '11 at 17:02
1  
Completely agree about meta participation. I only vote for people that I have seen make level headed decisions on meta. –  John Aug 9 '11 at 17:43
    
You will always have the issue with those who delay until the end of their measurement time span to make their quota. On the one hand, they have lent a hand with the workload, but the other 6/7 days, everyone else had to make up for them. –  Steve Jobs Aug 9 '11 at 19:13
2  
While moderators are few and especially picked for dedication, it is a deadline none-the-less, and a weekly one. A lot can happen in a week: people get sick, have crises, have babies, loose their jobs, or have other deadlines. It seems like a larger time frame might be better to reflect if the moderator is doing good work. I know this directly contradicts my statement above. –  Steve Jobs Aug 9 '11 at 19:16
    
@rlb.usa What I posted was just a suggestion for the start of a guideline (please take the question into account), and I did note that things do happen. –  Tim Post Aug 9 '11 at 19:25
add comment

The logical first step would be to 'reach out to' the 'underperforming' moderator to find out if it is a temporary hiccup (personal life), if they have misunderstood their responsibilities, or if they've just run out of steam. Said discussion is best undertook by someone with a modicum of sensitivity...

If they have run out of steam, maybe asking the question will make them realise that they don't really want to be a moderator anymore anyway. Give them an "I resign" button and the problem is solved.

Now I'm guessing that you've already done that, so now the question is (my inference):

Can SE decide to 'fire' moderators, and on what basis?

My answers:

Yes, if 'due process' is respected (as outlined above), and on a 'reasonable' basis that can only be determined (in my opinion) by the moderators, in conjunction with SE.

share|improve this answer
5  
I agree that this is the logical first step. Priorities in people's lives change over time, and not everyone is able to volunteer several hours every week. I'm only guessing, but I imagine a few of the current mods would be willing to step down if it meant someone with more time to volunteer would take their place. –  Bill the Lizard Aug 9 '11 at 12:30
5  
@bill to be very specific, it is time spent on Stack Overflow while not resolving any flags that is the issue. If you are not on the site at all, this does not apply. Obviously you can't resolve flags if you have no internet access, etc. –  Jeff Atwood Aug 10 '11 at 21:56
add comment

I don't have access to moderator tools. So, it is entirely possible that these options are already available. May be this is a stupid idea but here are my two cents about this topic:

In my opinion, I feel that moderators are doing a very good job. Moderators shouldn't be punished for a volunteer job that they do out of passion. Having said that, I would concentrate on ways to help them to do their job better.

Here it goes...

  • Classify the general users into four categories based on the criteria defined below.

Buckets

  • Gray bucket indicates users
    • with less than 500 flag weight.
    • and also have less than 60% of their flags being deemed valid.
  • Red bucket indicates users
    • with less than 500 flag weight.
    • and also have a minimum of 60% of their flags being deemed valid.
  • Yellow bucket indicates users
    • with flag weight between 500 and 700.
    • and also have a minimum of 80% of their flags being deemed valid. Anything less than 80%, they fall under Red bucket.
  • Green bucket indicates users
    • with flag weight above 700.
    • and also have a minimum of 95% of their flags being deemed valid. Anything less than 95%, they fall under Yellow bucket.

If someone belongs to Green bucket, we can make a reasonable assumption that the flags will be more likely legitimate and similarly the flags from users in Gray bucket would need more attention. So, the level of attention required on flags from Gray, Red, Yellow and Green bucket would very high, high, medium and low respectively.

However, there could be some disambiguations that may arise when Green bucket users flag a post that might need moderator's intervention. I believe this is where Moderator's should spend their valuable time instead of concentrating on flags from Gray bucket users. Following point discusses how to handle that.

Have a pool of volunteers from Green bucket for flag moderation. If someone has reached the Green bucket, I believe that they know what they are doing when they flag posts. These volunteers are not elected moderators but are very active members who are doing an excellent job in helping SO to maintain the quality of content. I call them Flag Validators. They can validate the flags from Gray bucket users, which I believe could form quite a percentage of 890 or so flags every day.

Flag validators should be given tools to validate only flags from Gray bucket users. Their good work as flag Validators can be used as a criteria during future Moderators election.

Diamond moderators can occasionally monitor how these Flag validators are doing their job and intervene if there are any serious disputes.

Diamond moderators can concentrate on flags from the top three buckets that need higher interpretation. This way they can better utilize their time avoiding flags from users who don't understand the concept of flagging.

I hope that I am making some sense here.

Flag Moderation

share|improve this answer
4  
In addition, users with 10K rep should be automatically made into "Flag Validators" where they can vote on flags. Two votes makes a Flag Valid. –  staticx Aug 9 '11 at 17:17
    
@0A0D: I was trying to suggest that Diamond Moderators don't have to watch Flag Validators closely thereby relieving the mods from checking trivial flags. The dotted arrow indicates that mods will intervene only when there is a dispute from the action taken by Flag Validators. If mods also vote on Flag Validators, then that doesn't lessen the burden of moderators. –  user162697 Aug 9 '11 at 17:23
3  
@0A0D: Sorry, never mind. I read 10K reps as diamond moderators. I didn't go by the rep points because rep points don't necessarily correlate with flag weights. Hence the reason, I classified users by flag weight because the topic is about flagging. –  user162697 Aug 9 '11 at 17:25
    
I think part of this are already done: the number and flag weight of the flags on one post/comment determine the order in which diamond moderators get to them. –  Paŭlo Ebermann Aug 10 '11 at 16:32
add comment

I think it's entirely reasonable to expect a minimum amount of flags processed on a weekly or monthly basis - I'm opposed to a daily expectation, as my own personal day-to-day can change drastically permitting time on SO one day, and none the next.

I do wonder if there's potential for some misunderstanding regarding the involvement of a moderator as measured by flags. If I'm not mistaken, flags only measure a moderators reactive events, and not their proactive events. For instance, while working on a flag five minutes ago I noticed two other non-answers in the same question that needed to be dealt with, but weren't flagged. Thinking that I may not get credit for these items, I went ahead and flagged them first, then addressed them afterwards (perhaps that's what I should be doing, rather than dealing with them immediately?)

One other improvement that could be explored is sorting flags not only by the the flag-author, but sorting them also depending on which moderator is reviewing them. For instance, I'm a PHP/CSS/jQuery guy, not a C#/Java/Python guy. If I see flags claiming some esoteric-sounding Java answer is 'Not an answer,' I'm really not in any position to evaluate that answer (in some cases the answer will be obvious, but not in all). Perhaps some type of shifting of content around for moderators can be done according to the tags or categories we're most active in.

I may not have a solution, but I do see the problem (and see my part in the problem). Glad to see we can discuss this and work toward resolving the issue, and bettering the experience for others.

share|improve this answer
1  
I would want the same (sort by tags) for the 10k user moderator tools. Of course, without letting flags without any moderator with specific field knowledge drop away. –  Paŭlo Ebermann Aug 9 '11 at 15:30
2  
I don't like a "minimum amount of flags processed on a weekly or monthly basis" as in most of the world people often go on holiday for 2 or 3 weeks. –  Ian Ringrose Aug 9 '11 at 15:33
    
"short by tabs" using the same system as the home page would be good for the "surgested edit" lists as well. –  Ian Ringrose Aug 9 '11 at 15:34
5  
You shouldn't pre-flag your tertiary actions. Flags imply a degree of community consensus, which is absent if you are processing your own flags. –  Robert Harvey Aug 9 '11 at 16:59
8  
@Robert That was what I suspected as well; however, that exposes a fault in the system that moderators who are proactive may appear to be inactive to some degree. –  Jonathan Sampson Aug 9 '11 at 18:14
add comment

I'm going to take this opportunity to suggest something I have been mulling over, but have yet to drop in a question.

I feel I'm chumming the waters in the -1 tank and then jumping in, but here it goes...

What about giving moderators some rep for clearing flags?

Not a lot... Less than accepted edits for < 2000 rep users.... .1 per flag?

The reason I suggest this is that we (should be) spending time clearing flags rather than answering questions, so we lose out on the rep we could be getting during the time we can allot to SO.

There, I did it. Seems petty, but it might help motivate mods to clear out more flags.

share|improve this answer
3  
mmmm... chum. –  Jeff Atwood Aug 9 '11 at 10:29
12  
After reading the first two sentences I was afraid the next line would mention bitcoin. In all seriousness I suppose that a bigger carrot at the end of the stick might help, but I'd (personally) like my rep to come from answers that I provide .. or (what little I have) kind of loses its significance, at least for me. –  Tim Post Aug 9 '11 at 10:50
1  
@TimPost: Yeah, I completely understand. But, as an active moderator, we do have to split our time between handling flags and answering questions. It isn't exactly an equal tradeoff (assuming the answers we would normally be providing aren't worthless), but at least we do get some compensation for our work (other than the occasional nastygram or "why did my flag weight just go down?" questions here). –  Won't Aug 9 '11 at 11:06
    
I always thought the ability to click that "invalid" button was compensation enough for you. Has this changed? –  Cody Gray Aug 9 '11 at 12:13
7  
@CodyGray: Oh, that gets me through some hard times, but without actually seeing the users driven before me and without actually being able to hear the lamentations of their women and children it just isn't that satisfying. –  Won't Aug 9 '11 at 12:17
    
@Tim Post: Let's not cast this bitcoin idea aside without giving it the consideration it deserves. ;) –  Bill the Lizard Aug 9 '11 at 12:24
2  
@Bill My worry is that they'll consider it during the next podcast, and we'll have to make all new bingo cards. –  Tim Post Aug 9 '11 at 13:13
6  
Rep has very little value once someone has enough to be a mod on SO. –  Ian Ringrose Aug 9 '11 at 15:28
8  
@IanRingrose: But what about catching up to Jon Skeet??? –  Won't Aug 9 '11 at 18:52
add comment

I'm all for giving more guidance to moderators about what is expected of them: I think many people run for election on various platforms like "hands-off" or what-have-you and have no idea, if they become a moderator, the amount of work that's involved in being one.

But reading your post and talking to waffles in chat, it sounds like replacing—instead of supplementing—moderators who are below-average in their activity is at least in part about making sure the moderators who are pulling their weight aren't demoralized or feel their moderatorship is cheapened because they have the same status. That it's cause for the hard working moderators to say, "hey, if Joe Smith isn't doing his fair share, why should I?"

It's all a little too Ayn Rand for my taste. Who is John Galt?

I can understand that it can be demoralizing to take care of a ton of flags, then check the moderator dashboard to see Joe Smith didn't handle very many at all this month but got 2,000 rep answering questions. But I think the suggested cause for the demoralization wrong: it's not because my diamond is cheapened because a "non-moderator" moderator also has one, it's because Joe Smith got something for his time on the site while all I got was another 100 flags. The incentive structure is messed up.

One way to solve that is to increase the incentives for doing moderator activities: facetiously, it might be to give rep for handling a flag or something silly like that. But besides rewarding rep for an activity that doesn't promote the core function of the site (Q&A), it attracts the wrong kind of moderator.

And it's really the other side to the same coin: whether you remove moderators who aren't as good as the diligent ones—thus rewarding the diligent ones with a "more meaningful" diamond next to their name—or you give rewards for doing moderator activities, you make moderating more about getting a carrot than about doing one's duty.

That is, people volunteer to become a moderator because they love a particular site and they want to do their part to keep it running smoothly: they consider it to be their fulfillment of the social contract. You want people who are actively engaged in the main site, even when the flag list is overwhelming, because they are the type of people to take it seriously: they have skin in the game, and they want a better site because they use it themselves. The reward for them isn't a more meaningful diamond, it's a better site.

At the same time, I do agree it is untenable to have an ever-growing list of moderators to supplement the changing interests of what are ultimately volunteers. But instead of making it about meeting a quota to ensure the diamond next to one's name is a substantial reward, make it a referendum.

Every six months (or if you think it's necessary to evaluate every 2-3 months, do it every quarter), hold elections: if someone wants to keep being a moderator, they can run. If they can't be bothered, they can leave and no harm no foul. Make the moderator statistics on the dashboard visibile during the elections and let the community decide whether that person is pulling their weight.

share|improve this answer
8  
I don't think the correct answer is to massively increase the number of moderator elections, and force all existing moderators to re-run every time. That'd create a lot of stress for everyone involved. This is also utterly and completely irrelevant on every single other site except Stack Overflow. It's sort of like the difference between being the mayor of a small town and a member of the US Senate. –  Jeff Atwood Aug 9 '11 at 9:35
4  
@Jeff Senator or mayor, you run for the gig knowing what it entails and you work with what you're given. The president doesn't get to fire senators who don't pull their fair share: if a senator doesn't do his job, he doesn't get reelected. If a senator thinks the job sucks, he can resign. While I think 6 months is an eternity in the life of an internet community, I don't think being a moderator constantly in fear of potentially getting the axe for being at the bottom of the list is any less stressful. –  user149432 Aug 9 '11 at 9:54
7  
no need to fear anything if you're performing the basic functions of the position you were elected to. We had a moderator on math who got elected and never even showed up, for example. These are extreme cases; I am not at liberty to share the data but subtle, it ain't. –  Jeff Atwood Aug 9 '11 at 9:57
add comment

I don't know what a reasonable weekly (daily, monthly, etc.) workload is for a Stack Overflow moderator, but I don't think it's at all unreasonable to expect moderators to perform the duties they were chosen to perform.

It's a volunteer position, sure, but I could go out and volunteer somewhere in my town and I can bet they'd politely tell me to not bother if I didn't actually do the work. There's no harm or shame in stepping down if one can't really fulfill the commitment anymore because their circumstances or interests changed.

There is also potential for absentee moderators to miss important announcements or changes. So when they come back, they might act in ways that are no longer fitting in with the site.

With that in mind, I think it's important to look at consistency of action instead of raw flag processing power. Someone who deals with 20 flags a week but is doing so consistently every week is a more active and involved moderator than someone who shows up every 3 months and blows through 240 flags in one go.

share|improve this answer
add comment

I suppose this is a logical next step after the history page breaking down participation per mod and per action. I would certainly expect mods to be clearing flags frequently, but as long as they're doing something (however minimal), is there a reason not to let them keep at it? In other words, what's the point of losing whatever amount of effort they are putting in? There isn't a maximum number of moderators -- more can always be elected -- so what's the benefit in dropping people that aren't dismissing enough flags? If someone clears 5 flags/day and you un-mod them, that's 5 more flags every day that the other mods will have to take care of, and I'm not sure what was gained

share|improve this answer
14  
the point is to make it clear that community moderators, if they wish to be elected, need to participate in the primary duty of a moderator when they are on the site. Otherwise they are "moderators" in name only. –  Jeff Atwood Aug 9 '11 at 6:48
add comment

I'm not a mod, so I don't know the behind-the-scenes details, but it seems reasonable to quantify somehow the expected work of a moderator.

There are two points that I think are important to make sure this stays reasonable, though, given the volunteer nature of the job.

First, you're talking about moderators performing their duties, primarily the processing of flags, while on the site. Such duties should not interfere with the moderator's ability to be a "civilian", i.e., use SE for their own research. In other words, I don't think a mod should be expected to be processing flags any time the system has them logged in. Your proposed "standard of duty" will probably help with this, in fact. Given a standard, moderators can know when they have put in their time for the day and can feel free to use the site in a personal capacity.

Second, I think throwing bodies at this problem is a good thing to try -- I don't think it's "avoiding the issue" at all -- and your proposal would also make this easier. Once you come up with a rough figure: "A mod should process N flags a day on average", you can do some simple division and figure out how many moderators a site should have in order to keep the per-moderator workload stable.

share|improve this answer
3  
if those "bodies" close ~50 flags in a 30 day period, while being active on the site the entire time.. and the average number of flags per day is ~900.. I'm sorry, how does that help again? –  Jeff Atwood Aug 9 '11 at 7:26
2  
I say "Yes" to the standard -- any new moderators will be held to the same standard. The "more bodies" part is, once you establish the standard, deficit = num_existing_mods - (900 / N) tells you that the site needs deficit new moderators. –  Josh Caswell Aug 9 '11 at 7:43
4  
@Jeff Josh is saying he supports your proposal -- but it's up to you to pick a good number for N (number of flags to close per day on average). He's simply saying that if "reasonable" still ends up with a deficit, having a standard will help you figure out how many more moderators you need. –  NickC Aug 9 '11 at 8:00
    
@renesis will we get soldiers who fight, or soldiers who only eat rations? It does nothing to resolve that problem... and adding a bunch more soldiers and/or rations does not either... –  Jeff Atwood Aug 9 '11 at 8:03
2  
Sure, but I'm a little confused, since it seems there is a misunderstanding. It's your proposal, I think this is just one argument in favor of it. –  NickC Aug 9 '11 at 8:11
1  
@Jeff: Which rations are inactive moderators eating? –  Paŭlo Ebermann Aug 10 '11 at 16:35
    
@paulo unless you want to have hundreds of moderators, they are "eating" a moderator slot without performing the most crucial function of a moderator. –  Jeff Atwood Aug 10 '11 at 21:23
1  
@Jeff: So, the problem is that you don't want to have to trust hundreds of people with the moderator powers? (This is okay, it just should be pointed out.) –  Paŭlo Ebermann Aug 10 '11 at 21:31
add comment

I don't think that forcing a moderator to meet some standard is the the right approach. Some of us have more time than others, or just happen to be able to monitor the sites more frequently. At least on the SE 2.0 sites, if you only are on in the evenings, say, you might miss the only flag of the day because another mod was able to watch the site at work. (Obviously this doesn't apply to SO; my point is that there are higher traffic times that if a moderator doesn't catch, he won't appear to be doing as much.) There are some mods who seem superhuman in flag-clearing abilities; they can rip through a couple hundred flags without missing anything, while others need a bit more time to think about each one.

If moderators are the community's exception handlers, shouldn't SE be the moderator exception handler? Meaning, if a community coordinator finds evidence that a moderator isn't pulling his fair share, he can look into the situation furthur. Base this on the moderator statistics if you want, but don't publish the criteria, and for sure don't make it something we have to worry about. Use it as a guideline to alert the community coordinators that there may be a problem, and then handle it on a case by case basis.

share|improve this answer
5  
I don't think Stack Overflow suffers from the "only flag of the day" problem. –  Anna Lear Aug 9 '11 at 13:45
1  
@AnnaLear Obviously not :) It still has it's high times and low times though, which was what I wanted to emphasise. Another thing - if several mods all clear flags from 10-3 say, and another mod is active at other times, those several mods will probably not be clearing as many flags apiece simply because there's more active at that time. –  Ghost User Aug 9 '11 at 14:01
    
I get what you're saying here, but I think you're still missing the point that there are plenty of flags to go around on Stack Overflow at almost any time of the day that you happen to be around. A lot of people who haven't been SO moderators simply can't appreciate the sheer volume of flags that are raised on that site. And having not been one myself, I can only go by stories and screenshots that I've seen the moderators post. I don't really think this is the problem when it comes to SO in particular. –  Cody Gray Aug 10 '11 at 4:05
add comment

Processing flags isn't the only useful thing a moderator can do. Judging performance only by flag processing seems unfair and not necessarily useful.

share|improve this answer
1  
it is of course not the only thing, but it is the primary and most important duty. –  Jeff Atwood Aug 10 '11 at 21:25
add comment

Perhaps diamond-hood should have a term limit?

Or, maybe, two types of moderators: full diamonds can box people and otherwise take radical steps. Mere paste moderators can just act on flags unilaterally. Maybe, oh, 30K makes a paste moderator?

share|improve this answer
add comment

I don't know what the statistics are on this, but bear with me...

Yes you cannot keep adding moderators to remedy the problem because the problem will keep growing as the user-base grows, thus more and more moderators... One thing that comes to mind, how often do moderators see people abusing flags? Are there steps in place to warn or maybe even take rep away from users who are caught abusing flags? Do the users know that they will lose rep when they abuse flags?

Also, the idea of using high rep users to sort out flags is an interesting thought...

Why not allow users with a considerable amount of reputation (at the moderator's discretion) "Apply" to be a "Flag-Moderator", and maybe give these Flag-Moderators a point system of their own, separate from the regular user point system based on number of flags covered in an hour/day/week/month/all time, the number of topics of flags covered, etc. This will give the same sense of "I MUST HAVE POINTS NOM NOM NOM" as the rest of the stack exchange community, while helping to remedy the problem.

On another note, when someone VOLUNTEERS for a position (such as Moderator) you DO NOT want to give punishment as stated in the first post

I propose that on Stack Overflow, all elected community moderators must close a 'reasonable' number of flags while they are on the site.

Yes moderators should be pulling some weight because they took on the role, but imposing a consequence of a type of "Quota" will only shun the idea of becoming a moderator. Instead install a reward system over going for a penalty system when it comes to volunteers. If moderators are inactive they should be let go from the position.

share|improve this answer
2  
Users don't lose rep for bad flags, but flags are sorted by flag weight -- if a user posts bad flags they'll get marked invalid, their flag weight will go down, and their flags will go down in the list. As for your last paragraph, you said not to have a penalty system and then ended with "If moderators are inactive they should be let go" -- I'm pretty sure that's all Jeff is talking about, the "penalty" is not being a mod anymore –  Michael Mrozek Aug 9 '11 at 17:49
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .