What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

What, in your opinion, are the duties of a moderator, and what activity can we datamine from the database that will give us a good idea whether a given individual already performs those activities, or activities like that?

For instance, I believe a good moderator should have at least a shallow understanding of a broad range of popular topics, so they can better understand moderation issues across several topics, rather than needing niche moderators for each topic. Therefore I found out which of the current nominated people have the generalist badge and noted it on the comments below them.

I suspect there must be detectable patterns elsewhere in the system that speak towards

  • Amount of moderation (level of moderation participation vs simply answering and voting)
  • Good vs bad moderation (??? Maybe agreement with community, maybe first to vote to close, or last to vote to close means something...)
  • Overall activity level on site
  • Level of participation in comments (guidance to others, comment votes)

While high rep is one useful measure of site participation, it hardly has any bearing on whether the person actively moderates in the form of open, close, delete, flag, acting on flags, etc. Reading comments from a given user could give a great deal of information on their ability to communicate with others, their adherence to site rules and policies, and their approach to situations that need attention - I'm not sure this could be automated into a nice number, but perhaps there's some keyword searching that can be done on comments to give a log of relevant (ie, moderation type) comments.

So what data is collected or could be collected, and how could that data be transformed into heuristics to give us a better feel as to how a potential moderator is already moderating, and whether their actions are in line with your desired goals for moderators?

share|improve this question
2  
FYI: This is already being done for diamond moderators to a degree. –  Diago Jan 20 '11 at 15:51
1  
The intention is to come up with a list of ideas that may be used in addition to reputation to determine who should be eligible for the primary in the elections - or in other words a slightly better situation than this: meta.stackexchange.com/questions/75527/… I don't expect we can implement anything in time for this election cycle, but if we can improve the next cycle it will be worth it. –  Adam Davis Jan 20 '11 at 15:51
    
@Pollyanna, when I requested such information I was told I was expected to "know" this, given my access to mod flags and such. But how a normal user should be able to judge this, I don't know... –  Ivo Flipse Jan 20 '11 at 16:16
1  
One thing I don't think is a great heuristic is having the 'generalist' badge. I don't think it's completely relevant to the election. –  jjnguy Jan 20 '11 at 16:54
    
@Justin - No, it's not a great heuristic, but it is my opinion that it's relevant for the reasons I wrote above. –  Adam Davis Jan 20 '11 at 16:57
1  
@Justin - As a 'specialist', I agree. Once any user has been on the site long enough, I'd hope that they'd be able to identify spam, offensive content, etc. no matter their areas of expertise. –  Brad Larson Jan 20 '11 at 17:00
    
@Polly, getting votes in a specific topic doesn't necessarily imply that a user isn't viewing other topics. So, having the badge is good I suppose, but not having it shouldn't be negative. –  jjnguy Jan 20 '11 at 17:03
    
@Brad, agreed . –  jjnguy Jan 20 '11 at 17:03
1  
@Justin - That is a reasonable way to view it. Again, I'm just trying to identify those traits and aspects that are visible and measurable that might help us to create a better selection process than just the 30 highest rep users that happen to nominate themselves. –  Adam Davis Jan 20 '11 at 17:09
    
@Polly, fair enough. (As a fairly high-rep user though, I like this process ;)) –  jjnguy Jan 20 '11 at 17:11
    
@Justin There are still benefits to increasing one's reputation beyond 10k, aren't there? It's very tempting to nominate myself though, kicking someone else off the list who would do a better job than I just to make a point about the silliness of the current metric. –  Adam Davis Jan 20 '11 at 17:16
    
@Polly, this is the first real benefit I've seen for having rep above 10k. (besides 10k tools) I do stand by a comment I made earlier though. - "Here is how I see it. The percentage of good mods in the 15k+ crowd is much higher than the percentage of good mods in the 3k+ crowd. So narrowing it down by rep is good for the community in general. (Even if it is detrimental to a few lower rep users.)" –  jjnguy Jan 20 '11 at 18:22

7 Answers 7

Number of questions vs number of answers.

I would expect better moderators to be primarily answerers. They are on the site to help other people and by browsing/searching the site for questions to answer they see more posts than someone who's just on to ask a question and wait for an answer.

I suppose this is measured by reputation and tag/generalist badges.

share|improve this answer
5  
To a point, I agree. But there's nothing wrong with a moderator's having a lot of answers and a lot of questions, either. I'm looking more for quality. –  mmyers Jan 20 '11 at 16:17

Their ability to communicate with the community is important so I would think that comment participation and votes would be a good indicator. Perhaps length of comment as well.

share|improve this answer
    
Length of comments... tricky one. Long comments could mean s/he interacts nicely OR s/he has trouble conveying the point briefly. I would go with up-votes on comments. –  JP19 Jan 20 '11 at 17:09
4  
@JP19 Upvotes on comments are just as tricky as length. Comments are just as likely to get upvotes for simply being snarky, since that's pretty much where you can safely snark. –  Grace Note Jan 20 '11 at 17:17
    
A good moderator should have a good sense of humour :) –  JP19 Jan 20 '11 at 17:24

Some good ideas can be got from the badge definitions ( http://stackoverflow.com/badges & http://meta.stackoverflow.com/badges ) :

Examples:

1) Flagging posts which went on to get removed.

2) How many upvotes s/he has given to competing answers

Infact as I am browsing the list and writing the answers, I realize a lot of them indicate potential to be a good moderator.

share|improve this answer

Disclaimer - I have nominated myself to be a mod.

Here is a quick list off the top of my head:

Community participation badges are important:

  • Copy Editor/ Strunk
  • Civic Duty
  • Sportsmanship
  • Enthusiast/Fanatic
  • Yearling (x2 preferably)
  • Pundit (maybe)

Also, quality of posts. This could be measured by votes per post.

Number of comments.

Number of votes - up and down.

First to vote to close on a question that gets closed and not re-opened.

share|improve this answer
    
I would point out that judging on Yearling is biased towards the Trilogy, which is the only place currently where it's even feasible to have a single year, nevermind two. But, really, the "top 30 by rep" is also only as much a problem on the trilogy currently. –  Grace Note Jan 20 '11 at 17:15
    
Also, judging based on the Yearling badge might exclude someone like Michael Mrozek from consideration. Given his involvement on SO and Meta, I consider him a solid moderator candidate even if he hasn't been a member for a full year. –  Brad Larson Jan 20 '11 at 17:42
2  
Don't forget the [string] badge. That one's key... –  Shog9 Jan 20 '11 at 17:53
    
@Grace - You can receive the yearling badge on sites outside the trilogy, but only those that were converted from SE 1.0 sites. I've already received at least on yearling badge outside of SOFU. –  Adam Davis Jan 20 '11 at 18:04
    
@Shog9 /me runs off to check... Rats! I've obviously got a lot of work to do... –  Adam Davis Jan 20 '11 at 18:06
    
@Shog, I do have all of the badges listed. After all, I do feel that I would make a great moderator. I do appreciate your joke though. –  jjnguy Jan 20 '11 at 18:24

How many questions has he closed?

Too bad this doesn't differentiate between a zealot who tries to close everything and someone who always delivers the 5th and final close vote. It also doesn't take into account when question are reopened later, but at least it shows that the user tries to keep the site 'clean'.

share|improve this answer
    
I guess the API doesn't provide this, but at least Googling for "Closed by -name user-" "off-topic" would give an indication –  Ivo Flipse Jan 20 '11 at 16:14
    
And a remark by @random: "If you can't spot a top tier from those running, they're not standing out as best as they could." –  Ivo Flipse Jan 20 '11 at 16:15
    
Actually @Ivo, there is a query on data.stackexchange to see how many close votes a user has cast in which areas. –  nhinkle Jan 20 '11 at 19:17

(Disclaimer: I've nominated myself for a moderator position on Stack Overflow.)

A moderator is there to keep the site clean and functioning, and not to ask or answer technical questions (although they are still community members and free to do so), so reputation or question / answer statistics should not be a primary deciding factor. Yes, you want someone who has shown commitment to the community, and familiarity with the tools you have access to at the 10k level definitely helps, but I don't think it matters much if someone has a 15k or 50k reputation or that their question / answer ratio differs from other candidates.

A moderator must have a thick skin and be polite and patient with other users, so I'd track how many offensive / abusive flags had been filed against their answers, as well as flags against their comments. While people can have others take revenge against them with frivolous flags, I'd hope that these wouldn't account for a significant number and that the overall number would show how many others thought their responses to users were inappropriate.

You could also track close vote numbers in various categories, particularly initial duplicate close votes, which take a little more effort to cast. However, one person's janitor is another's "close Nazi", so this is a subjective measure. Same goes for delete votes cast. Even as a subjective item, this can allow others to determine the moderation-related activity of this candidate.

In that same line of thinking, tracking the total number of edits performed by a user should reflect how willing they are to improve rough content rather than voting to close it or just leaving it to the tumbleweeds.

Spam flags and offensive flags cast would also seem to be good measures, and I think most could agree on the value of those.

Finally, interaction with moderators through the flags for attention would be a good statistic, particularly if the weight given to the flags cast by a user based on part performance (something that appears to be tracked internally) could be made public.

Everyone has the ability to keep contributing to the community through good questions and answers, so these non-reputation-based statistics would seem to track how well someone would take on the unique capabilities of a moderator.

share|improve this answer
    
I like measuring how much of their content has been flagged as spam, abusive, etc. –  Adam Davis Jan 20 '11 at 17:13
2  
I particularly like the idea of measuring how often they flag the moderators for attention, and how often that flag is proactively acted upon. This would be a great indicator of, "Obviously this person knows what the moderators should be doing, and runs into enough moderator work that we should skip the middleman and simply let them moderate." –  Adam Davis Jan 20 '11 at 17:18
    
I can't agree with the measuring of editing. I would expect and hope that users at a lower level act as editors to improve content, but moderators aren't editors, and they shouldn't be tasked with this role. It is easy to measure, though, and there may be some correlation. –  Adam Davis Jan 20 '11 at 17:18
    
@Pollyanna - I guess I thought of it as a counterbalance to the close votes. How readily do people give up on questions with rough English and just close them vs. how often are they edited? I, personally, would expect to help with editing for clarity as a moderator rather than just closing items. Also, can't people flag items for moderator attention to be edited? –  Brad Larson Jan 20 '11 at 17:35

The reversal badge might be interesting evidence of qualities a moderator might have - the ability to salvage even the worst question by providing a useful, valuable, and noteworthy answer.

share|improve this answer
    
I like that you only commented on users that have the badge this time. –  jjnguy Jan 20 '11 at 18:23

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .