Results of Pro Tempore elections experiment

Question

Last year I announced an experiment of electing Pro Tempore moderators. We've completed 26 elections (with 4 more scheduled), including the Parenting pilot. One election, Tezos, picked the initial slate of moderators for a new site. The other elections were for one or two moderators to replace people who were stepping down for various reasons. Two elections ended with no nominees. (One of those elections also precipitated the closing of Augur due to a lack of caretakers.) In summary:

Slots  Nominees  Elections  
-----  --------  ---------
    1         0          1 <- failed
    1         1          7 <- noncompetitive 
    1         2          1
    1         3          6
    1         4          1
    1         5          2
    2         0          1 <- Augur
    2         1          1 <- noncompetitive
    2         2          4 <- noncompetitive
    2         5          1
    3         5          1 <- Tezos

Further summarizing:

Result         Elections
------         ---------
failed                 2
noncompetitive        12
competitive           12

Whether this is a positive result depends quite a bit on your baseline expectation. Less than half of elections being competitive and 8% of them failing to get any nominees at all doesn't seem very healthy in terms of democratic best practices. On the other hand, virtually all of these elections were for our smallest communities where we've had a difficult time finding volunteers in the past.

Elections replaced a nomination process that went something like this:

1. A CM identifies people who might potentially be interested and qualified to moderate a site.
2. The CM emails their top choices to fill the vacant moderator slots.
3. Potential moderators consider the offer and (hopefully) respond within a few days.
4. If someone decides not to accept the role, the CM goes back to step #2 and contacts their next choice.
5. Once all the slots are claimed, the CM appoints those volunteers as moderators.

Step #3 can be time consuming since it's reasonable to let people have a few days to consider an offer to become a moderator. Also, vacation, holidays, unexpected busyness, spam filters, etc. and so on can result in the process being stalled. The other issue is sometimes people are reluctant to take on the responsibility and we don't find any suitable candidates. In that case, we'd usually post a plea for volunteers on meta.

From my perspective, the election process simplifies that step by giving everyone on the site a fixed period of time to decide whether they want to take on the role of moderator. Everyone, including the CMs, have the same timeframe (enforced by the system) for arriving at replacement moderators. Plus the process takes place on the site rather than via email, which eliminates several barriers. Instead of keeping track of in-flight emails and users who have declined, we just need to look in at key moments of the election process to be sure it's proceeding smoothly.

Speaking of which, when selecting moderator candidates, CMs have access to quite a bit of information about a user that's not readily available to the voting public. We manually remove nominees who have come off a suspension within the year, but there are also moderator annotations and participation patterns we use as red-flags. At least one candidate who I would not have selected for appointment was able to run in a Pro Tempore election. I'm not convinced our appointment criteria selects better moderators than a democratic election, but it does avoid candidates with obvious concerns.

While some sites did take the initiative to set up a questionnaire (notably Interpersonal Skills) most communities left discussion of candidates on the nomination page and (occasionally) chat. This isn't too concerning since these are smaller communities where everyone more or less knows everyone else's moderation philosophy. There was a little confusion when the questionnaire was published by a non-employee, but that seems like a problem with my communication more than with the process.

Summary

The experiment served our communities at least as well as CM appointments with considerably less manual overhead. As a part of the current site lifecycle reform, we plan to continue using elections to select moderators for all sites. We are also considering tweaks to the process that would make the process more transparent and fair. (Stay tuned!)