28

Please stop showing audits to Moderators. I'm not referring to all users, just Diamond Moderators.

We see so much crap while cleaning, we don't need the extra round of crap just make the system go - "nyah, nyah, lookit this and reject so that the engine can go hurrrah!!!11"

I mean really, having to click around just to reject an obvious crap audit like https://superuser.com/review/suggested-edits/146538 is utterly pointless.

Utterly pointless. So Just. Please. Stop.

  • 11
    A more general solution would be to show the audit less often for any user that regularly pass the audits. – Emil Vikström Sep 10 '13 at 12:31
  • 1
    @EmilVikström Which was already suggested by Gilles, linked to in the first sentence of Sathya's post. – slhck Sep 10 '13 at 12:36
  • 7
    I am a regular user, not a moderator. Iirc for past 3-4 months I passed few hundreds audits, failed none of them. I don't complain (except for audits that are bad / wrong design). What is that makes you so special compared to me? – gnat Sep 10 '13 at 13:37
  • 1
    @gnat There's an metric easily determined by code that determines a mod is a good reviewer. There isn't such a metric for you. Trying to limit audits for you, since it would be more complex than just checking an IsModerator flag, would both be a lot more work for developers, and have a much greater risk of also reducing or eliminating audits for users performing inappropriate behavior. – Servy Sep 10 '13 at 13:55
  • 6
    @Servy I don't see any mention of such a "metric" in this feature request. The way it is stated now, it sounds as if we should just take for granted that moderators don't need audits. With all due respect, I don't buy this – gnat Sep 10 '13 at 13:58
  • 2
    I wholeheartedly support this. In fact, loading the fake suggestions takes so long it's a waste of time to review pending edits. I've been getting one for every batch I reviewed.. at least it feels like that. It's just not funny anymore. If I want to review edits, I'm not doing it for any badges—I merely want to set the ground truth for what's acceptable and what is not. If the system assumes I'm not doing it with the greatest attention, SE should think again about how they trust moderators in general. – slhck Sep 10 '13 at 13:58
  • 3
    @gnat I suppose that is easily verifiable. If we look at the mod failure rate for audits, and it's not very, very high, then there is information to possibly be gained from having them do them. If they're passing all of the audits, then it means they aren't doing anything to catch mods reviewing poorly (either because the audits aren't good enough, or the mods are doing their job, either way, the audit is still pointless). – Servy Sep 10 '13 at 14:01
  • 3
    The suggested edit audits themselves are utter crap. – Michael Hampton Sep 10 '13 at 14:06
21

The first time I came upon a suggested edit audit, I almost send a mod message to the user who had suggested the edit. Only halfway through writing the mod message I realized there was a chance the - truly horrible - edit might have been an audit. I'm pretty sure a fellow mod shared a similar story in Teacher's Lounge once, so this isn't just me.

For regular users, an audit is a matter of a couple of seconds, at most. Moderators, on the other hand, may spend some time checking the editor's history, as we usually do when we spot trouble. Or, worse, send a stern mod message to someone completely innocent, as I almost did. So, auditing moderators, other than being utterly pointless, may also be a not-so-insignificant waste of time for us.

2

I'm sure you've never shortchanged an edit review, and could be made exempt from audits with no dire consequences.

...But there are 359 other diamond moderators on the network. Some of them have considerably less experience. Some of them handle a lot more reviews. All of them are human, and liable to make mistakes occasionally.

At the end of the day, there's nothing that prevents a moderator from becoming fatigued when reviewing other than his own individual skill and patience, which can be and are present in our best non-diamond reviewers as well. Therefore, I still think the strategy I proposed on Gilles' feature request is the proper solution here: reduce the frequency of audits for active, accurate reviewers. This was implemented on May 7th, 2014, for moderators and everyone else.

  • 2
    Sound reasoning, but I think Yannis's point above is a good one. – Adam Rackis Sep 10 '13 at 20:44
  • 7
    So what's next? Flag handling audits? Fake posts to see how we handle them? Fake user profiles? Randomize the sites around and see if we notice? Official-looking meta posts shown only to moderators to see how we react to the troll? – Gilles Sep 10 '13 at 21:27
  • 1
    Flag audits are not a terrible idea, @gilles. feature-request – Shog9 Sep 10 '13 at 22:59
  • @Gilles well as long as you are positive that all 300+ moderators have studied and understood Moderator Newsletter guidance ("Flags Too Often Marked [declined]..."), I would say flag handling audits would be a terrible idea. Oh wait... did they? – gnat Sep 11 '13 at 7:33
  • 1
    @gnat Of course there are moderators who haven't read the guidance. But that's not what audits are about. In order to make audits provide actual guidance, as opposed to merely reprimanding people who are cheating, you'd have to fine-tune them, and to have a lot of them. I proposed flag handling audits because it's such a stupid idea. A flag handling training, now, that could be a good idea. With guidance: what you should do in this or that situation. Not like audits, which don't provide useful feedback: you're told that a decision is wrong, but not why it's wrong. – Gilles Sep 11 '13 at 9:00
  • @Gilles flag handling audits to prevent robo-declines, why not. And please don't tell me this never happens, because, well, it does happen – gnat Sep 11 '13 at 9:25
  • @gnat That was not a robo-decline: that was a moderator looking over a situation and taking a considered decision to decline all of these flags. The fact that this decision was wrong required information that could not be gathered. It's not the kind of clear-cut case that can be caught by audits. – Gilles Sep 11 '13 at 9:31
  • @Gilles yeah sure. 20+ times in line. Without even paying attention at the flagger's history / weight. Without even taking a look at involved meta discussions. Call it taking a considered decision if you wish but to me, this thing just smells that “STOP! Look and Listen” thing would be useful not only for regular guys – gnat Sep 11 '13 at 10:15
  • 1
    @gnat “Without even paying attention at the flagger's history / weight” Citation please. “Without even taking a look at involved meta discussions” There was one meta discussion, which the moderator had missed. What audit would you craft that would have caught this? This was not a case of the moderator clicking “decline” without thought. – Gilles Sep 11 '13 at 11:03
  • @gnat This is the flagger's perception. The moderator's perception was different. Note that I'm not defending the moderator's decision: I agree that he should have noticed that something was going on and dug further. But it was a considered decision (there is a deleted answer in that same thread where the moderator explains his decision). By the way, I misremembered earlier: there was no meta thread specifically about that cleanup. So again, no, this is not a case of a moderator not paying attention, it's very far from what an audit could catch. – Gilles Sep 11 '13 at 11:28
  • @Gilles this, as you call it, "perception" has been followed by a direct intervention of SE developer. Flagger's perception? give me a break – gnat Sep 11 '13 at 13:14
  • 2
    @gnat SE agreed that the moderator's action was incorrect and reverted its effect. That does not mean that they agreed that the moderator showed “a total disregard” and “lack of care”. Again, it's the difference between deliberately making a hash job (which audits work against) and making a honest mistake (for which audits are irrelevant). – Gilles Sep 11 '13 at 13:38
  • @Gilles how about us sticking with the purpose of the audits, as these are stated in tag wiki? "Test items in review queues that are designed to help new reviewers hone their moderation skills, while nudging more experienced users that don't seem to be paying close attention to what they're reviewing." Per my reading, it's not exactly about prevention of deliberately making a hash job – gnat Sep 11 '13 at 13:48
-8

Moderators are expected to be able to easily tell the difference between obviously bad and good posts.

You are more experienced than most in this regard, and also have a bigger responsibility of doing the right thing. I therefore think auditing moderators is just as useful as auditing any other user, to get a measure both on the moderator performance and what failure rate we can expect from a good reviewer.

If even moderators fail the audits chances are that they either are not that good mods, or that the audit system is having the wrong idea about good/bad posts. In the latter case we should fix the audit system because it's meant to only include obvious posts.

A counter-proposal is that the audit probability should go down for any user that regularly pass the audits.

  • 6
    "A mod should easily be able to tell the difference between obviously bad and good posts." And so audits should not be necessary to begin with; they're an unnecessary hurdle. – BoltClock's a Unicorn Sep 10 '13 at 12:38
  • 2
    I mean that such a measurement can be useful. – Emil Vikström Sep 10 '13 at 12:42
  • 20
    I think we should just take for granted that a moderator will not just mash one button repeatedly for getting badges faster or reducing queue sizes and audits are not good enough to catch anyone besides that. – nijansen Sep 10 '13 at 12:42
  • 2
    No, auditing is a waste of time. I've been a mod for over 2 years and reviewed over 2800 edits. I can see a useless audit just by the edit summary. Having to make me read through the entire crap and force an action is an utter waste of my time – Sathyajith Bhat Sep 10 '13 at 12:52
  • @BoltClock'saUnicorn it's so easy to catch on slightly slippery wording to justify downvote, isn't it – gnat Sep 10 '13 at 14:01
  • 2
    @gnat You don't need any wording at all. If you want to downvote, you can downvote. If someone wants to justify their reasons to downvoting they can, but there is no obligation. – Servy Sep 10 '13 at 14:09
  • 1
    @Servy sure (did you forgot to add cliche votingisdifferentonmeta?) I merely vonder how it happen that upvoted senseless comments tend to stick with posts that appear downvoted without a clear reason – gnat Sep 10 '13 at 14:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .