27

Please stop showing audits to Moderators. I'm not referring to all users, just Diamond Moderators.

We see so much crap while cleaning, we don't need the extra round of crap just make the system go - "nyah, nyah, lookit this and reject so that the engine can go hurrrah!!!11"

I mean really, having to click around just to reject an obvious crap audit like https://superuser.com/review/suggested-edits/146538 is utterly pointless.

Utterly pointless. So Just. Please. Stop.

Improve this question
8
  • 11
    A more general solution would be to show the audit less often for any user that regularly pass the audits.
    – Emil Vikström
    Sep 10, 2013 at 12:31
  • 1
    @EmilVikström Which was already suggested by Gilles, linked to in the first sentence of Sathya's post.
    – slhck
    Sep 10, 2013 at 12:36
  • 8
    I am a regular user, not a moderator. Iirc for past 3-4 months I passed few hundreds audits, failed none of them. I don't complain (except for audits that are bad / wrong design). What is that makes you so special compared to me?
    – gnat
    Sep 10, 2013 at 13:37
  • 1
    @gnat There's an metric easily determined by code that determines a mod is a good reviewer. There isn't such a metric for you. Trying to limit audits for you, since it would be more complex than just checking an IsModerator flag, would both be a lot more work for developers, and have a much greater risk of also reducing or eliminating audits for users performing inappropriate behavior.
    – Servy
    Sep 10, 2013 at 13:55
  • 6
    @Servy I don't see any mention of such a "metric" in this feature request. The way it is stated now, it sounds as if we should just take for granted that moderators don't need audits. With all due respect, I don't buy this
    – gnat
    Sep 10, 2013 at 13:58
  • 2
    I wholeheartedly support this. In fact, loading the fake suggestions takes so long it's a waste of time to review pending edits. I've been getting one for every batch I reviewed.. at least it feels like that. It's just not funny anymore. If I want to review edits, I'm not doing it for any badges—I merely want to set the ground truth for what's acceptable and what is not. If the system assumes I'm not doing it with the greatest attention, SE should think again about how they trust moderators in general.
    – slhck
    Sep 10, 2013 at 13:58
  • 3
    @gnat I suppose that is easily verifiable. If we look at the mod failure rate for audits, and it's not very, very high, then there is information to possibly be gained from having them do them. If they're passing all of the audits, then it means they aren't doing anything to catch mods reviewing poorly (either because the audits aren't good enough, or the mods are doing their job, either way, the audit is still pointless).
    – Servy
    Sep 10, 2013 at 14:01
  • 3
    The suggested edit audits themselves are utter crap.
    – Michael Hampton
    Sep 10, 2013 at 14:06

3 Answers 3

Reset to default
20

The first time I came upon a suggested edit audit, I almost send a mod message to the user who had suggested the edit. Only halfway through writing the mod message I realized there was a chance the - truly horrible - edit might have been an audit. I'm pretty sure a fellow mod shared a similar story in Teacher's Lounge once, so this isn't just me.

For regular users, an audit is a matter of a couple of seconds, at most. Moderators, on the other hand, may spend some time checking the editor's history, as we usually do when we spot trouble. Or, worse, send a stern mod message to someone completely innocent, as I almost did. So, auditing moderators, other than being utterly pointless, may also be a not-so-insignificant waste of time for us.

Improve this answer
3
2

I'm sure you've never shortchanged an edit review, and could be made exempt from audits with no dire consequences.

...But there are 359 other diamond moderators on the network. Some of them have considerably less experience. Some of them handle a lot more reviews. All of them are human, and liable to make mistakes occasionally.

At the end of the day, there's nothing that prevents a moderator from becoming fatigued when reviewing other than his own individual skill and patience, which can be and are present in our best non-diamond reviewers as well. Therefore, I still think the strategy I proposed on Gilles' feature request is the proper solution here: reduce the frequency of audits for active, accurate reviewers. This was implemented on May 7th, 2014, for moderators and everyone else.

Improve this answer
15
  • 2
    Sound reasoning, but I think Yannis's point above is a good one.
    – Adam Rackis
    Sep 10, 2013 at 20:44
  • 7
    So what's next? Flag handling audits? Fake posts to see how we handle them? Fake user profiles? Randomize the sites around and see if we notice? Official-looking meta posts shown only to moderators to see how we react to the troll?
    – Gilles 'SO- stop being evil'
    Sep 10, 2013 at 21:27
  • 1
    Flag audits are not a terrible idea, @gilles. feature-request
    – Shog9
    Sep 10, 2013 at 22:59
  • @Gilles well as long as you are positive that all 300+ moderators have studied and understood Moderator Newsletter guidance ("Flags Too Often Marked [declined]..."), I would say flag handling audits would be a terrible idea. Oh wait... did they?
    – gnat
    Sep 11, 2013 at 7:33
  • 1
    @gnat Of course there are moderators who haven't read the guidance. But that's not what audits are about. In order to make audits provide actual guidance, as opposed to merely reprimanding people who are cheating, you'd have to fine-tune them, and to have a lot of them. I proposed flag handling audits because it's such a stupid idea. A flag handling training, now, that could be a good idea. With guidance: what you should do in this or that situation. Not like audits, which don't provide useful feedback: you're told that a decision is wrong, but not why it's wrong.
    – Gilles 'SO- stop being evil'
    Sep 11, 2013 at 9:00
  • @Gilles flag handling audits to prevent robo-declines, why not. And please don't tell me this never happens, because, well, it does happen
    – gnat
    Sep 11, 2013 at 9:25
  • @gnat That was not a robo-decline: that was a moderator looking over a situation and taking a considered decision to decline all of these flags. The fact that this decision was wrong required information that could not be gathered. It's not the kind of clear-cut case that can be caught by audits.
    – Gilles 'SO- stop being evil'
    Sep 11, 2013 at 9:31
  • @Gilles yeah sure. 20+ times in line. Without even paying attention at the flagger's history / weight. Without even taking a look at involved meta discussions. Call it taking a considered decision if you wish but to me, this thing just smells that “STOP! Look and Listen” thing would be useful not only for regular guys
    – gnat
    Sep 11, 2013 at 10:15
  • 1
    @gnat “Without even paying attention at the flagger's history / weight” Citation please. “Without even taking a look at involved meta discussions” There was one meta discussion, which the moderator had missed. What audit would you craft that would have caught this? This was not a case of the moderator clicking “decline” without thought.
    – Gilles 'SO- stop being evil'
    Sep 11, 2013 at 11:03
  • @Gilles there you go: "combination of * scale (it was applied to 22 flags I think) * the lack of care (it seemed in his answer that it was just an "its filling up the queue so I'll get rid of it cause I don't want to deal with it" (not hard quote obviously) type of thing) * the seeming lack of knowledge of the meta posts / mod comments etc that made me do what I did (quoted superbly by Daniel Beck)..."
    – gnat
    Sep 11, 2013 at 11:11
  • "... * but most importantly, a total disregard for what the "root cause" of this was. I would normally assume that 1. rejecting 22 flags in a row from a single user is extremely atypical and that 2. doing this to a user that has a flag weight of 700+ is even more exceptional..." And don't forget to add to this that SE developers had to manually clean up flags database to wipe out dirty laundry left after this considered decision
    – gnat
    Sep 11, 2013 at 11:12
  • @gnat This is the flagger's perception. The moderator's perception was different. Note that I'm not defending the moderator's decision: I agree that he should have noticed that something was going on and dug further. But it was a considered decision (there is a deleted answer in that same thread where the moderator explains his decision). By the way, I misremembered earlier: there was no meta thread specifically about that cleanup. So again, no, this is not a case of a moderator not paying attention, it's very far from what an audit could catch.
    – Gilles 'SO- stop being evil'
    Sep 11, 2013 at 11:28
  • @Gilles this, as you call it, "perception" has been followed by a direct intervention of SE developer. Flagger's perception? give me a break
    – gnat
    Sep 11, 2013 at 13:14
  • 2
    @gnat SE agreed that the moderator's action was incorrect and reverted its effect. That does not mean that they agreed that the moderator showed “a total disregard” and “lack of care”. Again, it's the difference between deliberately making a hash job (which audits work against) and making a honest mistake (for which audits are irrelevant).
    – Gilles 'SO- stop being evil'
    Sep 11, 2013 at 13:38
  • @Gilles how about us sticking with the purpose of the audits, as these are stated in tag wiki? "Test items in review queues that are designed to help new reviewers hone their moderation skills, while nudging more experienced users that don't seem to be paying close attention to what they're reviewing." Per my reading, it's not exactly about prevention of deliberately making a hash job
    – gnat
    Sep 11, 2013 at 13:48
-8

Moderators are expected to be able to easily tell the difference between obviously bad and good posts.

You are more experienced than most in this regard, and also have a bigger responsibility of doing the right thing. I therefore think auditing moderators is just as useful as auditing any other user, to get a measure both on the moderator performance and what failure rate we can expect from a good reviewer.

If even moderators fail the audits chances are that they either are not that good mods, or that the audit system is having the wrong idea about good/bad posts. In the latter case we should fix the audit system because it's meant to only include obvious posts.

A counter-proposal is that the audit probability should go down for any user that regularly pass the audits.

Improve this answer
7
  • 6
    "A mod should easily be able to tell the difference between obviously bad and good posts." And so audits should not be necessary to begin with; they're an unnecessary hurdle.
    – BoltClock's a Unicorn
    Sep 10, 2013 at 12:38
  • 2
    I mean that such a measurement can be useful.
    – Emil Vikström
    Sep 10, 2013 at 12:42
  • 20
    I think we should just take for granted that a moderator will not just mash one button repeatedly for getting badges faster or reducing queue sizes and audits are not good enough to catch anyone besides that.
    – nikolas
    Sep 10, 2013 at 12:42
  • 2
    No, auditing is a waste of time. I've been a mod for over 2 years and reviewed over 2800 edits. I can see a useless audit just by the edit summary. Having to make me read through the entire crap and force an action is an utter waste of my time
    – Sathyajith Bhat
    Sep 10, 2013 at 12:52
  • @BoltClock'saUnicorn it's so easy to catch on slightly slippery wording to justify downvote, isn't it
    – gnat
    Sep 10, 2013 at 14:01
  • 2
    @gnat You don't need any wording at all. If you want to downvote, you can downvote. If someone wants to justify their reasons to downvoting they can, but there is no obligation.
    – Servy
    Sep 10, 2013 at 14:09
  • 1
    @Servy sure (did you forgot to add cliche votingisdifferentonmeta?) I merely vonder how it happen that upvoted senseless comments tend to stick with posts that appear downvoted without a clear reason
    – gnat
    Sep 10, 2013 at 14:17

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .