26

Today, I failed an (unfair and incorrect) audit, the first one in a while, and I'm now banned from reviewing for two days. I naturally find this unfair and stupid, and I would really like to get the edit priviledge back.

I'm repeatedly failing these audits it seems, last time I brought it up in the chat with @Shog9, and recently I have not come across a single one that has been fair or even useful as an audit at all in my eyes. The most recent one, and the one that banned me, was this. In my opinion, this is a legitimate answer, other than the fact that the user chose to say that he encountered the problem as well, but it does provide an answer. Might be a good one, might be a bad one, but that's irrelevant in my opinion, an answer is an answer, no matter how correct.

Audits are incredibly annoying, and I know I'm a good reviewer, but I still fail at some. Please consider doing something about it, as it is really annoying to get this feedback when reviewing something that's actually correct, and on top of it all resulting in being banned from contributing through reviewing for two days, which happened to me. It isn't exactly motivating me to keep caring about reviewing.

Edit: It doesn't seem like anything has happened about this, I have complained again, and asked for clarification on how the audit system really works. Since this post, I have been quarantined/banned from reviewing 2-3 more times, once having it lifted (thanks Shog9), the rest waiting it out. As previously mentioned, it isn't exactly motivating me to continue reviewing.

12
  • 26
    These audits are getting out of hand. I thought the whole point was that they were supposed to be blatantly obvious to make sure people were paying attention. Even I failed one yesterday on a tricky one that got converted to a comment...
    – animuson StaffMod
    Commented Feb 6, 2013 at 7:54
  • 7
    I can assure you I'm paying attention, they're just flat out wrong most cases, and in other cases at least discussable at the very least. There isn't a single "good" or "bad" approach to reviewing, and the audit-system is ruining it in my opinion.
    – Emil
    Commented Feb 6, 2013 at 7:57
  • 3
    This is why we can't have nice things. Commented Feb 6, 2013 at 8:08
  • 4
    When in doubt, I now check the original post. This takes a bit more time, but it gives much more information. If that is not conclusive, I use skip. Commented Feb 6, 2013 at 8:13
  • The audit can be defeated easily with userscript...
    – nhahtdh
    Commented Feb 6, 2013 at 8:25
  • 14
    You have answered 0 questions in sql-server. Whilst this doesn't mean your unknowledgeable about the subject it's an indication. Why did you click No Action Needed rather than Skip? Commented Feb 6, 2013 at 12:55
  • 7
    This answer was clearly not an answer, at best, it was a comment and you should have either skipped it (if unsure) or indicated that it was a bad answer. I get that you want to help, we love that you want to help, but we need audits that are correct. The system is trying to tell you that (and might be overzealous in doing so). That said, we could soften the language here, and possibly make a feature request to have a moderator enable auditing for a user again if we feel that they aren't just being lazy, but really fat fingered, or are obviously improving their audit ability.
    – casperOne Mod
    Commented Feb 6, 2013 at 13:37
  • 4
    @casperOne I wouldn't quite say clearly. I think you might be juxtaposing your experience erroneously.
    – user50049
    Commented Feb 6, 2013 at 15:32
  • 1
    @TimPost Perhaps? But if you're at a point where you're allowed in the review queue, you should have been around the block enough times to know that this isn't what we're looking for on Stack Exchange.
    – casperOne Mod
    Commented Feb 6, 2013 at 15:48
  • 3
    While I can understand the reason for the audit system, it is clearly not up to snuff. I too failed one because there was no way to pass it (the only options available were to do nothing or skip it; I wasn’t going to skip it because I already checked it out, and since it was already deleted, do nothing was the only logical choice). Worse, the audits are often pointless because many of them are just so plainly obvious. Worst still, the audits are foist on everyone, including high-rep users, as though they are just johnny-come-latelies. ◔_◔
    – Synetech
    Commented Nov 29, 2013 at 17:25
  • 1
    I just failed one as well that I think could have gone either way. All this has done has frustrate me and make me not want to help review for fear that I'll get banned over another questionable audit. Could I have skipped? Sure... But I don't care that much about badges. The bottom line is this "auditing" "system" cannot be black and white. Period.
    – CDub
    Commented Dec 5, 2013 at 23:35
  • I once failed an audit because I was on a mobile device and tried to scroll, but I pushed the "No Action Needed" button by mistake.
    – NobodyNada
    Commented Apr 26, 2014 at 0:51

5 Answers 5

26

I'm the reason that answer was presented to you, because I'm the mod that processed the flags that collected on it indicating that it's not really an answer to the question.

I actually spent a bit of time on it, and almost edited it, but every edit I could think up left me with something that looked, at least to me like a comment at best. I did agree with the flags, but I generally favor improving terse answers that don't really meet our quality guidelines over just deleting them, if they can be saved.

So, when I elected to just delete it after a few minutes, it was because I felt like it was much more of a 'me too' comment left as an answer than an actual answer.

We have become rather strict with quality related flags for this very reason. When we validate them by taking a destructive action, we know there's a very good chance that they'll be used as a review audit. They need to be cut and dry, or at least darn close to it.

So, in this case, I think I probably made a mistake. I could have validated the flags (I did fully agree with them) and left the answer after at least a minor edit to improve it a bit, which would have kept it out of the audit. The fact that I deliberated for a few minutes means that it wasn't really cut and dry after all, and I will keep this experience in mind going forward (and, well, this is a good example of why we need to be extremely accurate). I'm a seasoned moderator with two years under my belt, if I had a hard time making heads or tails of it, I should have definitely made sure it didn't become an audit.

However, we are human, and we do make mistakes occasionally.

4
  • 16
    "So, in this case, I think I probably made a mistake. I could have validated the flags (I did fully agree with them) and left the answer after at least a minor edit to improve it a bit" I don't agree that you made a mistake deleting the answer; that does not mean that it should be an audit. Maybe a little check-box that you could unclick (less work) to stop something becoming an audit would stop the edge cases getting in there. Commented Feb 6, 2013 at 12:52
  • @benisuǝqbackwards I considered proposing that when I first learned how the system would work. As far as I know, this is the first case of this happening, if it becomes a bigger issue it's definitely something to consider. I'd rather not have more buttons and knobs in our tools, I'd rather it be automatic based on how much time we spend examining something expanded in the dashboard (if you could see the dash, that would make sense).
    – user50049
    Commented Feb 6, 2013 at 13:26
  • No, you are monkey, and I am unicorn. Commented Feb 6, 2013 at 13:30
  • 1
    @BoltClock'saUnicorn I am an ape, actually
    – user50049
    Commented Feb 6, 2013 at 13:31
13

Once upon a time people complained a lot how crappy the new review system is with users abusing it by randomly voting/flagging/approving without paying attention. Thousands of letters have been written in Meta and many days passed.

Then came the review audits. Trivial, simple and easy to detect. The robo reviewers have been finally defeated, and many people were happy again.

Now I can't speak officially and don't have stats at my disposal but common sense tells me that those trivial audits were not enough. Many bad reviews still take place, by users who honestly think they are doing the right thing. So it's time for the audit system to grow up.

From system meant to filter out those who just review blindly and/or randomly just to gain some badges, it grew up to a system that educates the reviewers, teaching them what is a good review and what is a bad review.

In my opinion, it's a blessed and required change if we want the review process to be effective.

Last but not least, regrading the specific audit you linked to: personally I'm not familiar enough with the relevant area of expertise so would have skipped it, but either that answer was wrong (experts in that area should know) thus deserved a downvote or the answer better fit as comment, especially if it repeated something pretty obvious without giving any real help.

The new audits are not unfair, they are harder to detect as audits and look more like the real reviews that's all. Fine by me!

2
  • I failed an audit because it was recommending a system for this question. I clicked through to the question first, but I clicked 'No Action Needed' because I saw it had already been closed. Should I have still flagged the answer nonetheless?
    – icedwater
    Commented Jun 19, 2013 at 7:46
  • 2
    @icedwater yes, "no action needed" means "the post is OK and should not be downvotes/closed in the first place". If you're not sure, or like you said saw it's already closed just click the Skip. Commented Jun 19, 2013 at 7:52
8

Failing an audit is no shame. But you need to learn from the experience.

My review process is continuously changing. Based on the change of the systems, the discussions on meta and an occasional failed audit. The later was a sign that my system was flawed and needed an update.

Right now, I only use the review screen if I'm completely sure. If in doubt, I check the post link to get additional information.

4
  • 4
    "You need to learn from the experience" - I disagree, 8 in 10 review audits are, imo, incorrect or highly discussable. There is nothing to learn from them. Of course, you can always just click skip everytime you're in doubt, but that's not contributing, is it?
    – Emil
    Commented Feb 6, 2013 at 8:32
  • 11
    @Emil: If you think that 8 in 10 review audits are incorrect then you are reviewing incorrectly. There may be a few bad ones but they are very rare.
    – interjay
    Commented Feb 6, 2013 at 11:37
  • There are lots of outliers. It's unpredictable when a post needs to be accepted or not. IMO this question: stackoverflow.com/questions/18357858/… should have been closed. I get these all the time.
    – allprog
    Commented Sep 10, 2013 at 7:39
  • But you need to learn from the experience. Learn what? Sometimes there are no good actions. I had to either skip (which was too late since I had already examined the post and determined it to be just a thank-you), or to do nothing which makes sense because it had already been deleted. The only thing I learned is that the audit system is still too new and has serious defects like this and to stop bothering to do any reviews at all anymore.
    – Synetech
    Commented Nov 29, 2013 at 17:29
6

I disagree. If you don't know if it's a good answer or not, you should click "Skip". By clicking "No Action Needed", you get that review out of the review queue, and thus are not allowing other people who could know if it's a good answer or not to review it.

Of course, this only applies to this audit. Perhaps other audits are incorrect, but I think this one is fair.

2
  • Knowing and feeling are not very far from each other. Especially if you are a human being and not a machine.
    – allprog
    Commented Sep 10, 2013 at 7:43
  • What if you do know it’s a bad answer, but the only options are to skip (which you already did not do by examining it), or do nothing because it has already been processed? It’s a trrapppp!
    – Synetech
    Commented Nov 29, 2013 at 17:26
1

I have to agree to an extent as it seems like some of the audits I have failed just appeared to be the result of the person not being able to speak proper English (which there is plenty of that on these sites). I wish the substance of the audit would be more easily detectable (really obvious vandalizing) vs the way it is now where sometimes I am wondering if the edits are bad simply because of the language barrier.

Not the answer you're looking for? Browse other questions tagged .