What an audit!? Would you fail?

Question

Came across this answer in the review queue (the answer has since been edited by Gilles, here's the version I reviewed).

Scanning through comments quickly I see that people are saying: I do not understand this, I do not understand that, Can you explain this and that, etc...

I wanted to flag this answer for either low quality or moderators attention. I could have edited it through creating the body based on the comments - BUT, I am not an expert in Haskell (I don't even know the basics) and didn't want to mess anything up or provide a misleading information through my edit.

While my intentions are good and helpful I am failing this audit. How's that for a reward to the effort I make?

Additionally,

This was an audit, designed to see if you were paying attention. You didn't pass. Your review was inappropriate. This was a high quality post and you should have considered leaving it as-is or even upvoting.

what would a low quality answer look like? an empty post? ;)

• Am I missing something here? How did I not pay attention?
• What would the right thing to do be in this case?
• What would you do and why would it be different than the path I took?
• Would you fail this?

Answer 1

I don't think you should have failed this audit, but not necessarily for the reasons you think. Yes, it appears to be a bad audit question¹, and had I decided to review it, I probably would have failed. But since I am not familiar with haskell I probably would have skipped it. Personally, I don't think you should have failed it because I don't think you should have reviewed it.

I've said it before on other answers about audits in other queues and I'll say it again in a first posts queue, when reviewing, especially when reviewing answers, if you aren't familiar with the topic, you should skip.

From one of your comments (emphasis mine):

@GeorgeDuckett yes, the question is faded out at the bottom of the page. But you are asked to review the answer not the question. In some cases I would read the question but in this case the given answer is too short, low quality and doesn't explain any code given.

If you admit you review answers without even reading the question, then you need to rethink your review process. Sure, you can judge that post on the apparent quality and conformance to the guidelines, but you are not performing half of the review. How do you even know that the answer, even a high quality one, answers the question unless you look and understand the question? (In fairness, you do say you don't always read the question so it is not like you skip reading the question every time).

One effective way to make sure you only review posts you feel qualified to edit is to filter. Depending on the queue, the filter link has multiple options (to review questions only or answers only in the First Posts, or by type of close vote in the Clove Queue). There is also a tag box in all of the queues so you can specify the tags you want to see.

^{1 - My position on this being a bad audit question is that the original version shown before Gilles' edits was not an ideal Stack Overflow post. Yes, this was an OK answer that was made exceptional by the aforementioned edit, but since audits are supposed to be more obvious and given the lack of a body in the original, I don't see it being a slam dunk. That being said, it is great for educational purposes, so if they audits are every changed to being more educational then it would have been a good audit.}

---

TL;DR;

Yes, as someone who is unfamiliar with the technology as well, it appears to be a bad audit, but since you aren't familiar, you should have skipped.

When answering questions, it takes 5 extra seconds to review the question and the tags. That doesn't mean you need to take the question into account if it is clearly a bad (non-conforming) answer, but you also don't need to rush through the review either - that is the whole purpose of audits, to make you slow down and pay attention. If you learn to slow down and review everything as a result of this audit, then it actually did its job.

Answer 2

This is a code-only answer. A code-only answer should not be deleted on sight. To call it a high-quality answer is be stretching it, but it is not so low-quality that it needs to be deleted.

An answer should be deleted if it does not contain any content to someone who may be looking for the answer. A code-only answer may be deletion-worthy in some circumstances — for example, if the same code is contained in another answer with explanations, or if the code is completely irrelevant (and even in that last cast the normal reaction should be a downvote). But if you do not have enough technical knowledge to ascertain that the code is useless, then you should skip that review.

The comments should have been another clue that the answer is worth keeping. Yes, there are comments that say “I don't understand this” — and there are replies to these comments! This is an indication that the right thing to do with this answer is to edit it to merge in the information located in the comments. Also, the first comment, which has several upvotes, begins with “last ['a'..] is clever” — if it's clever, surely it's worth having around.

You even seem to have realized that the answer needs to be edited. If an answer needs to be edited, it means it's worth keeping. The definition of a low-quality answer is that it is unsalvageable and needs to be deleted. An answer that needs editing is not a low-quality answer and doesn't need flagging.

If you don't have the technical knowledge to edit the answer, that's fine. Skip the review item.

---

I've gone ahead and edited the answer, I think I've added in all the relevant information from the comments.

Answer 3

(Correct me if I misunderstood the source. But I understand Tim Post is involved.)

The highlighted quote apparently is not a comment from an individual contributor but official StackOverflow prose. That's most worrying. Under no condition should you use such language towards volunteers. I would strongly suggest reprimanding whoever wrote that.

In particular, it consists of 5 accusations in a row, with the same message. That form of repetition confers the message that the author believes the reader to be so dumb that repeating it 3 times would be insufficient, a message that's reinforced by the use of short phrases such as "You didn't pass".

Now, back to the questions at the end. It's unnecessary to answer them individually. What would I do? Well, I would drop out of reviewing and go back to answering. Strike that, I will. I haven't even gotten this message personally, but its mere existence is insulting.

[edit] To opt out of this reviewing, you can use the following bit of custom CSS:

span#hlinks-nav {
  visibility: hidden !important;
}

Answer 4

This post helped me understand where I went wrong on this audit.

• Am I missing something here? How did I not pay attention?

Maybe lack of the below points was the reason for my original decision. I think I have misjudged the usefulness of the answer even though it was a low-quality one.

You should Delete a post

• [ANS] If the answer asks another question
• [ANS] If an answer is actually a comment (such as asking for more info, thanks!, etc.) [possibly flag]
• [ANS] If the answer is incomprehensible and you are at least a bit familiar with the subject.
• [ANS] If the answer duplicates another, better answer to the same question (e.g. a link-only answer where another answer provides the same link with an explanation).
• [ANS] Is obvious spam (tries to sell you cheap sunglasses) [possibly flag].
• [ANS] Note that wrong answers should not be deleted: deletion is for answers that “aren't even wrong”.

The answer is a low quality one, and there is an extended discussion going on in the comments section about the answer, but it is NOT one to delete according to the above points. It doesn't ask another question, not a comment, not a duplicate, not a spam.

What is faulty here - it's the option given to me. It just doesn't feel right hitting Looks good! while there are visible problems - ie. Low quality question.

What got me really confused is the fact that you cannot flag an answer for being a low quality one.

So an audit asking me if the answer is a low-quality one automatically got me thinking - YES, this is a low quality post without even thinking BUT IS IT A LEGITIMATE ANSWER AND SHOULD IT BE DELETED?.

I wish I had a It's a low quality question but don't delete it button in this case. It could be the correct action to take to actually get through the audit.

I hope you agree and see the lack of a low quality but don't delete it button in this particular case.

• What would the right thing to do be in this case?

recommending deletion would have been inappropriate in this case. The answer does not fall under any of the above quoted points therefore is a legitimate answer.

skip is what I probably should have clicked - but then maybe the next and next, and next person would have the same problem as me trying to judge if the answer was specifically a low-quality one or not. The question was: "is this a low quality post?" and the answer apparently was "NO".

looks good - thought it would fail me as the answer only provided 4 lines of code with no explanation, however this would have been one of the two appropriate actions to take. It makes no sense to me to give a looks good opinion on an answer with 4 lines of code and no explanation even if I am not familiar with the topic. It's just general idea of answering.

Edit - this is another good solution to pass the audit. Just edit it and create the body for the answer based on the comments posted. I would only do that if the tags and the topic was familiar to me.

• What would you do and why would it be different than the path I took?

• Would you fail this?

Honestly, if I could have a go again at this I would just SKIP it. ( which is what most of you suggested in your comments and answers ) - thanks for your input.

Conclusion:

Well, I do get more familiar with the website on daily basis, I contribute by giving answers and helping with moderation through reviews and edits, but at this point I will rather try to avoid the review queue until it gets improved.

Answer 5

tl;dr it was right that you failed this audit.

You failed the audit, and rightly so. It was a question and answer where you have, by your own admission, insufficient knowledge.

The right thing to do in those circumstances, whether it's an audit or a genuine review, is to skip. As soon as you realise you are not fully qualified to assess the post, you hit skip and move on. Any other action is incorrect, and should result in a failed audit. Because in reality for a real (non-audit) review, any other action you would take is adding noise to the signal - you'd be making things worse, not better. Such actions are worse than worthless, they give a net reduction to the value of the site.

The audits are there to help us to learn how to review well.

The lesson from this audit should be very clear. As soon as you realise you are not fully qualified to assess the post, you hit skip and move on.

Answer 6

I don't think this should be an audit answer.

Audits should mostly be vandalism (when combining C and JS is common), and clear-cut to those who are actually reading the question/answer.

For this answer, if it wasn't already high-voted, it could be go either way. This is the type of answer that is often found in the low-quality posts queue for being short and having no explanation.

I would have reviewed it as "Skip," based on having no knowledge of haskell.