Let me start out by saying that I do my best to give every question due diligence in the Close Votes queue. However, every now and then, Stack Overflow likes to throw in a gotcha question to make sure that I'm not just blindly clicking through the edit queue trying to get a badge. I don't know if there is any kind of consequence for getting these wrong other than it displaying a demeaning message that pretty much says to the affect of "Hey idiot, you're not paying enough attention." Actually, I was giving all of my attention.

I ran into one gotcha just a few moments ago that was a real head scratcher when I got it "wrong". This isn't the first time I've honestly voted to close a question that I thought should be closed, only to find out that I was "wrong" for trying to close it.

I took a screenshot of it as I couldn't think of any better way to show what I'm talking about.

enter image description here

Here's what I noticed when reviewing this question:

  • The author says he is trying to follow some directions on an installation site, and he encountered an error after he entered some command.
  • He says he's not sure. (very informative)
  • He posts a log of his bash console, which includes some error messages.
  • He provides absolutely zero dialogue about what he's tried to remedy the error.
  • After reviewing the audit (thanks @rene for the link), I noticed he does list some vague, limited dialog in the comments. That brings for me up questions about whether comments should be included in the audit process.

Now, maybe I'm wrong here, but as an objective observer, this seems like a very low quality question. The author didn't provide much information about what he'd done prior to this. He offers little explanation about what he's tried to resolve the error. He didn't give much information about his system setup.

Was I truly wrong in voting to close this question? On one hand, the error message is right there on the screen. On the other hand, the author has left us with little information other than "here's my problem. fix it for me".

Some gotchas are just blatantly wrong. I come across them sometimes where words have just been substituted in random places that make no sense. However, ones like this seem too subjective. Looking at this question, I'd vote to close it every time because I feel that it's simply low quality.

What is the general consensus on this? Is there a penalty for getting these wrong?

  • Related: What are review tests audits and how do they work?
    – jscs
    Commented Jan 24, 2014 at 20:41
  • 4
    I have to agree with @CodesInChaos when he said I'd like a "I still disagree, your test if flawed" button for "failed" tests. That's how I feel right now.
    – crush
    Commented Jan 24, 2014 at 20:44
  • Yeah, there's an actual feature request for that somewhere that I was trying to find.
    – jscs
    Commented Jan 24, 2014 at 20:51
  • This is the actual audit
    – rene
    Commented Jan 24, 2014 at 21:07
  • @rene Thanks for tracking that down.
    – crush
    Commented Jan 24, 2014 at 21:09
  • 2
    Here's a question I just got in the review queue: stackoverflow.com/questions/1840231/…. It has 54 up votes, but seems to clearly violate the FAQ. How do I vote on it? I'm going to vote to skip it.
    – crush
    Commented Jan 24, 2014 at 21:19
  • 1
    If you ever see a review and you're on the fence about it, you can click through to the question, and if you see it actually has no close votes, you know it was an audit and you should click "leave open." The audit questions always have 0 votes to hide that the question actually does have upvotes. Commented Jan 24, 2014 at 21:45
  • Doing this is not really getting around the system, because you still are slowing down and taking the time to evaluate each question. Commented Jan 24, 2014 at 21:59

1 Answer 1


A couple of thoughts on review audits

  1. If you fail one innocently, it won't count against you. The users that get hit for audit failures are the ones who rarely pass an audit, because they are robo-reviewing.

  2. If you have a doubt (i.e. the question appears to have been sanitized to turn it into a review question, or you're not sure why it's in the queue), open the question link directly in a new window, so you can see it in its original form. Robo-reviewers won't bother doing this.

It's not a perfect system. I'm a mod, and I still fail a review from time to time (although I'm getting better at spotting them). It's not an exam; the point of the occasional audit is to see if you're paying attention.

  • 2
    Thanks for that. I feel better knowing that I'm not being penalized for the occasional miss. It did damage my self-esteem a little bit, but I'm over it now... I just got this question in a review queue. It has 54 up-votes, 58 favorites, and an accepted answer with 47 up-votes. However, it seems to clearly violate the FAQ because it is requesting an off-site resource. It has a single close vote. How do I vote on this? I chose to skip it...
    – crush
    Commented Jan 24, 2014 at 21:22
  • That was a good choice. That particular question gets a pass because it's an interesting one, but I just noticed that it attracted a spam answer from a 1 rep user, so I'm going to protect it.
    – user102937
    Commented Jan 24, 2014 at 21:24
  • 1
    gee Robert this "interesting" question is a blatant shopping followed by 10 meh answers, 9 of which are link-only crap... I'm out of votes today but tomorrow I expect to lose 9 precious rep points voting on this "known good" stuff
    – gnat
    Commented Jan 24, 2014 at 21:39
  • @gnat: You can't always get what you want. ♫
    – user102937
    Commented Jan 24, 2014 at 23:03
  • 2
    @RobertHarvey Wouldn't it make more sense to apply one of those special close reasons that says "this question is interesting but not actually on-topic, so we're not deleting it but don't take it to mean you can make new questions that look like this"?
    – neminem
    Commented Jan 25, 2014 at 0:11

Not the answer you're looking for? Browse other questions tagged .