At first, this may look like a duplicate, but the other one was downvoted, it seems, because people disagree for my reasoning on why I failed. Please keep your votes centered around the request.

I just failed an audit on a question that should be closed. Why? Because it has 6 upvotes and no close votes:


Look at the question. It's obvious it's one that the community likes, but is that a valid reason to put it as an audit? Maybe not, but because it has 6 upvotes and no close votes it should be put in the queue!? NO.

This question is obviously opinion based, and something needs to be done about it. What I've been thinking is something along the lines of a audit invalidator.

We would need to start by showing multiple people the same audit. We then compare their results, and decide to invalidate an audit if enough people fail it. Now, I know you people like numbers (and unicorns), so here is an example scenario:

Let's say we show an audit to 6 people. 5 fail, but only 1 passed. It's pretty obvious it was a bad audit. What do we do? We invalidate the audit. Ultimately, it is still counted as a review, but it works like this: The people that failed lose that "mark" in the system, while the person that "passed" has no effect on him.

I know what you're asking: What do we do with the people who "pass" a lot of bad audits? Simple. Remember that person who "passed" in the last paragraph? Yeah, that one. We show him the audit again. If he chooses what would have been a "fail" in the first one (which is now a pass), it appears he's learned his lesson. So we let him go. If he fails this (now reversed) audit, we warn him that he should read the messages, then disable the "I understand" button for a longer period of time.

Now, the second part of this proposal (showing the audit another time) is just me brainstorming. Nothing required. However, the first part (invalidating audits), is the main part of this proposal. Please don't downvote if you disagree with the second part. Please focus your votes on the main part - the first part.

  • 2
    Is it time to rethink how you vote to close? Aug 5, 2013 at 10:44

1 Answer 1


You keep making the same mistake with these proposals: you fail an audit and assume you did so because the review audit selection process is flawed... Then you build your case for changing it on this assumption.

Now, there are most certainly flaws in the review audit selection process. But picking questions you think should be closed doesn't quite capture their essential nature; after all, if no one failed them, they'd be useless! I'll elaborate a bit more on this later, but for now let's look at your rationale for this specific question:

This question is obviously opinion based, and something needs to be done about it.

Why? Why does anything need to be done about it? A huge number of questions on SO are "opinion-based" in some fashion; that's not usually a problem. When it is a problem, closing isn't necessarily even the best option.

Let's refer to the help section for the "primarily opinion-based" close reason:

primarily opinion-based - discussions focused on diverse opinions are great, but they just don't fit our format well.

Many good questions generate some degree of opinion based on expert experience, but answers to this question will tend to be almost entirely based on opinions, rather than on facts, references, or specific expertise.

Ok; this leads off with a link to a list of specific patterns to be avoided (none of which apply to the question you saw), but we don't even have to go that far - the close reason's description itself contains a simple test that "C# Event handling comparison" fails: "answers to this question will tend to be almost entirely based on opinions".

See, this isn't Tabs vs. Spaces - there are objective differences between the two approaches he outlines. The two approaches can potentially behave differently - therefore, an answer need only outline the scenarios and behaviors in which they will differ; no opinions necessary.

Something must be done: edits

That's not to say there weren't a couple of issues with that question: the code had some formatting problems, and the title wasn't particularly descriptive. I've edited to address both; you could have done the same. However, neither problem was severe enough to warrant closing the question.

Actual problems with the close/reopen audit selection

This question actually makes for a great audit, since it's neither particularly flashy or awful. It's the sort of meat & potatoes question that you'll find all over the place on SO - and therefore, the sort of thing you should learn to recognize when reviewing questions in these queues, since folks who aren't reading carefully might inadvertently vote to close them based on a couple of keywords.

Unfortunately, the current audit selection criteria don't guarantee these sorts of questions. You can also end up with stuff like this or this, topics which somewhat niche, are reasonably popular in their niche but are also off-topic as-asked. We've made a few tweaks to minimize this, but haven't prevented it entirely.

  • I've seen MANY comparison questions closed. Here's one. Here's another
    – Cole Tobin
    Aug 5, 2013 at 2:36
  • Explain how those two examples are anything like the one you used above?
    – Shog9
    Aug 5, 2013 at 2:38
  • They are comparison questions. Comparison questions asking which is better. To me, they are no different than asking if someone should use Int32 or int in C#. Feel free to enlighten me. In addition, I specifically asked that the focus of this is the feature request. I didn't ask why I failed. I was providing background. I didn't even add the support tag. I only added feature-request.
    – Cole Tobin
    Aug 5, 2013 at 2:44
  • 3
    You keep typing "comparison question" as though that should mean something. Either you can explain what was wrong with the question you were tested with, or you're in no position to argue for changes to the tests.
    – Shog9
    Aug 5, 2013 at 2:48
  • 1
    Comparison questions are typically opinion based. Ok. The holy war. Tabs vs spaces? Which is better? That's a comparison. I'm asking which is better. That's opinion based. By the logic of closing that as opinion based, it seems opinion based means open ended comparison questions. How is asking for the pros and cons of a question any different!? Questions Asking for the pros and cons have been closed before. That's what those two questions are. Asking for the pros and cons. Disregarding that, ...
    – Cole Tobin
    Aug 5, 2013 at 3:36
  • 2
    ... the question I failed an audit on is an open ended comparison. The OP is asking which he should use. The answer depends on his needs. Asking JS vs jQuery, the obvious answer is jQuery. However, the question will still be closed as opinion based! It's happened before (Under 10k, so I can't find any)! If opinion based doesn't mean the same as asking which is better (or for pros and cons), then the new close reasons are horribly written and need to be replaced, again. Find me a dictionary or ...
    – Cole Tobin
    Aug 5, 2013 at 3:37
  • ... any (reputable) source (besides Meta) that says asking for the pros and cons is different than being an open ended comparison question and I will reward you a 500 rep bounty.
    – Cole Tobin
    Aug 5, 2013 at 3:38
  • 3
    There's an entire blog post about "versus" questions, @Cole. The key take-away? State the actual problem you're trying to solve.
    – Shog9
    Aug 5, 2013 at 4:33
  • That still doesn't state the (non-existent) difference I mentioned.
    – Cole Tobin
    Aug 5, 2013 at 5:08
  • In the meantime, please enlighten me on whether I should use ImageMagick or OpenCV
    – Cole Tobin
    Aug 5, 2013 at 5:12
  • @Cole: you're trying to distinguish between a question on the best invisible indentation character for every situation ever, and a question on handling UI updates from asynchronous data events?
    – Shog9
    Aug 5, 2013 at 5:12
  • 1
    Heh... That one is Gorilla vs. Shark.
    – Shog9
    Aug 5, 2013 at 5:13
  • You know what, forget this. I give up. The new close reasons may work, but they are crap.
    – Cole Tobin
    Aug 5, 2013 at 5:14
  • 2
    No, I don't particularly care to spend three hours answering your question and then delete it because you got tired of it.
    – Shog9
    Aug 5, 2013 at 19:21

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .