What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

I just failed this audit here, which was shown as being closed as "Not a real question".

Seeing as the vote count was positive, I figured it was an audit, but I selected "Leave Closed" anyways because to me

It's difficult to tell what is being asked here. This question is ambiguous, vague...

There's no code examples, and the phrasing is clunky at best.

This is my second failed audit in two days for difficult questions (first failed audit is here, also now closed), so if I get temporarily suspended, it won't be the end of the world (I'll think of it as a forced vacation).

That said, is there a way to dispute these audits? Or is posting the audit here sufficient?

share|improve this question
That question has never been closed up to now, so something's wonky here. (It should be closed very shortly, however, because it's awful.) –  Josh Caswell Feb 20 '13 at 18:21
That question looks off-topic to me. –  Aaron Kurtzhals Feb 20 '13 at 18:23
I really hate that feeling when you know you're on an audit, know the answer it is expecting, and know that it's wrong. It puts you in the position of either doing the wrong action to pass the audit or doing what's right and failing. –  Servy Feb 20 '13 at 18:24
Jeez. Audits are getting worse. –  Emil Feb 20 '13 at 18:24
@Servy What about skipping? –  ughoavgfhw Feb 20 '13 at 18:25
@JoshCaswell - yeah, I was really surprised that question had a vote count of +6 when I reviewed it. And now it's closed for real :) –  LittleBobbyTables Feb 20 '13 at 18:27
@ughoavgfhw That's probably the best of the possible options, but it still shouldn't be happening to begin with. Automatically generated audits are just inherently broken since the mechanisms the system relies on aren't 100% perfect, or aren't very clear. They should honestly all be fabricated by trustworthy people (i.e. mods) so that the "correct" answer is actually correct, and so that ambiguous cases are avoided entirely. –  Servy Feb 20 '13 at 18:27
I was temporarily suspended after my last meta audit related question ;). A lot of my failures ended up as bug/support questions on meta... It's been a little weird bit being able to click review constantly but it's given me more spare time :). –  ben is uǝq backwards Feb 20 '13 at 18:35
I've asked for a button "suggest as honeypot" to be available to "good reviewers" - it's kind of buried right now in a question about there not being edit audits - should I ask it again as a feature request? –  Kate Gregory Feb 20 '13 at 18:43
The question had 6 upvotes, so it must be good! A similar problem reported here How do we select audits for Reopen review?. –  Bo Persson Feb 20 '13 at 18:46
@KateGregory: That's a good suggestion. I suspect SE is trying to come up with an algorithm to automate this, though. The problem with automation in general is that a machine can't divine meaning, and therefore might not be the best arbiter of human judgement. –  Robert Harvey Feb 20 '13 at 18:54
@KateGregory - by all means; in fact, I think I upvoted you last time as well. –  LittleBobbyTables Feb 20 '13 at 18:58
This is slightly off-topic, but the edit performed on the question in question is completely and utterly idiotic. What's the point of adding all those line breaks? Argh! –  J. Steen Feb 20 '13 at 19:10
meta.stackexchange.com/questions/168374/… is the feature request –  Kate Gregory Feb 20 '13 at 19:26
Press reopen to pass the audit then go to the question and vote to close –  SztupY Mar 11 '13 at 11:06

1 Answer 1

up vote 11 down vote accepted

Posting here is sufficient.

Right now, these audit questions are selected using the following criteria:

  • Recently asked
  • Score between 5 and 15 (inclusive)
  • Never locked, migrated, or deleted
  • No close votes or downvotes, ever (only for close/reopen audits)
  • (On Stack Overflow) at least 100 views

The biggest weakness here is the Score criteria. If you ask a question and five of your friends up-vote it, you're in the running - not saying that's what happened here, but it's definitely been a problem.

While Kate's suggestion for hand-picked audits would be nice, that really can't scale; we need a lot of these generated on a regular basis.

share|improve this answer
Two questions spring to mind: (1) If I come across a question like this, and I skip the audit, will anyone else get it? (2) If the answer is "Yes" to question 1, what happens if I downvote and/or vote-to-close the question? Will someone else still get it, or will the negative votes remove it as an audit? –  LittleBobbyTables Feb 20 '13 at 19:25
Why wouldn't you pick questions that had actually been reopened to audit reopen reviews? Not enough of those? –  Josh Caswell Feb 20 '13 at 19:27
That should disqualify it from being used in future audits, @LittleBobbyTables –  Shogging through the snow Feb 20 '13 at 19:46
@Josh: well, the intention here (and with most audits) is to pick "known good" posts - examples that are clearly good or bad. If a post has been re-opened, that still means at least someone thought it should've been closed at one time. Picking closed->edited->reopened posts might improve this, but... –  Shogging through the snow Feb 20 '13 at 19:48
@Shog9 is number of question views taken into account in selection? –  gnat Feb 21 '13 at 5:56
@gnat: no; that's not a bad idea - there have been a small number of audits with < 60 views that've had more than their fair share of failures. Above that, it doesn't look like it'd make much difference. –  Shogging through the snow Feb 23 '13 at 23:52
Would choosing questions that have had close votes, not been closed, and actually been judged as suitable to leave open be better candidates for this? Then we know that 'critcal eyes' have passed over the question. Relying primarily on votes is bound to lead to some - errr - questionable items being used as audit fodder. –  martin clayton Mar 7 '13 at 23:18
@gnat: FWIW, we implemented your suggestion and now require at least 100 views on a question in order for it to be considered for a "known good" close/reopen audit on Stack Overflow. I'm reasonably happy with the results of this so far. –  Shogging through the snow Apr 24 '13 at 14:53
@Shog9 I see. 100 views make good sense at SO (and probably at SF/SU as well). What amount views would you consider for smaller sites? With less traffic there, amount of "audit item candidates" with 100+ views might be quite small. –  gnat Apr 28 '13 at 11:27
The pools are already very small on sites that aren't SO (to the extent enabling audits at all on most smaller sites isn't really feasible). Should be fewer weird outliers though too. @gnat –  Shogging through the snow Apr 29 '13 at 2:12
I'd like an "I disagree with this ruling" button... but I can see that being abused rather swiftly ;) –  Taryn East Jun 25 '13 at 7:13
Either your answer or animuson's is incorrect: "t's a bad audit. There were three close votes for that off-topic reason that had previously expired on that question. It just didn't get enough attention." –  РСТȢѸФХѾЦЧШЩЪЫЬѢѤЮѦѪѨѬѠѺѮѰѲѴ Feb 25 at 16:29
Different queue, different rules for audits, @Łukasz웃Lツ –  Shogging through the snow Feb 25 at 16:35
Only for close/reopen audits, @gnat. –  Shogging through the snow Jun 24 at 14:06
Seems to be an issue with caching, @gnat - looking into this. –  Shogging through the snow Aug 20 at 22:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .