32

My question pertains to the review audits that occasionally appear when using the "review" feature on Stack Overflow. I know that if you fail too many of these audits, your reviewing privileges can be suspended. This was implemented (according to Manishearth) because:

A lot of people are blindly upvoting everything in the queue without even reading the posts, just so that they can get the badges. So, some posts in the review queue are designed to catch such users...

However, I think there is a loophole in this system. A user can conceivably pass every audit without reading the post by using the following method:

  1. Immediately select "flag" or "Recommend Deletion" without reading the post.
  2. If the question/answer being reviewed was an audit, then the results will automatically appear when the user flags the post. Since (as far as I know) the audits are always negative tests; i.e. they are posts that should be flagged or deleted and not posts that should be approved, if the audit results do not automatically appear, it means the post is not an audit.
  3. After determining if the post is an audit, the user can ultimately choose not to flag/recommend the post for deletion by Xing out the screen that asks for the reason the user wants to flag/recommend for deletion. They can then approve the post without reading it and without running the risk of failing an audit.

Especially since posts can be flagged within the ~3 seconds during which the system does not allow the user to click "No Action Necessary," this is a simple method to pass every audit while wasting no time in moving through review posts. Hence users can continue to blindly allow posts or edits that should be flagged/deleted/undone to earn badges, to the detriment of the Stack Overflow community.

My question: Wouldn't it be better to force the person to confirm the flag or deletion recommendation before displaying the audit results?

3
  • 9
    or even adding "positive audits" so if the person clicks flag on a legit post she fails Commented Feb 24, 2013 at 16:55
  • There are positive audits on the close queues (I've personally had one), where a perfectly good question came up and you were expected to "leave open". Commented Feb 24, 2013 at 17:31
  • @MadaraUchiha well that is probably the only queue with positive audits as I've never seen one in the "first post" nor "late answers" queue, and I think there could be gains with positive audits in these queue. As long are they are not too on the edge. Commented Feb 24, 2013 at 17:44

2 Answers 2

26

This was a major annoyance as far as I'm concerned. The close and flag dialogues have finally made their review audit debut in build rev 2014.5.15.2250 on meta and 2014.5.15.1608 on sites. In order to fail or pass an audit you now have to actually submit a flag or close vote. Await some robo-reviewer lash back.

1
  • 6
    Cue the complaints from possibly-genuine reviewers that "my two extra clicks were used for an audit!! I want my two seconds back!" Commented May 15, 2014 at 14:58
19

Immediately select "flag" or "Recommend Deletion" before reading the post.

If you do actually read the post after that, I don't care if you "cheat". The audits are, first and foremost, meant to be educational - "stop - you weren't paying attention, this is what you missed"; failing an audit means nothing compared to missing an opportunity to catch a real problem.

The folks completing these things in three seconds with the same action chosen every time... If they're cheating, they're cheating themselves, throwing away an opportunity to become a helpful member of the community in exchange for... a brief spell on the leaderboard.

They're also pretty easy to detect, with or without audits.

3
  • 4
    This is exactly what I was thinking... If you have the presence of mind to realize that you don't have enough information to make a decision (some details are hidden in an audit) and go to the actual post (or anywhere else) to get more clarification, mission accomplished.
    – user102937
    Commented Feb 24, 2013 at 17:44
  • But there is also the potential for people to not read the posts after hitting the "flag" button. If they hit "flag" and it does not show them the results of the test, they can simply select "no action necessary" and continue to the next review. Of course they are doing a disservice to the community by not helpfully participating; but isn't this one of the reasons the audits/suspensions are implemented to begin with? It seems like an easy way to rapidly "review" posts while circumventing the system that was put place to prevent such mindless reviews from occurring.
    – JSW189
    Commented Feb 24, 2013 at 20:56
  • 4
    The basic assumption here @JSW189 is that most reviewers want to help, but some of them are just... really bad at it. There's also the danger of fatigue causing mistakes. Actively malicious reviewers are another story entirely.
    – Shog9
    Commented Feb 24, 2013 at 21:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .