9

In the Meta Stack Exchange FAQ of review tests (audits), it is mentioned:

Also, if you find something in the review audit system that makes audits extremely obvious (as in one can tell that a review is an audit in just a few seconds, with little to no extra investigation), feel free to report that flaw.

So, the goal of this post is to report such a flaw.

Screenshot of review test (audit)

This is the audit in question.

In the above screenshot we can notice that the question has 0 answers. So it's pretty obvious to the reviewer that this review is an audit.

If the answer we are looking at is deleted, but the question is shown in its current state, it will cause the answer count not to be the number of other answers +1, as it always should be.

This flaw is immediately obvious if one is looking for it, so it seems highly exploitable.

I asked about this on Ask Ubuntu Meta and one of the moderators in Ask Ubuntu does agree that this is a flaw which needs to be reported.

14
  • 7
    This is by-design. Audits aren't meant to be hard. The fact that you notice the amount of answers on the question is already a sign that you are paying attention. You'd be amazed how many would fail an audit like this, because they aren't paying attention. – Luuklag Sep 8 '20 at 10:37
  • 7
    @Luuklag Dedicated robo-reviewers would look for any easy giveaways like this, I think. I doubt it's by design that these numbers get mismatched when the post is deleted. Also, mods can't remove audits... – Zanna Sep 8 '20 at 10:40
  • 4
    :p it used to say “other answers: -1”: meta.stackexchange.com/q/165094/323179 – Laurel Sep 8 '20 at 10:53
  • 2
    I was the one who added that paragraph to the FAQ. The idea was that obvious ways to detect audits, as in those that can be exploited by a robo-reviewer to pass them while continuing their robo-reviewing, are OK to report. I'm not entirely sure that this constitutes such a thing, but I'll let others discuss it. In the meantime, I'll edit that paragraph to more clearly state its purpose. – Sonic the Anonymous Hedgehog Sep 8 '20 at 11:28
  • 2
  • Dedicated robo-reviewers have even easier tells than the number of answers being zero. A quick glance at the network tab reveals so. – John Dvorak Sep 20 '20 at 16:49
  • @JohnDvorak What is network tab? – Random Person Sep 20 '20 at 16:51
  • network tab of the developer tools in (firefox|chrome) – John Dvorak Sep 20 '20 at 17:07
  • Is there a notice when you pass a review? I've done hundreds of reviews now, and I've never seen one of these mysterious audits you guys keep talking about. – SurpriseDog Sep 21 '20 at 14:34
  • 1
    @SurpriseDog If we pass a review audit, the website will say you've passed the review audit. Read this post for more information. BTW, review audits are currently active only on Stack Overflow, Server Fault, Super User, Software Engineering, Ask Ubuntu, Mathematics, Stack Overflow em Português, and Stack Overflow en Español. – Random Person Sep 21 '20 at 14:39
  • @technastic_tc Why is it only limited to those sites? – SurpriseDog Sep 21 '20 at 14:42
  • @SurpriseDog I'm not sure of the reason. But do check this comment: meta.stackexchange.com/posts/comments/861317 – Random Person Sep 21 '20 at 14:48
  • "the question has 0 answers. So it's pretty obvious to the reviewer that this review is an audit." How so? Some questions remain unanswered without being an audit. – Mast Oct 8 '20 at 11:57
  • @Mast I guess you didn't get the point. The review here is about a first answer by a new user. If you can see at the bottom right of the screenshot, it's mentioned that the question has 0 answers. So how come I review an answer when there are no answers for the question? Got my point? – Random Person Oct 8 '20 at 12:13

You must log in to answer this question.

Browse other questions tagged .