9

In the Meta Stack Exchange FAQ of review tests (audits), it is mentioned:

Also, if you find something in the review audit system that makes audits extremely obvious (as in one can tell that a review is an audit in just a few seconds, with little to no extra investigation), feel free to report that flaw.

So, the goal of this post is to report such a flaw.

Screenshot of review test (audit)

This is the audit in question.

In the above screenshot we can notice that the question has 0 answers. So it's pretty obvious to the reviewer that this review is an audit.

If the answer we are looking at is deleted, but the question is shown in its current state, it will cause the answer count not to be the number of other answers +1, as it always should be.

This flaw is immediately obvious if one is looking for it, so it seems highly exploitable.

I asked about this on Ask Ubuntu Meta and one of the moderators in Ask Ubuntu does agree that this is a flaw which needs to be reported.

Improve this question
14
  • 7
    This is by-design. Audits aren't meant to be hard. The fact that you notice the amount of answers on the question is already a sign that you are paying attention. You'd be amazed how many would fail an audit like this, because they aren't paying attention.
    – Luuklag
    Sep 8, 2020 at 10:37
  • 7
    @Luuklag Dedicated robo-reviewers would look for any easy giveaways like this, I think. I doubt it's by design that these numbers get mismatched when the post is deleted. Also, mods can't remove audits...
    – Zanna
    Sep 8, 2020 at 10:40
  • 4
    :p it used to say “other answers: -1”: meta.stackexchange.com/q/165094/323179
    – Laurel
    Sep 8, 2020 at 10:53
  • 2
    I was the one who added that paragraph to the FAQ. The idea was that obvious ways to detect audits, as in those that can be exploited by a robo-reviewer to pass them while continuing their robo-reviewing, are OK to report. I'm not entirely sure that this constitutes such a thing, but I'll let others discuss it. In the meantime, I'll edit that paragraph to more clearly state its purpose.
    – Sonic the Anonymous Hedgehog
    Sep 8, 2020 at 11:28
  • 2
    See This review is obviously an audit. Is this a bug?.
    – Sebastian Simon
    Sep 8, 2020 at 15:01
  • Dedicated robo-reviewers have even easier tells than the number of answers being zero. A quick glance at the network tab reveals so.
    – John Dvorak
    Sep 20, 2020 at 16:49
  • @JohnDvorak What is network tab?
    – Random Person
    Sep 20, 2020 at 16:51
  • network tab of the developer tools in (firefox|chrome)
    – John Dvorak
    Sep 20, 2020 at 17:07
  • Is there a notice when you pass a review? I've done hundreds of reviews now, and I've never seen one of these mysterious audits you guys keep talking about.
    – SurpriseDog
    Sep 21, 2020 at 14:34
  • 1
    @SurpriseDog If we pass a review audit, the website will say you've passed the review audit. Read this post for more information. BTW, review audits are currently active only on Stack Overflow, Server Fault, Super User, Software Engineering, Ask Ubuntu, Mathematics, Stack Overflow em Português, and Stack Overflow en Español.
    – Random Person
    Sep 21, 2020 at 14:39
  • @technastic_tc Why is it only limited to those sites?
    – SurpriseDog
    Sep 21, 2020 at 14:42
  • @SurpriseDog I'm not sure of the reason. But do check this comment: meta.stackexchange.com/posts/comments/861317
    – Random Person
    Sep 21, 2020 at 14:48
  • "the question has 0 answers. So it's pretty obvious to the reviewer that this review is an audit." How so? Some questions remain unanswered without being an audit.
    – Mast
    Oct 8, 2020 at 11:57
  • @Mast I guess you didn't get the point. The review here is about a first answer by a new user. If you can see at the bottom right of the screenshot, it's mentioned that the question has 0 answers. So how come I review an answer when there are no answers for the question? Got my point?
    – Random Person
    Oct 8, 2020 at 12:13

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .