2

I recently underwent this review audit, which consists of random text inserted at random points throughout the post and some minor, unrelated deleted text.

It was pretty easy to detect and flag as spam / vandalism. In my opinion, it was too easy.

Assuming these are generated randomly, I think some improvements could be implemented.

My favorite suggestion is an alteration to the existing insert-only vandalism audits, like the one shared above. Opt to remove original words from either or both ends of a section where spam words are inserted. This simulates a realistic attempt to alter a section of text and may cause someone to be more careful when they review.

A second, lesser, idea would be to generate new vandalism review audits with only random sections of the original text deleted and nothing new inserted. If these sorts exist, I haven't found them.

Improve this question
5
  • 6
    edit reviews are trivially easy if you actually look at the change at all. Sadly, this suffices to find the robo reviewers. We apparently are not seeing many people who pass these audits but do poorly on real edit suggestions. The audit doesn't need to be harder, in other words.
    – Kate Gregory
    Aug 21, 2013 at 15:44
  • 5
    So easy, yet 17 people still failed in the last 24 hours...
    – animuson StaffMod
    Aug 21, 2013 at 15:47
  • 1
    Strongly related, possible duplicate, though it's not tagged feature request: meta.stackexchange.com/questions/171204/…
    – Esoteric Screen Name
    Aug 21, 2013 at 15:55
  • @EsotericScreenName - It's related, but not a duplicate. He's looking for ways to improve it; I suggested some ways to improve it.
    – JoshDM
    Aug 21, 2013 at 16:22
  • @animuson - were the failures due to non-recognition, or due to clicking something that is actually an appropriate action, but the review audit isn't geared to accept as valid? I ask because that's a different issue altogether, and it exists, but it's not related to this.
    – JoshDM
    Aug 21, 2013 at 19:30

1 Answer 1

Reset to default
4

It's worth noting that audits exist just to see if you're paying attention. Which, you apparently were. Robo-reviewers will get snared by the random text, which is all we're really after anyway.

False positives, on the other hand, need to be minimized, and we've been getting a significant number of those lately over technicalities (you can find them here on Meta, if you look).

I failed an audit the other day, because I wanted to look at the actual post before making a decision (it didn't smell right). Munging information for auditing purposes is OK, I guess (even if it causes me to not always believe what I'm seeing in the review queue), but people shouldn't be punished for investigating.

In other words, review audits are just little Turing Tests, and need be no more difficult to solve than a Captcha.

Improve this answer
1
  • 1
    Well, the problem with your audit failure is another issue altogether : that the review audits should provide success on certain other tasks, and not be as strict as they are. I've failed a few for doing things that otherwise should have been deemed as the correct action, and sat through the penalty text for it.
    – JoshDM
    Aug 21, 2013 at 16:26

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .