I have been surfing meta for a few weeks now and I am seeing a large number of reports about failed audits and review bans. Obviously if someone is getting banned they are going get on meta and vent hoping they can get back to reviewing. I, too, have been in this boat. There are obvious flaws in the honeypot system as you can see by all the posts on meta. But I wanted to break down some of the issues I have seen and offer a couple suggestions:

Issues with the current audit system

  • It is an automated honeypot system. This means that if you have different view points for verifying if a post is valid than the rest of the users you are assumed to be wrong. This is great for black and white post where it is clear that a user is spamming or wrote a two page answer with references, but it is less helpful for gray-area posts. If people can fall on either side of an issue, the honeypot can trick you.
  • Audit bans are exponential. Meaning if you fail audits and get banned you are banned for longer and longer periods of time. This is obviously a good system for spammers, but what if you started reviewing before you understood the system and you accrued a few failed audits. You could then pass the next 100 audits and fail one, and you are banned for a month. I know this, it has happened to me.
  • Commenting and editing are considered to be inherently bad. So if you go to comment on a post you think is good, you will get a failed audit.

These are just a couple of the issues I have spotted, but there are surely others.

Here are my suggestions:

  • Review audit atonement - If you get a bad audit, don't just incrementing a failed audit count on your account, make the reviews a ratio. If a person has a pass:fail of 20:10 then their failed audit ratio would be .5. The more passes they have, the lower their fail ratio will be. This would allow people to atone for bad audits. You could then determine at what ratio a person is considered to be doing more harm than good. You could even build a tracking system to see if people have been at a high fail ratio for a long period of time. You could then weight failed audits higher for that person.
  • Ban review queue - If a person is banned for failed audits, allow them to have their ban reviewed by their peers. List out the failed audits and let others decide whether or not a ban was warranted. Additionally, you could allow the user to comment on their ban - this could avoid the legions of posts on Meta about audit bans. You could also allow peers to determine if the length of the ban is justified (I envision a "shorter ban length", "longer ban length" and "this ban is justified" buttons). If someone is banned for a month because they keep commenting on good posts, they obviously shouldn't be. Which brings me to my next point...
  • Adjust commenting and editing - The current functionality is flawed. There is no reason that a user should be penalized for commenting or editing. I see two possible solutions. Either allow a user to comment/edit without incrementing their review count like a standard audit OR treat comments/edits as a "skip this review" and simply redirect them to the next audit like you would a failed review. Perhaps even mention that "other users found this post to be satisfactory without the need of edits or comments".

I realize that there are already tons of posts about audits on Meta. However, that is exactly one of the issues my question is trying to resolve. I ask that you read my question in its entirety before plastering duplicates all over the place.

Please let me know what you think! Thanks folks!

  • 6
    "Eventually, you will be banned forever." - I'm pretty sure it caps out at 30 days, which is also the maximum a moderator can manually suspend from review for. -- Also, why did you tag this with suggested-edits? The Suggested Edits review audits are by far the easiest audits to pass because of how obnoxiously bad they are... – animuson ModStaff Sep 18 '13 at 18:00
  • I thought this too, however the team at SO can ban your IP if you are found to be spamming posts, comments or reviews: meta.stackexchange.com/questions/168980/… – Matthew R. Sep 18 '13 at 18:03
  • 3
    How is that related to review audits though? That is very much a manual process reserved for very extreme cases. – animuson ModStaff Sep 18 '13 at 18:05
  • 1
    Failed review audits will lead to a ban. The bans are exponential. Read the second bullet of my suggestions. The point is to avoid excess audit posts on meta – Matthew R. Sep 18 '13 at 18:06
  • 3
    Review bans are only exponential up to 30 days. You cannot be automatically banned for more than 30 days, nor can a moderator manually ban you for more than 30 days. I'm pretty sure my first comment covered that. Stack Exchange does not like permanent bans, especially not automatically. That's why they only allow moderators to suspend users from the site for up to a year. – animuson ModStaff Sep 18 '13 at 18:07
  • 1
    Okay - I will update the question. I dont want this to get off topic because of one line in the question. – Matthew R. Sep 18 '13 at 18:09
  • 2
    I have not seen too many review ban appeals on meta. I've seen lots of I failed this audit when I shouldn't have, but that's all. The people in those situations are generally people paying attention and doing at least a reasonable job of reviewing, and as such very rarely actually fail enough to be banned. People messing up enough reviews to be banned virtually always know they're bad reviewers, and so don't bother to appeal. What's left is a small enough number that it has (based on my observations) been a manageable number not requiring a special ban review queue to support. – Servy Sep 18 '13 at 18:18
  • 5
    You really shouldn't have three rather independent proposals in a single post. The third proposal is a duplicate, and the other two are different enough that they should probably be discussed independently (even if they link to each other as being related). – Servy Sep 18 '13 at 18:20
  • I think the problem with how review bans are issued is that the system looks at all of your review stats from the last x days. I don't think it actually keeps a running tally on fails vs passes, but looks at your ratios over so much time before failing the current audit. The problem with that is that when you manually unban a user from review, it only take one more fail to re-trigger the ban since unbanning doesn't really clear anything. – animuson ModStaff Sep 18 '13 at 18:20
  • 1
    ...basically, when you get review banned: you worked hard to earn it, and we wouldn't want to cheapen your hard work. – user7116 Sep 18 '13 at 18:20

You must log in to answer this question.

Browse other questions tagged .