I would like to suggest a couple changes in the First Post/Late Answer review process.
1. Rework the Reviewer and Steward badges to something more like this:
Completed at least 250/1,000 constructive* review tasks. (*No Action Needed Not Included) This badge is awarded once per review type.
or as Animuson suggested:
Why not go back to the requirements the Reviewer badge used to be? Reviewed x posts, with at least y posts actioned. – animuson♦
As mentioned in other posts on this problem, robo-reviewers tend to just hit the no action needed button to try and fly through as quickly as possible.
My understanding of badges, is that they are used to encourage behaviors that benefit the community.
When does "No Action Needed" really benefit the community?
Note: I'm not counting not creating additional problems as a benefit...
- If a post is good up-vote it.
- If a post is bad: down-vote, edit, leave a comment, and or flag.
- If you're not sure: click Skip (Not no action needed)
To better illustrate my point a gratuitous freehand circle-
I know there are cases when there doesn't seem to be any problem with a post, but you don't feel it warrants an up-vote, hence the No Action Needed button, but
should we be handing out badges to people who just hit the button 250 or 1,000 times?
2. Set up an automated flag to trip whenever someone uses the same review response X number of times in a row.
- Up-voted in review X times in a row: Flag
- Down-voted in review X times in a row: Flag
- No Action Needed in review X times in a row: Flag
- And so on...
Of course we wouldn't want to auto-ban people who tripped a flag in this way, there is an outside chance that someone really did run into 5 or 10 posts of the same quality in a row, but as its unlikely, it would be an easy way to spot troublesome review patterns and bring them to a moderator's attention.
If we can catch things like vote fraud with an automated process, why not try something similar with robo-reviewing?