I don't want to call out any specific users, as I would rather "discuss actions not users, behaviors not people" but I've come across a few users who seem to do reviews at an alarming rate...

Perhaps I'm a little slower than the average reviewer, but it seems to me, that averaging 6-12 reviews per-minute makes for pretty obvious evidence of robo-reviewing or at least questionable review quality.

Would it be worth it to put together an automated flag that simply looks at the number of reviews per-minute and if that rate exceeds X reviews per minute bring it to a moderator's attention?

  • 6
    6-12 reviews per minute? I go through comment flags faster than that. 6 reviews in 10 seconds is a bit more alarming. – animuson Oct 1 '13 at 16:11
  • I suggested something similar, though for edits, not specifically reviews. – JDB still remembers Monica Oct 1 '13 at 16:13
  • I think something like this already exists in the mod toolbox, but I could be wrong. – psubsee2003 Oct 1 '13 at 16:13
  • 2
    @animuson but comments are second-class citizens anyway, right? ;) And are usually way shorter than answers or questions, and way simpler to judge. – Mołot Oct 1 '13 at 16:17
  • 2
    @animuson We do want reviewers to actually read the posts they're reviewing right? Less than 10 seconds per post seems a little fast to both read and appropriately respond. Comment flags are another matter, comments are usually only a few sentences at best... – apaul Oct 1 '13 at 16:17
  • 2
    @apaul34208 My AvgDuration for Low Quality Posts is currently 9.4 seconds. – animuson Oct 1 '13 at 16:19
  • 2
    @animuson low quality posts are another case where you're usually looking at 1 or 2 sentences. What's your average on close votes or suggested edits? – apaul Oct 1 '13 at 16:21
  • @apaul34208 12.3 over 8 reviews...That one's not very helpful. I'll spend a day in a few queues and see what I end up at. Moderators don't really have very good stats in the review queues. – animuson Oct 1 '13 at 16:23
  • @animuson I suppose it is possible for some users to be both quick and accurate, but I would guess that, that is the exception rather than the rule. That's part of the reason I suggested an auto flag rather than a rate limit. – apaul Oct 1 '13 at 16:26
  • 2
    One might want to consider the types of closures. Off topic - "minimal understanding", "describe the problem", and "recommend or find a tool" ones may go quite fast as they can be easy to spot. Other ones where there's a good question at the core and need to be duped or need a custom off topic may take a bit more time. – user213963 Oct 1 '13 at 16:30
  • @MichaelT - Or we could just adjust the "flow-rate" flag to be a bit more universal. If a 10 second flow-rate produces too many false-positives, reduce it to 8 or 6, etc. – JDB still remembers Monica Oct 1 '13 at 16:31
  • 1
    @MichaelT That is also a very useful indicator. I normally filter by "off-topic" only when I review in the Close Votes queue and fly through them. – animuson Oct 1 '13 at 16:32
  • @animuson - Wouldn't that be easy to beat/hard to query, though? A user could just not apply a filter and then vote to close everything as off-topic. Or how would you measure the flow-rate for someone who just reviews whatever comes next (no filter)? – JDB still remembers Monica Oct 1 '13 at 16:34
  • @animuson: is there anything that detects if the suggested duplicate is not looked at for close reviews? – Qantas 94 Heavy Nov 14 '13 at 13:30
  • @Qantas94Heavy I don't think you can close as a dup. without the original popping up in the close dialog. Granted some may not take the time to read it, but it does show it to them. – apaul Nov 14 '13 at 13:34

You must log in to answer this question.

Browse other questions tagged .