(I'm talking about the /review page here)
Definitely worth taking votes into account: this one showed up as low quality, but it's accepted with (currently) 12 upvotes.
Also, could you have some way of marking things as "Taken care of", so other people know they don't have to bother? (Or at least as a marker for myself). If you don't trust one person to 'TCO', you could have a 5-vote style system like for closing...
