I found the review instructions confusing, mostly because I was expecting some kind of "item goes away from the list after I've done something to it" action. After quite a bit of poking around I finally get that that's not what happens, but I still have no real solid idea of when items leave the list.
For perspective on where I'm coming from, I have 4100 points on SO so it's not like I'm new to the site. My flag weight of 135 shows I am a fledgling in reviewing but I'm willing to help occasionally.
My feature request
How about creating some clear instructions for new reviewers so they can understand what the heck to do and don't click around getting all confused (like I did)? Answers by themselves without questions made it hard to review them, so I kept clicking to the actual question page, then when the item didn't disappear from the review list I figured I'd done something wrong and I had to make my upvote/comment/edit right in the review page. I lost a bit of time experimenting with that approach and still saw no result. I wondered what I'd done wrong and spent a fair amount of time thrashing around.
You could put the link to the instructions on the right side of the review pages so anyone who wants to read more about it all can do so and quickly get with the program. I'm imagining something very similar to the "How to Format" box to the right when writing a question. It has links at the bottom to pages with extended help, in case the brief explanation is not enough.
Building in the right incentives
I have to say it's a bit of a demotivator to participate in review to know that what I do won't in fact make items disappear. Because then I don't get that tiny intrinsic payout of "I made the number of items go down." Why would I spend my own valuable time reviewing if I can't see that I did anything? I need to SEE it, not just figure that I must be helping somehow in some nebulous way. Some kind of positive feedback loop that helps me see the result of my effort will be much more likely to make me come back and keep helping. I mean, I can see my reputation points go up almost immediately when I participate, but there's nothing like that when reviewing except delayed flag weight increases that are not visibly tied to a specific action I took.
For one example of seemingly spinning wheels, why does an answer to a closed question show up in the list? Does the answer even matter at that point? And what about comments? If I comment, does that do anything to the review items? Or if I edit does it make them go away? You get the idea.
Helping ensure the work is done
Last, given there are hundreds of items needing review, it seems like it would be important to have a list of truly unreviewed items so that some assurance can be had that each item needing review actually got it. If the review list is random, depending on how many people are volunteering to work the list, you may have items fall through the cracks that never get a review. By making some clear process that actually removes items from the "unreviewed" category, I think you will get better coverage of the task.