I think it would be beneficial to see the current votes on questions in the close vote review queue. And also be able to see which users chose to leave it open. It can be anonymous if that is a concern (but it's already showing when you already reviewed a question).
Every now and again I click on close just to see what votes were cast thus far when I need a nudge to make up my mind (whether this is good practice is probably arguable) (it's typically between closing or skip, or leave open or skip, not between leave open and close).
Something like this: (which is identical to what you see when you already reviewed a question, but with the reason added, which I see no reason not to add to that view as well)
I realize one of the comments would most likely be that this may turn into sort of a peer-pressure scenario - voting the same as everyone else. If this is a valid concern, my only thoughts currently is to have this display be a higher-reputation privilege (or unlocked once you've cast many close votes) (presumably users will then be able to make up their own mind).
Ok, maybe not the best idea, but how about being able to see the screen after you've skipped the question, so you can at least see what other people thought (without being able to vote on it then). So that, over time, you'll get a better idea of what should and shouldn't be closed.
I realise this will require a change in the way things work by disallowing voting after skipping a question (at least I think).