I like that you carefully review the queue. I think that not every question in the queue should be closed and that robo-ing through the whole thing casting "the right close vote" could result in unwarranted closures.
It would be very nice if there were a way to get date ranges from the review queue but that isn't an option at the moment. Some people suggest to just sort by a narrower tag.
I am not sure there is way to use the data explorer to check for current close votes (or at least not that I am aware of).
If you have 10k+ reputation
One way to do this which would take a semi brutal amount of time would be to wade through the old reviews yourself. You can make a rough estimate of where to start in the history. There are 50 reviews per page in the history. There are currently 118k reviews in the backlog. This means a rough estimate would be to start at page 2360 of the history.
Once there, you could then look at every question which has not yet been
[closed] and review it if it still had any close votes. This user script will help filter the links out which are already closed.
return $(this).find('td').eq(1).text().indexOf('[closed]') > -1;
This is definitely a slow process, but it is one way that you may visit older content which has previously been in the review queue.
I would advise that you just cast your close votes as you encounter questions in the wild and not worry too much about the size of the queue, but it is your prerogative and if you feel the need to review really old items in the queue, then this may be an approach which you could use.