This confusion came up in comments on another Meta question.
On the site I use most actively (RPG.SE), I've noticed that the counts for each review queue appear to only indicate the reviews available to me requiring my action. Just to be clear, I'm talking about this number here:
Since I'm a 10k user, that's pretty easy to tell: even after doing all my reviews, and the review page saying there's 0 items in each queue, I might still have that 10k notice in my top bar saying there's  reviews pending, or some other number. So they must still be there, just waiting on other users to finish them, and it just doesn't show them to me. This is fine, I like this behaviour.
However, on Meta SE about an hour ago, there was 1 item in the close vote review queue. I reviewed that item. The count stayed at 1 on the review page. When I clicked to re-enter the queue in case there were new items, I was presented with this:
Another user in that meta question I linked claims this is normal behaviour for them on their own site (Christianity.SE I think?).
What's going on here? Are the review queues supposed to be customised just to what I personally can actually review at the time? Is there a bug, are sites customised differently, or is there just some kind of caching thing leading to a time delay on the review count getting updated? (As they say, it is always caching, but is caching really the culprit here too?)