That people would use this feature the way you're suggesting you'd use it, probably means we should not have this feature.
Suppose a question that should not be closed has one close vote. Then if it gets another vote, that's not so bad--it's still not closed.
On the other hand, suppose a question that should not be closed has four close votes. Then if it gets another vote, it's closed. Now it will either remain wrongly closed, or people will have to hassle with reopening it.
The presence of the new review system for close votes makes it much easier to get questions closed than in the past, and since we don't yet have a system for reviewing reopen votes, the balance is already tipped somewhat against any question that is wrongly close voted.
Encouraging people to be more willing to vote to close when the question has lots of close votes would substantially magnify this problem, and there is really no benefit associated with it.
The closer a question is to being closed, the more reluctant we should be to vote to close it.
Furthermore, suppose a question has just one close vote, or zero (as I believe questions enter the close queue just by being flagged for closure by users with insufficient reputation to cast close votes). If (even a few) people were to see this and uncritically click Do not close, that would keep the new review system from getting low-visibility questions closed. Since that is the primary benefit of the new review system for close votes, that would be a bad thing.
In conclusion: The idea that more close votes means you should be more willing to cast a close vote is a widespread intuition, but it is incorrect. This feature, if implented, would magnify this wrong intuition and give it way more power than it currently has.