There are cases where it is not clear if the question should be closed as off-topic, and can be debated either way to no end. What is generally the action taken if there is a disagreement on if the question is on topic and no consensus can be reached, should we generally leave questions open as a default?
The site already provides a mechanism to "go back and forth". That is, 5 people can vote to close the question initially. If another group of 5 people agree that the question is actually on-topic, they can vote to re-open the question. And then another 5 people can come along, think the question is off-topic, and vote to close it again. And so on and so forth.
As you hint, if this cycle happens more than once, there's probably an issue with the question. It means that either the question is barely straddling the border of being on-topic, or that the question is written or otherwise framed in such a way that it's making a lot of high-rep users think it's off-topic.
In either case, the only way to save the question is to fix the question. Some type of community consensus needs to be reached, and the comment thread on the question itself is not a good place for that type of discussion. If only there were a discussion site set up to discuss issues about Stack Overflow itself: like the questions that can be asked there, what is on topic, how to fix questions...
If you see a question like this (or you see that someone has asked a question like this), the best course of action is to open a new question here on Meta with a link to the question in question and tagged with the specific-question tag. The community here can help you come to a consensus about whether or not the question is on topic, and if so, help you to reword the question to make it obvious. And if necessary, a moderator can even step in to lock the question, preventing the close–re-open cycle from proceeding indefinitely.
But no, definitely the default should not be to leave "borderline" questions open. If anything, I think the default should be to close them until a consensus can be reached and/or they can be fixed.