I disagree with Jeff. On sites like Physics, sometimes a discussion between a truckload of people{*} is necessary. This is usually when we have to beat out the correct answer.
Here's an example. Nobody really was sure which method was correct, and we ended up having cross-discussions between comment threads. @pcr created a room for it, though the question sort of got resolved before we all really had a powwow in the room(a few of the answerers checked in, and discussed). In this case, a user created a chatroom, and linked to it in each one of the comments.
And yes, the chat/discussion did end up improving some of the answers. I personally do believe in the fact that comments should improve the question and then be trashed--the same goes for such chat rooms. The topical content is moved off to chat, but it ends up improving the answer in the end.
The reason we have the "get a room" feature is that two users go off on some tangent (or worse, bicker) for 10+ comments each.
Not so. Most of the time, explaining something or finding out the correct answer is annoying via comments. The "move to chat" is perfect for this. I once had a nice chat discussion with a student who wanted to make a presentation on Quantum mechanics. This would have been impossible via the comment system. True, it didn't improve the quality of the post, but someone did get helped, right?
does not mean that comment discussion is necessarily relevant to the question or any other answer on that question
I've never seen this happen in my short time on the SE network (I stay away from SO though, that may be why). On Physics the comment discussions are pretty relevant, though they sometimes tangent of to "please explain this". Which I feel is OK. We usually don't have many "my answer is better than yours"/discussions about irrelevant arcane stuff. Usually.
Conclusion
I think that this is necessary. It can be done with existing functionality by just creating a chat room, commenting on every post, and blockquote-header-ifying the chat link to your question. But yes, a less comment-spammy way of doing it would be nice. It should NOT be automatic though.--more of a 'create a chat room for this question' link'
To avoid automatic two-user chat room creation, there's not much you can do.. I guess that the question-chatroom should automatically merge itself with individual comment threads or something. With existing functionality: post a link to the new chat room in automatically created rooms. Preferably have the room created before the argument reaches autochat levels. Maybe ask a mod to import posts from individual rooms(not sure if its possible)
*Unfortunately, we don't have a truckload of people on Physics.SE :(. But the point is still valid.