The conversation often continues in the Comments and important information is held in the comments. I'd like to have threaded comments there.
Jeff really hates threaded comments. Personally I like them - when represented in a useful way, with easy ways to mute/highlight appropriate subthreads, but I doubt that we'll ever see them on SO.
Having said that I like them, I'm not even sure they'd be appropriate for SO anyway, which is more designed for Q&A than discussion. Sometimes I wish there were some way of leaving the questions/answers as they are, but launching a new discussion (with threading) where suitable. Can't see it happening in the near future though.
Threaded comments only appear to be more coherent at a superficial first glance.
What actually happens is that the discussion forks and that the same things are being discussed at different places or that you fail to notice important arguments because you are focused on a different thread. This is what I observe on Reddit, where coming to a consensus or forming an opinion on a matter is often impossible because everyone gets lost in the multithreaded, unchronological confusion.
The tangle of threads becomes even more incoherent when people don't reply in the appropriate thread but at the bottom of the discussion, as often happens on Facebook.
But the most inportant argument against threaded comments is that on this site all comments (should) focus on the question or answer they are attached to, and because of this unifying singe focus they should all remain part of a single discourse. Threaded comments would be like groups of people wandering off and discussing things among themselves instead of partaking in a shared decision process. Threaded comments would be like having each answer to a question on their own webpage so they no longer refer to each other.
In a comment to this answer, @bobobobo claims that "[t]hreading removes the need to make visual jumps upward across 10-15 posts to see what a new comment is in response to". That claim is utterly false.
Below is a screenshot of a threaded discussion on Reddit. The arrow connects the first comment to the opening post (at the arrow head) with the tenth direct reply to that comment (at the tail of the arrow). In a non-threaded, chronological view, a reader would indeed have to jump upward across nine other comments from the tenth reply to the comment it is a response to.
But in the threaded view, this need to jump is not reduced. On the contrary, it is increased, because in the meantime no less than 200 comments have been posted in reply to the first nine replies, making it necessary to jump upward across 200+ comments from the tenth reply to the comment is was a reply to.
This example shows that both in a chronological and a threaded view are there potentially equally many comments to jump. The major difference lies in the focus of the answering options. In threaded comments, there is an option to reply to each individual comment, which narrows the focus of the poster to that comment and reinforces the impulse to create ever sub-branching comment threads. In chronological comments, with a single textarea below the comment thread, the focus is on the opening post and the whole discussion.
As Jon seems to address, threaded comments are great for following discussion. Jeff has said before that he explicitly wants to avoid discussion on SO, so if threaded comments help discussion, it would seem that adding threaded comments would, in a sense, be giving a blessing of sorts to discussions on SO. It's nice in theory, but it goes against the current mission of the site.
I do like the idea of a discussion area, though. Often times, I have a question or discussion topic that I would very much like to bring in front of this particular community, but it's not necessarily programming related. Sometimes you want to ask your "work peers" questions not necessarily related to your "work."
I wouldn't mind threaded comments that much, but I can make an argument against them:
It is too much of an investment in a distraction.
Discussions in comments shouldn't really happen: There's meta, and there's chat. The good comments directly relate to the question, not to other comments. So it's reasonable not to motivate even more use of comments for discussions than we already have, by making it more convenient.
(To be fair, one could argue meta sites are somewhat different than regular ones and the rule doesn't apply to them - but we're not talking about a meta-site-only feature, right?)
Threaded comments would be useful.
Compare these 2 views of the same discussion
(Visual jumping highlighted in red)
(Where the [-] indicates a collapsible region)
It is reasonable to object to threaded comments because you don't want a tree that is too deep
If you limit it to one level deep, that would give some more flexibility while still keeping the conversation organized.
I truly wonder why discussion is so discouraged herein. Some answers can be very difficult to get at and discussion about the answer could help a really good answer to emerge instead of having an endless list of partial answers or a hard to follow comment stream that needs to be completely followed through to get at the real juice.
On the other hand, I realize that many questions in the Stack Exchange sites are simple enough and warrant that a single right answer can do it.
A certain form of argumentative questions can be supported through the wiki mode. Perhaps having threaded comments only in the community wiki mode would be of great help in getting these hard questions to emerge. Since there are no reputation points involved in community wiki mode, editing them also solves the difficult task of assigning proper reputation points though there are some solutions to this.
It is true that arguments can be difficult to manage and often degenerate. That said, the community participating in the Stack Exchange sites is thus far very disciplined in this regards and seldom stray off topic. Yes, sure you can find plenty of examples where things just went wrong, but overall it is the exception.
Stack Exchange sites have gone a long way since their inception and are quickly becoming THE reference in many subjects. But it still lacks a certain depth, discouraging discussion certainly helped with the discipline, but it also gave the community a glass ceiling, shackles of sorts, that prevents more difficult answers to emerge on some questions.
In short, encouraging argumentation and discussion at large would most likely not be a good idea. But allowing some question to generate discussion and providing a specific UI enhancements to support discussion would benefit the whole community. Perhaps Community Wiki is the way, but perhaps a new mode of providing answers would be required to support argumentation in a constructive way.