50

This question is mostly an excuse for sharing some research I've been doing. It essentially replicates the data Shog gathered a few years ago. I think the results suggest some systemic and social solutions to our ongoing problems with question quality on Stack Overflow (and increasingly other large sites). In particular, I think we've created a system that encourages bad questions at the expense of reasonable ones. Fixing that system will depend on having an accurate understanding of what motivates people to ask questions.

The experiment population

One way to find out what causes people to ask questions is to ask them. We've been doing that research, but it's hard to avoid a skewed sample. People who respond to interview requests (such as the call at the bottom of the recent blog post) are probably not typical. Fortunately, we have a massive sample of users who have asked a question, received various types of feedback, and either asked another question or not.

For this experiment, I looked at:

  1. Users who asked their first Stack Overflow question in 2018,
  2. Have not answered any questions, and
  3. Whose initial question was not flagged as spam.

I also looked only at events that occurred within 28 days (4 weeks) of the initial question. That's plenty of time to accumulate feedback data and gives a standard cutoff for when we stop looking for followup questions. In other words, if a question gets a comment, vote, answer or whatever more than 28 days after it was asked, that feedback is not counted in this analysis. (Also, questions younger than 28 days old have been filtered out.)

Model inputs

To have a really complete model, we'd ideally have access to all the factors that go into a user asking a question. It would be nice to know, for instance, how many programming problems the user faces in a week. Then we'd want to know how many of those problems have solutions findable via Google and an estimate of how skilled each user is at searching. Unfortunately, we don't know these inputs. Instead, we're going to look at factors that are recorded in our database:

  1. number of answers to the initial question,
  2. comments by other users,
  3. upvotes,
  4. downvotes,
  5. edits by other users,
  6. whether the question was closed,
  7. whether the question was deleted, and
  8. the sentiment of any comments the question received.

You probably know what all but the last of those mean. I'm using the AFINN lexicon I used to evaluate comments on Interpersonal Skills. It's not particularly geared toward technical prose. (Variations of "kill" and "die" are rated -3 on a -5 to +5 scale, but would usually be neutral on Stack Overflow, I'd guess.) But I was able to verify that comments flagged and deleted as "rude or abusive" do, in fact, have a lower sentiment score than average. In the future, it might be better to use a lexicon tuned for technical writing. (I recently learned of this paper which is a step in that direction.)

The model

I used R's lm function to generate a simple linear model. Compared to physics models I've used in the past, the fit is not particularly good (adjusted R2 = 0.01819). But that's to be expected in a social science context. There are just too many factors we could never account for to make a robust model. The important thing is that this model gives us a good idea of which actions are significant to future participation plus the direction and magnitude of that influence.

Here's the model summary for how many followup questions a user asks based on the response to their initial question:

Call:
lm(formula = FollowupQuestions ~ AnswerCount + CommentCount + 
    Upvotes + Downvotes + EditCount + Closed + Deleted + sentiment, 
    data = first_question)

Residuals:
   Min     1Q Median     3Q    Max 
-3.210 -0.394 -0.287 -0.158 48.540 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept)   0.247344   0.007434  33.271  < 2e-16 ***
AnswerCount   0.104152   0.005008  20.796  < 2e-16 ***
CommentCount  0.026986   0.002302  11.723  < 2e-16 ***
Upvotes       0.006195   0.004482   1.382    0.167    
Downvotes    -0.050615   0.002640 -19.171  < 2e-16 ***
EditCount     0.036656   0.006569   5.580 2.41e-08 ***
ClosedTRUE   -0.093085   0.014601  -6.375 1.84e-10 ***
DeletedTRUE   0.209662   0.014104  14.866  < 2e-16 ***
sentiment     0.014276   0.014209   1.005    0.315    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Residual standard error: 1.097 on 62131 degrees of freedom
Multiple R-squared:  0.01832,   Adjusted R-squared:  0.01819 
F-statistic: 144.9 on 8 and 62131 DF,  p-value: < 2.2e-16

Allow me to draw your attention to the coefficients one at a time:

  1. The (Intercept) coefficient suggests that askers whose first question received no feedback at all would go on to ask 0.25 more questions. That's only a little lower than the average of all askers (0.39 follow up questions).

  2. As you might expect, the more answers a user gets on their first question, the more questions they will ask in the future. This is a large effect (adding 0.1 question per answer) and significant. People come for answers, and when they get them, that encourages users to ask again.

  3. Comment count similarly correlates with followup questions, but to a far smaller degree than answers. I think one reason this happens is that people sometimes answer in the comment section. But there might also be an "all attention is good attention" effect in which getting a comment is seen as positive, because it means someone interacted with the question.

  4. Getting an upvote encourages followup questions to a very small degree. This makes sense because new users generally don't care about the game. Rather, they want an answer to their question. Sometimes I see an angry comment asking how a question could have been upvoted. If they are worried about encouraging more questions, this model suggests they wasted their effort.

  5. But not all votes are created equal. Downvotes discourage more questions half as much as answers encourage them. (Insert my thoughts on the value of downvotes.)

  6. Edits are roughly as encouraging as comments. We sometimes hear new users complain about their questions being edited, but it actually seems to be welcomed by most askers.

  7. Closing a question discourages new questions more than a single downvote. This could be a result of decreasing the potential number of answers (which is the primary purpose of closing). It's also likely closed questions are associated with more downvotes. I didn't separate out duplicate closures, which might be interesting to do in the future.

  8. Contrary to previous results this model indicates that having your initial question deleted increases the odds you'll ask again. It may be that once you account for all the other variables, the main effect of deleting a question is that it is no longer going to produce a helpful result for the asker. So, naturally, the asker is incentivized to ask again. It's also possible people delete their questions with the intention of wiping the slate clean to start over. (I didn't separate out self-deletions.) According to this model, the positive effect of having the initial question deleted is greater than the effect of getting an answer. I'd like to look into this further.

  9. I haven't talked about the significance tests yet, but comment sentiment are not significant according to this model. (Upvotes are also not significant in this sample.) A quick way to tell is to look at the absolute value of the coefficient estimate and compare it to the standard error. If the error is large relative to the magnitude of the coefficient, the estimate clearly isn't telling us much. In this model, positive sentiment decreases followup questions, which is hard to explain unless it's just noise.

So, um, comment sentiment doesn't matter, right?

Not using this method of calculating sentiment and according to this model. It's possible a better lexicon will turn up a stronger effect. It's also possible the effect is only significant with extremely negative (and positive) comments. But I think it's far more likely that the effect of getting a comment drowns out any effect of how the comment was worded.

I should also point out that we are only looking at the effect on the asker themselves. Some questions are seen by hundreds or even millions of people. As much as we like to think comments are second class content, we put comments on questions in first class position:

Comments drowning out the answer

To casual visitors, question comments are more visible than answers. We know from talking to people who don't contribute that one reason is because they see discouraging comments on the site. In sum, leaving a negative comment on a bad question:

  • encourages the OP to ask again and
  • discourages anonymous users from asking a question.

Unless we are confident that non-users ask worse questions than we already have on the site, this seems like a poor design.

  • 15
    This is really cool data, Jon! However, I feel the need to leave behind some constructive criticism; I hope you don't mind too badly :). When you write certain actions 'discourage' and 'encourage' new questions, you are already providing an interpretation of the data. Another interpretation, for example, could be that people who get answers to their questions are better at formulating questions, and hence ask more. Perhaps nitpicking here, but there is a subtle difference. – Steven Jeuris Jun 20 '18 at 16:23
  • 7
    @StevenJeuris: That's a completely fair criticism. Overall, I'd say the data is suggestive of causations without proving them to any degree. I think it's a good framework to keep in mind, but more research is needed! – Jon Ericson Jun 20 '18 at 16:33
  • "Sometimes I see an angry comment asking how a question could have been upvoted. If they are worried about encouraging more questions, this model suggests they wasted their effort." I expect people complaining about that are more worried about people seeing the question in the question feed, going to the question thinking it's actually a good question, and then finding a horrible question. In particular, they themselves may have gone to the question because it was upvoted, and been dissapointed to see an awful question. – Servy Jun 20 '18 at 17:23
  • 3
    @Servy: Yeah. Whatever the motivation, this model suggests the very act of commenting encourages the OP to ask again whereas the upvote did not. That people feel so powerless in the face of bad answers is a problem with the system. It's especially frustrating since a simple downvote probably works better than any amount of negative commenting. – Jon Ericson Jun 20 '18 at 17:38
  • That outlines part of the problem quite well: Just read a bit, find something not clear, put in your 2¢ - but often not so construction a criticism. It's noise and negatively with a sprinkle of things not well though out that's causing some to say: "Back to the search engine, I'll find my own answer on the Internet; rather than restricting myself to the 'who, what and how' presentation style". That's unfortunate because our various sites have some expert answers to various complex and multilevel problems, and researching the answers is often a learning experience too. Maybe stats.SE can assist – Rob Jun 22 '18 at 17:05
  • Sort of a stat side-question (I'm not trying to nitpick): the response variable here is integer counts? Thoughts on poisson/negative binomial approaches instead? I'd be curious to see how skewed the distributions of follow-up question counts were, whether there are tons of zeros. – joran Jun 26 '18 at 22:15
  • @joran: More than fair question. I did do a logistic regression with the dependent variable being whether or not the user asked another question. I decided to not present those results because I've never used that model before, it seemed harder to explain and the results were similar in terms of direction and magnitude of coefficients. I'll look at other distribution next since I expect the difference in effect is greatest between 0 and 1. 10 comments are not likely to be 10 times as motivational as one comment. – Jon Ericson Jun 26 '18 at 23:01
  • 1
    When “people who don't contribute [say] that one reason is because they see discouraging comments on the site”, it’s an indicator that Stackoverflow guided them to the wrong questions as the good Q&As don’t have such comments. One reason could be the search function about which users complain since a decade or so. But no, we don’t fix the search function, we tell the users to “be nice”. That’s much better. Since niceness can’t be measured and ironically, the rudest questioners complain the loudest about the others not being nice, you can always blame the lack of “be nice” for any problem. – Holger Jun 27 '18 at 7:30
  • 1
    @Holger Google frequently recommends to me questions that are mediocre score or closed as top hits for terms. It's a wider issue than SE search, especially since most of SO's visitors arrive from an external search engine. – Troyen Jun 27 '18 at 8:29
  • @Troyen that is still better than Stackoverflow’s own search engine which occasionally doesn’t find any hits when you know for sure that there are some. This leads to the situation that new questions get immediately closed as duplicates by those, who know the existing question, and downvoted by those who think the questioner didn’t research, not knowing how unhelpful the search engine is. On the other hand, when I used Google with problem descriptions, I always landed on helpful SO content, that’s how I came to Stackoverflow initially. – Holger Jun 27 '18 at 8:47
  • @Holger: We are working on search right now. You might very well see some improvements soon. But the work to improve search is difficult and slow. Testing changes takes time to converge on meaningful results. Even when we see improvements, they tend to be minor. It's a hard problem. But it's sorta beside my point. The people I'm talking about (and have talked with) always come via Google. They aren't asking, so they don't see the duplicate search results. But they do see the useless comments complaining the OP didn't search. My results suggest they are totally counterproductive comments. – Jon Ericson Jun 27 '18 at 18:53
  • 2
    What is considered the "ongoing problem"? Is it some form of negativity or the inability of our users to digest criticism like an adult without needing systematic and social solutions to provide them with a safe space. I believe that criticism by its very nature sparks introspection, thought, and discussion, and should not be deemphasized in any way. – Zhro Jun 27 '18 at 21:54
  • @Zhro: I mostly agree with you, but there's good reason to suspect criticism produces far more resistance than introspection in most cases. The trouble is we tend to think we ask useful questions and have the best answers. Comments that say otherwise produce a backfire effect that causes us to double down. I've seen it happen. A terrible asker was convinced that their questions were gold because the commenters misunderstood them. As a result, this person rushed headlong into a question ban to prove their point. – Jon Ericson Jun 27 '18 at 22:09
  • @JonEricson: Are you suggesting that because valuable criticism can be offensive we should instead encourage a position of apathy and ignorance? Wouldn't this support the very fallacy you cited? – Zhro Jun 27 '18 at 23:05
  • 3
    @Zhro: No, I'm saying that unless you are good at writing a critical comment, there's a good chance you'll be encouraging the exact behavior you want to discourage. Statistically, a comment (whether critical or not) encourages another question to some degree. But downvotes are another matter. A downvote clearly discourages more questions, costs almost no effort and doesn't reinforce new users belief that our network is basically a place for jerks. – Jon Ericson Jun 27 '18 at 23:13
16

Interesting results! Thanks for sharing this work.

On the subject of deletions increasing the likelihood of future questions, you might want to look at timing. You did this study on SO, where questions can be closed and deleted within a few minutes. It seems possible that some of those followup questions are users thinking "huh, I thought I submitted my question but it's not there so maybe I need to retry", because there is no notification for deletions. Trying to detect "re-asks" among the pool of questions could either support or falsify this hypothesis. One way to do this would be to look for questions asked within a short time of the deletion. (Definition of "short" TBD, but on SO it must account for the rolling rate limit taking effect immediately.) Another (harder) way would be to do textual analysis on the content to try to determine similarity. Also, you'd want to exclude self-deletions here.

  • at Stack Overflow the impact of votes down and closure has to be mostly caused by a cool-down effect because at this site rolling rate-limits kick in immediately (as opposed to other sites where original approach that ignores the first question is followed) – gnat Jun 20 '18 at 15:51
  • 1
    @gnat thanks; I didn't know that SO had customized that. I've edited. – Monica Cellio Jun 20 '18 at 15:58
  • my assumption of cool down effect playing a role can be checked by collecting and comparing similar stats from non-SO sites. Per my observations at SE.SE it seems that many users tend to give it a second try even when their first question is voted down and closed (10K link to one recent example) – gnat Jun 20 '18 at 16:25
  • @gnat Interesting to see an old question of mine again :D – πάντα ῥεῖ Jun 20 '18 at 17:15
  • 3
    I suspect that just eliminating self-deletes will make the effect go away. I had this post mostly written up before I noticed I hadn't excluded them. (My SQL will have Taryn in tears if she ever sees it.) I hope to have time to re-run the model with that correction in a week or so. – Jon Ericson Jun 20 '18 at 17:16
  • @jon Well, delete notifications might be a useful thing. It should be as transparent as possible for new users what's actually going on, and to deliver them with reasonings from the start. – πάντα ῥεῖ Jun 20 '18 at 17:18
  • @jon "(My SQL will have Taryn in tears if she ever sees it.) " Dripping slowly onto her blue feet ;) – πάντα ῥεῖ Jun 20 '18 at 17:20
  • @jon I noticed her ;) – πάντα ῥεῖ Jun 20 '18 at 17:28
  • 1
    @JonEricson Regardless of what is shown for the effects of deletion when excluding self deletions, it just happens way less, and when it does happen, it typically happens much later than self deletions, so whatever the effect per instance is, the net effect of the whole site has to be tiny, due to just how infrequent it is. Maybe the effect switches from positive to negative, maybe not, but either way, it's effect on the site as a whole loses relevance. – Servy Jun 20 '18 at 17:28
  • @Servy: Yes, that's my intuition too. It's also a bit awkward that we use other signals in the model to trigger automatic deletion. – Jon Ericson Jun 20 '18 at 17:32
  • 2
    @JonEricson I think the big thing though is that anytime I ever see someone asking their question, it's only ever a result of things that happen to the question with at most a day. I just don't see people re-asking a question because it was closed/downvoted/deleted/etc. a week (let alone a month or a year) after they asked it. – Servy Jun 20 '18 at 17:35
  • @Servy Well, they usually come back to these (if still attending), after they got question banned :3 – πάντα ῥεῖ Jun 20 '18 at 18:01
  • 1
    @Servy: My data cuts off after 28 days, but I suspect I'd get similar results looking anywhere between 3 days and a week. I chose the longer window because I don't really know what the cadence is for people to have potential SO questions. Remember some of the askers will be respectful of the resource and I'd like to capture those user too. – Jon Ericson Jun 20 '18 at 18:03
  • @JonEricson I mention a week because that's a relevant timeframe for the roomba (along with month/year), moreso than anything else. – Servy Jun 20 '18 at 18:06
5

Thanks for this, it's really great to read. While Let's hold language in comments to the same standard as posts talks about how to address the comment sentiment problem, there's still another problem you brought up in your post: we have to scroll to see the answers even if just a few comments exist.

I'm sure I have brought this up in Tavern on the Meta before, but I have had a solution for that since early 2016 (JSFiddle link) (view Snippet in Fullscreen for best effect):

html { margin: 0; }
body {
    width: 768px;
    margin: 0 auto;
    position: relative;
    overflow: hidden;
    font-family: "Arial";
    margin-top: 16px;
}

h1 {
    font-size: 22px;
}

p {
    border-bottom: 1px solid #E0E0E0;
    margin: 0;
    text-align: right;
}

label {
    border: 1px solid #E0E0E0;
    border-bottom: 0;
    padding: 5px 5px 2px 5px;
    background: white;
    position: relative;
    z-index: 2;
    cursor: pointer;
    font-size: 0.9em;
}

input[type="checkbox"] {
    position: fixed;
    top: -20px;
}

.container {
    width: 768px;
    height: 509px;
    position: relative;
}
.container div {
    position: absolute;
    width: 768px;
    height: 509px;
}
.container .first {
    top:0px;
    left:0px;
    background-image: url("https://i.stack.imgur.com/6ZjLp.jpg");
    transition: all 0.7s ease-out;
}
.container .second {
    opacity: 0;
    top: 0px;
    left: 800px;
    background-image: url("https://i.stack.imgur.com/Wecao.jpg");
    transition: all 0.7s ease-out;
}
.cover {
    position: absolute;
    top: 0;
    left: -100px;
    z-index: 1;
}
.cover.one {
    width: 67px;
    height: 27px;
}
.cover.two {
    width: 79px;
    height: 27px;
}
input[type="checkbox"]:not(:checked) + p label.primary {
    border-top: 2px solid orange;
}
input[type="checkbox"]:checked + p label.secondary {
    border-top: 2px solid orange;
}
input[type="checkbox"]:not(:checked) ~ .one {
    top: 34px;
    left: 615px;
    z-index: 5;
}
input[type="checkbox"]:checked ~ .two {
    top: 34px;
    left: 688px;
    z-index: 5;
}
input[type="checkbox"]:checked ~ .container .first {
    transform: translateX(-800px);
    opacity: 0;
}
input[type="checkbox"]:checked ~ .container .second {
    transform: translateX(-800px);
    opacity: 1;
}
<h1>Why doesn't position: sticky work in Chrome?</h1>
<input type="checkbox" id="question" />
<p>
    <label for="question" class="primary">Question</label>
    <label for="question" class="secondary">Discussion</label>
</p>
<div class="container">
    <div class="first"></div>
    <div class="second"></div>
</div>
<div class="cover one"></div>
<div class="cover two"></div>

The "Discussion" tab should be visible and always show all the comments. When the first two or three comments are posted, they also show up under the question on the "Question" tab as well... once comment #3 or #4 is posted, no more comments show up under the "Question" tab, and you only see a link to the comments/add a comment under the question.

Done this way, you'd still be able to highlight parts of the code or question to copy it and reference it in a comment.

  • 4
    I love this idea. We did something similar on Documentation based on Wikipedia's Talk pages. I'd be in favor of putting all comments in a Discussion tab and ditching the sometimes confusing hidden comment logic. It would be especially helpful for meta comments: "That solved my problem!" or "Possible duplicate of". (The later still have an effect on the Question tab since the links will show up in the sidebar. Maybe they should be highlighted somehow as more relevant.) – Jon Ericson Jun 27 '18 at 19:01
3

Note that this is more of a counter-argument but I felt that it deserved at least some note as it relates to whether we should intervene (encourage or discourage) a certain response.

I think the results suggest some systemic and social solutions to our ongoing problems with question quality on Stack Overflow (and increasingly other large sites).

What is considered the "ongoing problem"? Is it some form of negativity or the inability of our users to digest criticism like an adult without needing systematic and social solutions to provide them with a safe space. I believe that criticism by its very nature sparks introspection, thought, and discussion, and should not be deemphasized in any way. If the response is inappropriate then we have facilities to address that.

As much as we like to think comments are second class content, we put comments on questions in first class position.

Comments are first-class citizens and are often just as important as the answers. Comments are not only the driver for channeling a solution for the author but also help others to decide which answer, if any, best solves their problem.

Many new users come here under the false assumption that there is only one answer to their problem. In my opinion, I think that a lot of their frustration comes not only from this realization but also any inexperience they may have at confrontation. They want an answer to their question and don't want to have to interact with anyone to get it.

If negativity in comments, a terse but capable avenue for driving emphasis, is a barrier then that is a failing of the student to appreciate criticism. As long as the negativity isn't itself hostility then I don't see a problem with it. If what we value is a better community then we should be educating our users on the value of criticism to help them become more open to opposing views.

If a comment is both negative and nonconstructive, that is what the flag feature is for. If the comment persists, it can be assumed that it provides added value and is visible as such. It's up to the reader to decide how to react.

I don't see anything wrong with the current emphasis. What I do see is an attempt to shift the perceived problem to somewhere less visible and is at risk of making otherwise useful information less accessible to those of us who are mature enough to appreciate it.

  • 1
    I think that part of the reason this question and Tim's question about kindness in comments exist is to encourage people to actually flag those sorts of comments that are both negative and nonconstructive. Part of the problems that I've seen as a mod is that these sorts of things are largely under-reported to moderators, so handling them is spotty. Encouraging flagging and helping moderators know what to remove is a big part of the discussion. – Catija Jun 27 '18 at 22:39
-5

This post is meant to agree with Jon's data and conlusion that comments are a problem. What it does is propose a different reason for why they are the problem and proposes a novel solution. I disagree that the root cause is the comment writers and that the comment readers are just as much of the cause, if not even more of the cause.

This is a post I made on meta.stackoverflow.com. I think it is a valid response to this question here.

TL;DR you are responsible for your feelings, not someone else and everyone needs to learn how to take criticism. But this is the internet ...


The first thing they teach you in art school is how to take and give criticism as a professional.

Criticism is not just a statement to the contrary, it also must contain a reason for the criticism and must be accompanied a proposed alternative solution.

Before leaving a comment basically stating. You are wrong! and nothing else, think about how productive that is in the whole. Without the proposed alternative, it is nothing more than a verbose -1. Think about how that proves my point about the comment system.

Downvotes on meta mean "I disagree". Why post a comment that basically states "I disagree and some generic negative comment" and nothing else, is illustrating the point and just providing objective data to back up subjective feedback. Why not just down vote and move on, that is the current community mantra and party line.

For the uninitiated that is almost just as unwelcoming, at least they do not see a comment that they can internalize a some kind of abuse directed as them personally. It is not a solution, that much is clear.

On the main site, the down votes have real consequences and the comments are taken as criticism, with no positive alternative solution proposed.

Problem: Comments

The presumption of the argument is based on comments are the root cause as evidences in every case cited that people are complaining about.

So comments no matter how neutral are causing feelings of unwelcoming and no comments are considered just as unwelcoming. Damned if you do, damned if you don't.

I make no assertion that any of these feelings is legitimate. That is impossible because feelings are subjective. Jay Hanlon is the one that made the erroneous jump to:

"When someone tells you how they feel, you can pack up your magnifying glass and clue kit, cuz that’s the answer. You’re done. And a lot of devs feel like Stack Overflow is an intimidating, unwelcoming place. We know because they tell us."

I am doing nothing but proposing something to try and change what he thinks is wrong.

The current comment system is the majority source of the feelings because of two polcies that exacerbate the inference and amplify things perceptions that are not there or completely unintended.

  1. comments are not for conversation: that means they are terse, one liners and people are loath to respond to lines of questioning to avoid conversing. Which typically devolves into arguing because of the terse one liners.

  2. vote and move one: this community policy is recommended because of the first problem. It causes its own problems and exacerbates the first one. No follow ups on down votes, close votes, etc. No comments are just as problematic for new community members as terse comments with no follow up.

So we should just remove the ability to comment. That much has been expressed in the past ( which I actually disagree with that proposal, it does not go far enough and is what prompted this post ) and tries to address the root cause of most of the problems that Jay Hanlon's blog post tries to make a point about but gets lost in the clumsy way he blames the volunteer community members that contribute the content to the site that makes it what it is and also whatever financial commercial success it may have.

This is my attempt to address the problem, nothing more.

The in these cases where comments are taken as offense almost always have devolved into a back and forth between one or more people. Which just exacerbated it the problem. Someone feels like they are being ganged up on or ignore at some point.

It is a damned if you do damned if you don't situation, has been for the at least last five years, from my experience.

Chat rooms get suggested by some heuristic now, usually long after it should have been taken to chat and everyone is pissed and wants nothing to do with any more communication.

Solution: Chat Room for the OP

Have a public but not visible unless you click to see it chat room instead of comments. Remove the comments as they exist now.

By definition comments are not for conversation and they are about the worst way to hold one if conversation was allowed in them.

By definition chat is for conversation and real time interaction, which would go a long ways to changing the unwelcoming perception that the no conversation in comments commenting system evokes now.

This by no means completely removes the problem that some people are just going to be assholes no matter what, it just removes some of unintended perception that people that are not intentionally being an asshole to a form of communication that is more suited to conversation instead of tossing terse one liners over a fence.

The logical conclusion is that the ability to leave comments be removed.

I suggest that that every question gets a chat room instead.

You could make the OP the room owner and moderation would be from the community members with badges in the tags of the question.

Lock the chat room when an answer is accepted or abandoned for some period of time. Make it vote to reopen or something, with previous participants getting higher weighted votes, or flagged for moderation just like comments would have been.

Delete the room when the question is roomba'd, delete voted, has no activity for a period of time or by a moderator just like comments would have been.

The technology is there, refactor it where it will scale to this requirement.

  • 1
    Right now chat moderation is practically non-existent. I'm not sure how much you know about it, though. It's also worth noting that there are three chat servers - one for SO, one for MSE and one for the other 170+ network sites. Without extreme changes being made to how chat works, your suggestion would be technically impossible. As far as I'm aware, chat doesn't know what badges you have (or on which sites), only how much reputation you have. Social changes may rely on users each making small changes, but they don't take dev time away from the primary focus of the site - Q&A. – Catija Jun 28 '18 at 18:14
  • So comments no matter how neutral are causing feelings of unwelcoming and no comments are considered just as unwelcoming. No, it is not about "comments, no matter how neutral"; neutral comments aren't the issue here. It's rude comments that people are claiming to be neutral, like "did you even bother trying?!" ...that's not neutral. – TylerH Jun 28 '18 at 18:28
  • 1
    Just so you know, I found this answer both condescending and uninterested in engaging with my question. I see very little evidence you grappled with the evidence I presented. I agree that I'm responsible for my own feelings and therefore I intend to move on with my life and not let you ruin my afternoon. Honestly, it looks like you have some good ideas there, but you have to understand that the way you present them matters. Whether consciously (as I'm doing in this comment) or subconsciously (as is more common) people will tend to ignore input that makes them feel bad. – Jon Ericson Jun 28 '18 at 19:05
  • I think it meshes very well with your question, we agree comments are the problem your data seems to back that conclusion up, even if the causation may be arguable ( readers vs writers ) of them, and we can disagree on that, the end result is not. Moving where the are on the page will just be moving the problem. I think the entire concept of comments is the problem. Completely changing the paradigm is on the only way to prove one way or the other, but I expect that some baby steps/half committed solution will be tried and fail because that is what baby steps solutions are designed to do. – Someone Who Used To Care Jun 28 '18 at 19:56
  • @TylerH - I know you see it all the time, neutral comments that may be terse get complained about just because they are terse. Comments that make no reference to gender implied, inferred are imagined to be misogynistic and ranted about on twitter/tumblr blogs and get quoted on meta.SO on a regular basis as "evidence" of such behavior. I see it every day on low quality questions that have some very neutral boilerplate comment like that goes out of its way to be neutral and point someone to "How to Ask?" and the comment gets attacked viciously. So neutral comments are problematic. – Someone Who Used To Care Jun 28 '18 at 20:02
  • @Catija - let everyone moderate the question/answer chat sessions, that scales for the rest of the site now. I am familiar with the problems that chat has, that is why I wrote the last sentence. refactor it where it will scale to this requirement. If FaceBook, Twitter and Discourse can do it anyone can. – Someone Who Used To Care Jun 28 '18 at 20:05
  • Everyone already can use flags in chat... 10K+ users and mods see them... your answer specifically states that tag badge holders would be the moderators, which is what I'm saying wouldn't work with how the system is currently designed... and I'm not even sure how that makes sense. Why would having a badge in a specific tag make you better able to determine whether someone is behaving in an abusive way? – Catija Jun 28 '18 at 20:09
  • @Catija - moderation in as in reopening closed sessions, etc. Just like hammer privileges are now. – Someone Who Used To Care Jun 28 '18 at 20:54
  • FWIW, I have myself argued in favor of the general idea that "comments are the problem" and that they should simply be made much less common via some method perhaps even removing them entirely, or the UI for "question improving/clarifying discussion" should be radically redesigned. That in itself has tended to be rather unpopular. But I agree with @JonEricson that everything else you've put in this post surrounding that basic idea I find rather off-putting. – joran Jun 28 '18 at 22:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .