21

I'm a regular reader of the site of the British newspaper The Guardian. The quality of the articles is generally high and they cover a wide range of topics. Today I was surprised to come across an article referencing Stack Exchange -- and then I was dismayed. The article is called Politeness costs nothing, but it may stop people understanding you:

Does it always pay to be polite? Dear reader, the answer very much depends. Academics in the US have analysed hundreds of thousands of answers on the Q&A website Stack Exchange, where users ask such vital questions as: “What is to the west of Westeros?” and “Are there any German nonsense poems?” They discovered that polite answers are more highly rated by the asker, something that determines how prominently they are displayed. But they may not actually be the most clear, authoritative or helpful.

Okay ... Small factual error: answers' ratings are not determined by the asker. Furthermore, as I write this, the highest rated answer to the Westeros question is by a new contributer named Nitwit who, in spite of his moniker, gives a convincing answer quoting a dialogue between two characters. (Disclaimer: I know nothing of GoT). The accepted, and highest-rated, answer to the German poetry is by someone starting a community wiki. Both answers, imho, are polite, as well as clear, authoritative and helpful.

The research paper behind the article can be purchased for $10 at MIS Quarterly.

One could say, 'Who cares?' and move on but it bothers me. Does Stack Exchange take action on this? Verify the quality of this research? Educate the article writer? Or is it just not worth the trouble?

  • 4
    If they refer to the asker giving a "display prominence", I guess they meant a checkmark, not votes. – Jenayah May 22 at 17:43
  • 3
    Perhaps stack exchange should only take action insofar as reading the research thoroughly, considering it, discussing it. The fact that you want the research attacked reflexively is more of a problem than is the thoughtful consideration of the research and its merits. Perhaps there's a valid point to the research? Let's consider it? – Namaste May 22 at 18:09
  • 3
    The article isn't really criticizing the site or its users, the author argues that answers that you the second pronoun "you" are perceived to be less polite and therefore less likely to be upvoted. I've not noticed polite posts being more upvoted myself, but there have been users on EL&U who were confused with the impersonal or generic, pronoun "you" and thought they are being targeted on meta. basically, I think it's a cultural thing. – Mari-Lou A May 22 at 18:10
  • 6
    I think this is (yet another) example of the media just... getting things wrong because they don't fully understand what they're talking about. From other accounts I've read, and the study's abstract, the authors of the study seem to be very aware that they're comparing the overall score of the answers with which answer is accepted... I've not read it to know if they're accounting for things like accepted late answers or not... – Catija May 22 at 18:19
  • 5
    It's not entirely clear to me what you're after in this question. Is it the supposed mismatch between the article/study's statement and the referenced example questions? In that case, I'm not so sure the article is even trying to make that connection. Nor can they be part of the study, since they're only 2 days old. For what it's worth, it seems more like the author just went to stackexchange.com (i.e. the HNQ list) and picked two sufficiently cool/weird examples as "eye-candy". – Christian Rau May 22 at 18:20
  • 2
    I updated my answer after reading some of the primary author's dissertation. This looks like a case of bad journalism. – Jon Ericson May 23 at 5:11
  • @ChristianRau Well, yes, I think you're right that the examples were chosen for their fun factor. But for me, that on StackExchange one could receive a thoughtful answer to a question about the color of the mane of Legolas' horse, has value. There's also the insinuation of an inverse correlation between politeness and quality, i.e. that fruitful discussion can only be achieved by rudeness and that a polite exchange of ideas is necessarily shallow. – Elise van Looij May 23 at 10:16
  • 1
    @ElisevanLooij Your latter points might be viable criticism of the article or the study/results it references. It is, however, not in any way a "misrepresentation of SE", rather than merely a different viewpoint/opinion/research experience, on something that isn't factual to begin with. – Christian Rau May 23 at 11:05
25

Stack Exchange is a pretty popular source of papers I think because we make so much of our data available for free. (Anecdotally, I met with a group from a local university who outright told me that's why they used the network for research.) When people contact us to let us know they are doing research, we've been happy to help. But often people don't contact us and papers get written that don't quite grasp our system. In fairness, it's vastly more complicated than it appears at first glance. In the past I've tried to duplicate results, but often it's not worth the effort. In many cases, the paper isn't intended so much to understand the network as it is to further the authors' academic goals.

This particular case is complicated by the fact that we aren't reading the paper itself, but a summary written by a journalist. The two questions listed were not in the paper (because they weren't asked until 2 days ago), but just examples the journalist picked to illustrate their point. (Their point seems to be that questions on the network are trivial, I think?) Some journalists do a good job writing popular treatments, but others just don't. It's pretty unlikely that a journalist will understand Stack Exchange better than a researcher, however. There aren't nearly enough papers about the network for journalists to specialize on them. Correcting their misunderstandings is unlikely to help them write better articles about the network in the future if they never write about it again.

Ultimately, we can't make misinformation go away. But we can highlight good papers, produce our own research and encourage people familiar with the sites to use our data. There's also a role for our marketing team to promote a more accurate way for people to look at our products as we experiment with Stack Exchange advertising. Still the sites are complex enough you can pretty much count on mistakes and misinformation getting around. You just gotta pick your battles, you know?


I haven't read the paper, but I did find the primary author's dissertation online: "Essays on the Interaction between Users and Information Systems" by Shun-Yang Lee 2016 - The University of Texas at Austin. The author clearly does understand how Stack Exchange sites work and has some interesting conclusions. For instance, here is an excerpt from "Chapter 4—Is Best Answer Really The Best Answer?"

This implies that the design of answer display rules in current CQA platforms, if relying too much on the question asker’s own answer assessment, might be inadvertently sacrificing quality for politeness, which would negatively impact the community’s collective knowledge building process. As a consequence, even though question askers seem to be the most qualified individuals to determine whether the answers have successfully addressed their questions, the CQA platform should reconsider to what extent question askers’ choices of best answers should be used in determining the quality level of answers so as to improve the effectiveness of the platform as a whole.

To put it in practical terms, the research suggests that answers that use certain linguistic features are more likely to be accepted than answers that don't use those features. Examples of these features, according the the dissertation:

  • pronoun usage,
  • percentage of articles contained in the text,
  • length of the text, and
  • number of words with more than six letters.

According to Politeness theory, these linguistic features threaten or save "face", which is the asker's (in this case) public self-image. An asker might accept an answer not because it did the best job of solving their problem, but because the answer was not an affront to their feelings of self-worth.

Or to sum it all up: maybe we shouldn't pin accepted answers.

  • An alternate approach might be a combo of both (the one linked in the last line, and the one I advocated on SFF a long while back - specifically, ability for $community - however defined - to strip an accepted answer from accept flag if it was deemed by SMEs that it is simply wrong). Combining the two ideas together, perhaps a better more nuanced approach is warranted and plausible - allow SMEs/$community, to un-pin accepted answer, while keeping it accepted. – DVK Aug 12 at 2:08

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .