76

In the We’re Rewarding the Question Askers, which makes me wonder if the number of questions will increase (and as usual, the quality won't follow), it's mentioned (laconically paraphrased):

It was like that in the past [...]. Then it was changed to +5 for a question. This decision may have been the right call then, but we have seen the effects it has had on our community.

Which specific effects is the blog talking about?

  • 15
    Stack Exchange has listened to their moderators and chosen not to mention that point in the blog post. Could we please cut them (and us) some slack and avoid bringing that topic out ourselves? – Frédéric Hamidi Nov 13 '19 at 20:39
  • 5
    @FrédéricHamidi tbf, the details are (at least partially) leaked on reddit, and the post is still alive after 8 days. It might as well be stated officially, it's already out there – Zoe Nov 13 '19 at 20:45
  • 6
    Probably talking about hostility to new users. If you venture outside SE (reddit, 4chan, facebook, anywhere else), you'll notice that SE itself has carved an horrible reputation. Judging from outside discussions, the new user experience is just awful. We like to think of us as power users and quality vigilantes, but the truth is that we are perceived as toxic. – Von Huffman Nov 13 '19 at 20:45
  • 3
    @FrédéricHamidi I'm inclined to agree, but then again, if you don't want to discuss effects, don't mention effects. Just say "hey, we think good questions deserve as much of a reward as good answers, and here's why" ... – Bart Nov 13 '19 at 20:47
  • 15
    @Lyd perceiving users as toxic for enforcing rules of a site to maintain the quality and goal of the site is utterly idiotic. – Script47 Nov 13 '19 at 20:55
  • 22
    @Lyd: Veteran users have been trying to coax Stack Exchange to make changes that would make it easier for new users to navigate the sites better for years! – user102937 Nov 13 '19 at 20:55
  • 4
    @Script47 I don't think that perception comes from the control of quality itself, but from how some users choose to go about it. Your comment is a good example, ironically. – Von Huffman Nov 13 '19 at 21:05
  • 14
    @Lyd Eh. No matter where you go, enforcement gets a bad rap. Nobody likes the people who maintain the place. The big difference for SE is the expectations new users come in with, and that runs headlong into enforcement. We still have to clean up the messes; it'd just be nice if there was a sign at the door that asked you to take off your shoes before you track mud into the house. – fbueckert Nov 13 '19 at 21:08
  • 9
    @Lyd After being on SE for less than a year, I think I qualify as a "new user", but I've never found any hostility in being a question asker (aside for electronics SE). If you take the time do a little searching first to make sure your question is unique and thought out, then everyone here has been happy to help. – SurpriseDog Nov 13 '19 at 21:10
  • 1
    @fbueckert Yes, there's always bad rep for enforcement, but that doesn't mean that we can't do better, or that the current enforcement dynamics are optimal. – Von Huffman Nov 13 '19 at 21:18
  • 6
    No one's saying they are, @Lyd. Just that the perception of toxicity doesn't automatically mean the place is actually toxic. There are ways to fix it, and there's been...honestly, I lost count a while ago, let's say lots, of ideas on how to improve that. The majority of which are ignored in favor of these sorts of changes. We can do better. We all want to. But the solution isn't to bow to the crowd that's already written the network off, but try to teach those that come after what we're all about, better than we currently are. – fbueckert Nov 13 '19 at 21:22
  • 14
    @Lyd Soooooooo...what it your goal here, then? People care. They have an understandable need to express that. You can write off their opinions if you want, but I'm not sure what you're hoping to accomplish by telling the community they're toxic and ignoring any counterpoints I and others are making. I'd disagree, but there's no argument there to actually disagree with. – fbueckert Nov 13 '19 at 21:28
  • 3
    Some of the early leaks of the policy change pointed to diversity issues. I can't keep track of all the deleted posts (low rep). Can someone remind us of what the diversity concerns were, or confirm they're real? – Scott Seidman Nov 13 '19 at 22:14
  • 2
    @ScottSeidman See this comment thread – Rubiksmoose Nov 13 '19 at 22:18
  • 4
    @Lyd: Re "discussion I find in outside sources has us painted in a very dark light": There are a lot of those, but it is not difficult to find very positive stories. That being said, Stack Exchange ought to invent something qualitatively very different that addresses the expectation of a place for beginners and for learning in general (can not be in the Q&A form as we know it). – P.Mort. - forgot Clay Shirky_q Nov 13 '19 at 23:27
40

I don't understand why Stack Overflow Inc the company chose to hide away the reasoning behind this change, but a Stack Overflow moderator (Cody Gray) has shared a partial account of it over on Meta Stack Overflow. Here are some snippets:

This decision was made with the benefit of hindsight: Jeff's solution [halving rep for question upvotes from 10 to 5] seemed logical enough at the time, but after nearly 10 years of it being the standing policy, there is now enough data that we can look back and judge whether it was really having its intended effect. Sara unfortunately doesn't go into very much detail about this data-led re-evaluation in her blog post, presumably because it is intended for a more general audience than the folks who read Meta.

The fact is that the changes made back in 2010 simply weren't having the intended effect of improving question quality. Reducing the amount of reputation gained from upvotes on questions did little or nothing to address the problem of users earning reputation from low-quality questions. Worse, it merely served to make it that much harder for users who were asking useful, high-quality questions to earn privileges.

Data tells us plainly that voting skews heavily towards answers. Even if the net reputation gains are made equal (as they will now be), folks who write good answers will still be earning reputation faster and more easily than folks who write good questions.

It then goes to talk about how the company has sought to improve question quality in other ways:

...Recently, increased effort has been targeted specifically on addressing the issue of declining question quality, including some of the things that Sara calls out in her blog post: a wizard to guide users through the process of asking questions, improved post notices that do a better job of communicating why a question was closed, and better moderator tooling behind the scenes to deal with less-than-stellar contributions.

The logic presented there actually sounds quite reasonable.

According to a comment from a staff member who wasn't in the core team behind this change but was indirectly involved, there was also some work done involving surveys and focus groups, though it looks like none of the results of that will ever be shared:

I wasn't heavily involved in that project, but this was FAR from a unilateral decision executed haphazardly. What I could see was weeks (maybe months?) of discussion and outreach with satisfaction surveys, focus groups, and an extensive discussion with the Moderator representatives of your communities about if/how to roll this out. Maybe it was felt we couldn't crowdsource this decision and discuss improvements with thousands or millions of users every time; I don't know. Meta SE hasn't exactly been a constructive place to bring anything lately (which is understandable, but still...) – Robert Cartaino ♦


If this reasoning had actually been shared with us, and opened for any sort of discussion, I think my only concern would have been whether increasing the rep reward for question upvotes might lead to a decrease in upvotes on questions as upvoters raise their standards; leading to a worse experience for conscientious new users as even more reasonable-quality questions get no response whatsoever. That's a question that can be answered with data in a trial period.

I think the response to a post like this would have been cautiously positive:

Reducing question upvotes to +5 rep didn't improve quality: we intend to reverse it and boost quality smarter

Nine years on, halving the rep for question upvotes from 10 to 5 clearly hasn't reduced bad questions. Tools like the SO question wizard and improving question closing have been much more effective. We're planning on continuing the "pearls not sand" principle by focussing on those tools, and rewarding quality questions by restoring them to +10 per vote.

Here's the data behind this [link]. Before we go ahead with this change, is there anything we didn't think of?

I don't understand why this has been presented the way it has been: cooked up in secret, presented as if it's a rejection of the idea of question quality mattering that is being imposed unilaterally, when a good case for it could have been made that I think many users would have agreed with.

It feels like the leadership at Stack Overflow inc are now deliberately going out of their way to present things in a way which will anger and alienate their users. Why?

|improve this answer|||||
  • 23
    "I don't understand why this has been presented the way it has been" - because the actual reason, given in the leak, had no mention of improving the other tools. All appearances are that it's an after-the-fact justification created to hide the real reason. – Izkata Nov 14 '19 at 2:01
  • The change was shared with moderators before being implemented. So not really secret. – Raedwald Nov 14 '19 at 7:19
  • 12
    But multiple (two, possibly three, I don't know?) attempts to start a discussion about it publicly were deleted and the users who posted them penalised. I'd call that secretive behaviour. – user56reinstatemonica8 Nov 14 '19 at 9:00
  • 11
    @Raedwald According to the comments I got here: meta.stackexchange.com/a/338222/285661 ... the only thing that was shared and discussed was about the announcement process. The idea itself wasn't. HUGE difference. – GhostCat salutes Monica C. Nov 14 '19 at 9:38
  • 4
    @user56reinstatemonica8 SE decided to discuss the change with moderators only. A moderator broke the trust SE placed in them and posted a copy of the moderator only post. All those "discussions" were essentially copying that. They were repeating embargoed information. Of course they were deleted. – Raedwald Nov 14 '19 at 12:11
  • 3
    +5 reputation for questions didn't reduce the number of bad questions, therefore make it 10. That's not logical. Why not 15 or 20? Why not reverse or modify any or all decisions that didn't improve question quality? Saying that +5 didn't help doesn't explain why +10 will. An aspirin didn't work, so it's logical to assume that two will? Why? Sorry, the "reasoning" is just a veneer over a pandering giveaway. – Scott Hannen Nov 14 '19 at 12:26
  • 4
    @Raedwald The information shared in mod-only channels is not automatically privileged. If someone shares something about their private life, that's private. If there's a discussion about site mechanisms, who said what is private but the information isn't. Moderators discouraged from discussing site mechanisms in private (except for moderation mechanisms where revealing exact rules would make it easier for e.g. spammers and sockpuppetteers to avoid detection), and are encouraged to use meta instead. Disclosing information about reputation changes wasn't unethical, keeping it secret was. – Gilles 'SO- stop being evil' Nov 14 '19 at 12:45
  • I upvoted, but haven't accepted (yet?). Laconically, what you are saying is that officially, we don't know what specific effects let to that change of policy, right? Or maybe I missed something... – gsamaras Nov 14 '19 at 13:32
  • 1
    @Izkata See this comment thread. That was not the "real" reason for this change. – Rubiksmoose Nov 14 '19 at 13:59
  • @Raedwald "embargoed information" is information kept (temporarily?) secret. That's what those words mean. You can debate whether or not this was a legitimate case for putting an embargo on this information, but legitimate or not, if an embargo was placed on it, it was being held secret – user56reinstatemonica8 Nov 14 '19 at 15:24
  • Was the change also based on link.springer.com/article/10.1007/s10664-019-09685-x ? – mic Nov 18 '19 at 3:51
  • 1
    @mic There's a whole other question about that, meta.stackexchange.com/questions/338381/… – user56reinstatemonica8 Nov 18 '19 at 11:55

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .