1544

I feel like there's a problem with Stack Overflow, as the number of people prowling it increases.

Each question's answers are sorted by descending score and then by descending time of posting. This means that if a person sits down and answers a question in a long, thorough way, going through every nook and cranny, once they post their answer, it will already be one of about seven different ones, some of which have already been upvoted. This wouldn't be a problem if those answers were as thorough as the one this guy's posting, but they usually aren't. Some of them are downright wrong, some aren't even answers to the question asked because their poster didn't bother to read the question all the way through.

This causes a problem I like to call Stack Overflow's Fastest Gun in the West Problem. I've come to a point where I'd rather just send a short, simple, correct explanation, than to go and do some proper research, write a whole blog post about it or even make sure the code I post even compiles, just so it will be noticed, as opposed to the incorrect ones.

I'm sure I'm not the only one doing this and that it despairs many people from even trying to answer questions.

I've long ago learned to try to always raise a problem with a solution in hand, rather than just say "This is a problem, handle it," so my question, after this long tirade, is:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

I promise to vote on answers I like, even if it takes you a long while to post them. :)

Improve this question
11
  • 264
    create a quick answer, and then edit the detail in steps??
    – Jonathan.
    Commented May 24, 2011 at 19:39
  • 39
    @Jonathan. That's still too bad, we're always in a hurry.
    – Ramy Al Zuhouri
    Commented Jan 9, 2013 at 3:33
  • 4
    @RamyAlZuhouri, the time to click post and edit is insignificant compared to writing an answer that is long enough and thorough enough to need to click post and edit. (in other words if your in a hurry the answer your writing will be shorter and less thorough so you won't have to edit in the detail in steps anyway)
    – Jonathan.
    Commented Jan 9, 2013 at 13:28
  • 3
    @Jonathan. But the answers should be sorted considering also the edits then.
    – Ramy Al Zuhouri
    Commented Jan 9, 2013 at 13:46
  • 5
    I read the first two answers in descending score order (and their comments), then skimmed a bit more and grew weary of the topic. Are there great answers among the other 48 (as of now)? I don't expect I'll ever know. It's not in my nature to go hunting for under-appreciated answers, even though I've felt the sting of under-appreciation on S.O.
    – Griffin
    Commented Oct 2, 2014 at 14:14
  • 56
    This answer was practically identical to the accepted answer to the same question, but posted 10 seconds later. The accepted answer got 838 upvotes; the linked answer got 8 upvotes. There's no way in hell that the 8300 rep difference between what those answerers earned is fair or represents the value of their contributions.
    – Mark Amery
    Commented Nov 8, 2015 at 21:51
  • 3
    I posted a possible solution here meta.stackoverflow.com/questions/337187/…
    – Bradley Thomas
    Commented Oct 31, 2016 at 19:47
  • 3
    The rush to answer is because current scoring incentives are not temporally aware. The resulting dependence on the chronology of answers gives those who slight the question and rush to answer a statistical advantage. The skewing of reputation points based on temporal placement of events is a challenge in learning, the study of it, and the design of learning algorithms. The goal should be (to the degree possible) chronology independence. I tried to introduce that term here: meta.stackexchange.com/questions/291364/…
    – Douglas Daseeco
    Commented Mar 14, 2017 at 17:27
  • 4
    The problem with quick answer followed by edits that flesh out and elaborate the answer is that people are also quick to downvote an answer resulting in a quick, incomplete answer getting downvoted.
    – Richard Chambers
    Commented Oct 10, 2017 at 3:19
  • It's an interesting problem. On the one hand people coming back to the site to understand a problem will upvote the best answer. So the best answer will get more votes overall. However the person asking a question is often >< close to missing a deadline and getting sacked, so for that chap, he wants the quick and dirty solution, not a protracted discourse of the whole life story and history of C++. So I don't think anything needs fixing here. There are necromancer medals on stackoverflow for a reason!
    – Owl
    Commented Oct 25, 2018 at 16:06
  • @Jonathan. replying to your first comment; don't you think this is sweeping the problem under the carpet?
    – user718628
    Commented Apr 8, 2021 at 13:23

51 Answers 51

Reset to default
1
2
804

This problem was solved over 90 years ago. (See here for xkcd author Randall's explanation).

Of course, this answer is late, so it will never get upvoted :)

Update: There are now two user scripts that augment answer lists with Wilson scores: Sort Best First and Wilson confidence rating calculator. You are welcome to try them in order to assess how much of an improvement it would be to switch to sorting by Wilson score.

[Edit] Layman summary: we want to determine what the upvote-percentage would be if everyone voted. But since we only have a small sampling of votes, we use fancy statistics to determine a range of percentages we can be fairly certain the real percentage falls under. We take the lower end of that range, to err on the side of caution.

Here is the output of the equation. You'll notice that when there are many votes, the output is close-ish to positiveVotes/totalVotes, but when there are few votes it's much smaller. This is exactly what we want.

Here is some code:

///<summary>
///Returns a rating for the given post.  Larger is better.
///Based on the equation found at http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
///</summary>
public double GetPostRating(int numPositiveVotes, int numNegativeVotes)
{
    int totalVotes = numPositiveVotes + numNegativeVotes;
    if(totalVotes == 0)
        return 0;

    const double z = 1.96; //Constant used for 95% confidence interval in a p-distribution
    double positiveRatio = ((double)numPositiveVotes)/totalVotes;

    //Crazy equation to find the "Lower bound of Wilson score confidence interval for a Bernoulli parameter"
    //Again, see the above webpage
    return (positiveRatio + z*z/(2*totalVotes) - z * Math.sqrt((positiveRatio*(1-positiveRatio)+z*z/(4*totalVotes))/totalVotes))/(1+z*z/totalVotes);
}

Note that the above equation assumes upvotes and downvotes have the same frequency. Since upvotes are way more common, downvotes should ideally be weighted more harshly (in other words, three downvotes says a lot more about an answer than three upvotes).

Also, I believe newer answers should be given preferential treatment, at least for a few minutes (see my comment below).

But even without these, this is a neat improvement.

Improve this answer
20
  • 24
    +1 infinity, although the formula would have to be modified to take into account the pinned "correct answer" somehow. Even Randal of XKCD fame is behind on this one!
    – Gleno
    Commented Aug 2, 2011 at 22:36
  • 88
    "newer answers should be given preferential treatment" - I should explain. When people are racing to answer a brand new question, the answers posted a few minutes later tend to be slightly longer but higher quality; but, by then the early answers have gained too much first-five-minutes-momentum. If you add a factor of say, someConstant/secondsSinceAnswerWasPosted to the above rating, this would give the later answers a precious minute or two at top to gather some votes of their own. And usually when someone an older question it's because none of the existing answers were satisfactory.
    – BlueRaja
    Commented Dec 10, 2011 at 0:19
  • 19
    @BlueRaja: mostly right, but I've seen questions with accepted answers still getting answers a year later, often very poor answers from new users who don't yet know The Ways Of Our People.
    – sarnold
    Commented Dec 10, 2011 at 0:28
  • 13
    @sarnold: In that case, the new answer will be conveniently at top for us to flag (or upvote). It will only be there a few minutes though, so it will likely be only the people who spend all day hitting F5 on the "active questions" page who see it at top, not the people who actually need to find an answer to that question.
    – BlueRaja
    Commented Dec 10, 2011 at 0:38
  • @BlueRaja: Ha! Good point. :)
    – sarnold
    Commented Dec 10, 2011 at 0:40
  • 6
    Statistics FTW! However, this would require that the cost of downvoting is more or less the same as the cost of upvoting. Currently you get points for upvoting, and lose points for downvoting. That will seriously screw up the statistics (although there might be a way to adjust for it).
    – naught101
    Commented Feb 6, 2012 at 6:58
  • 2
    @naught101: This is completely unrelated to a user's reputation, it's only for sorting answers on a page.
    – BlueRaja
    Commented Feb 6, 2012 at 17:03
  • 81
    Yes, but the facts that a) you require more reputation points to downvote, and b) you lose reputation points for downvoting (and gain them for upvoting), means that down votes are much less likely, and are therefore more meaningful than upvotes (people probably put more consideration into their down votes than their upvotes - I know I do). The above equation doesn't take this into account at all, although it could, if you weighted the downvotes higher than the upvotes (eg. by total votes on all sites/down votes)
    – naught101
    Commented Feb 7, 2012 at 7:06
  • +1, even though I strongly disagree regarding up/downvote weights (I like them as they are). Note that the confidence score statistical method is completely independent of the process that generated the data (actual voting).
    – Jirka Hanika
    Commented Mar 11, 2013 at 10:06
  • 7
    This would be a perfect solution, but unfortunately I feel people are put off downvoting due to the negative reputation it gives - especially new users with low reputation.
    – James Donnelly
    Commented Mar 22, 2013 at 10:04
  • 4
    I think that "negative" votes should be number of view of that answer. In long run every answer will have similar number of views. This will create 3rd option for voting: Great answer +1, ok answer 0, Wrong answer -1. Down vote should have bigger weight, and numbers of views less weight.
    – Yankes
    Commented Jul 25, 2014 at 15:27
  • 4
    upvoted even if your maths is poor. 1927 was 84 years ago at the time you posted (presumably 2011). Now 87..
    – CashCow
    Commented Oct 2, 2014 at 15:33
  • 23
    A problem with this formula is that, for posts with no upvotes, it always returns (or should return, if there were no round-off errors) a zero score regardless of the number of downvotes. (In fact, numerical errors may cause downvotes to slightly increase the score.) A better choice could be to use the midpoint of the 95% Wilson score interval instead of the lower bound, which also leads to a simpler formula: score = (upVotes + z*z/2) / (totalVotes + z*z). As a bonus, for high vote counts, this actually favors uncontroversial posts a bit more than using the lower bound would.
    – Ilmari Karonen
    Commented Mar 12, 2015 at 19:53
  • 5
    I have cast thousands of upvotes, but not a single downvote. I've thought about it a few times but usually the answer had been downvoted already so I didn't feel good about downvoting it even more. What you could do instead of comparing up and down votes, is taking into account how many people have upvoted an answer, in comparison to how many people have read it. I wrote a blog post about that, debiki.com/-9qb49/…
    – KajMagnus
    Commented Jun 15, 2015 at 11:04
  • 14
    Of course, this answer is late, so it will never get upvoted :) Current score: 497. Did you just disprove OP's point?
    – Mast
    Commented Sep 3, 2015 at 18:33
390

I do NOT want to, in any way, discourage the quick and dirty answer.

I've asked questions that have received an immediate answer with enough information to get me past my block, but not served on a platter as you propose, with all the information I might need.

This is extraordinarily helpful - I know I can post something on here, day or night, and get an answer within minutes, often seconds, that will be better than searching through books, online, etc, even though it's a throw-away post that took someone 35 seconds, who may have only answered for the reputation. If it's at all helpful, I upvote it.

Later on I'll review my question again and select the answer that is what I consider the best of the bunch (I try and wait a day or so for long answers like you provide).

But I DO NOT want to 'adjust' the system to discourage short quick answers, or make long thoughtful answers that come a day late so highly valued that no one posts the short ones anymore.

Further:

Let's look at it from the perspective of what the site is supposed to be:

A long term repository of searchable, free, and freely editable information.

In the long run, it doesn't matter if the long, good answers don't get the initial rush of votes. They will ultimately go to the top because

  • There are generally few answers per question, so future searchers will read the entire post and may upvote the better answer. The special case of the question with tons of answers is just that - a special case, and it will be gamed and mined for rep a dozen ways from Tuesday, so it's not worth fixing - it's less than 1% of total content - don't design the system around the special case, or else all your cases will become special.

  • Once someone posts a new answer, the post gets a new chance to be on the front page, which will entice some more people to see it that haven't - they'll upvote the good, long answer at the bottom of the page if they feel it's better than what was initially offered.

  • The person who initially asked the question will often come back later and review it, and if your post is truly the better answer they'll select it as the new accepted answer.

A key point, though, is that if someone is looking for the information later, they'll read all the answers unless it's obvious that the first one is good enough for their purposes.

So that completely guts your argument that it's important that the longer, complete answers should get a boost to the top, somehow. The only reason left to complain about the tradeoff between short posting and long posting, then, is because you may not get as much of the initial rep as you would otherwise.

Improve this answer
24
  • 41
    I disagree too. Even in beta there are enough new questions that old answered questions get rotated off the front page. Most of your argument assumes that the person who asked the question performed due diligence to maintain that question. I don't think that happens as much as it should do.
    – tonylo
    Commented Sep 11, 2008 at 17:49
  • 5
    tonylo, the main problem being discussed here is that there's a lot of initial activity on a question, most of which makes spending time on a great answer less advantageous. So the issue is really only with questions that will stay on the front page and deciding whether to make a great post or not.
    – Pollyanna
    Commented Sep 11, 2008 at 20:13
  • 29
    Taking your stance as valid, I think there is still an issue whereby most question-posters will not come back a day+ later to upvote/select the best answer. If there was some [more obvious / greater] incentive to do that then I could agree with you more completely.
    – Sparr
    Commented Sep 20, 2008 at 2:07
  • 6
    The bounty system is working very well to give the detailed long answers a chance for great rep increases...
    – Pollyanna
    Commented Jan 28, 2009 at 17:19
  • 93
    I disagree. As to your key point, I don't think people who come much later read through all the answers and upvote the good ones. A more likely scenario is that they go through the first couple of answers, because in the mind of that person, the first couple of answers probably a) are the best b) contain 80% of the information one would get by reading all of the answers, which might take considerable time. That person is not likely to review all the answers, even if the first couple do not contain all the details he cares about. Would you start from the middle or end of the list of answers??
    – allyourcode
    Commented Mar 8, 2010 at 2:12
  • 49
    In that case, the quick and dirty answers that start at the top tend stay there, and the carefully thought-out ones tend to stay near the bottom. You can't dismiss this by saying "when that happens, it means the quick and dirty answers are deserve to be on top". The problem is that quick and dirty answers benefit from a positive feedback loop: the higher an answer is in the list, the more likely someone is to view and upvote it. Long and thorough answers, on the other hand, have to fight an uphill battle in order to gain attention, a prerequisite for upvotes.
    – allyourcode
    Commented Mar 8, 2010 at 2:21
  • 12
    @allyourcode - but if the quick and dirty answer is right and sufficient to resolve the issue, then who is to say that the longer, more ponderous message is righter or sufficienter than the quick and dirty answer? Why do people hold so tightly to the idea that a longer, perhaps even more complete answer, is better than the shorter answer which also resolves the problem, and has come earlier to boot? Keep in mind that we're not talking about wrong answers. Both are right.
    – Pollyanna
    Commented Mar 8, 2010 at 15:02
  • 4
    @allyourcode - further, my own experience shows that longer posts which come later still receive significant upvotes over time. Otherwise the necromancer badge would never be awarded. People still vote on posts I made 2 years ago. Your assumption that people don't go back and upvote posts, even shown way down on the page, is flat out wrong. But don't take my word for it - examine the data dump and draw your own conclusions.
    – Pollyanna
    Commented Mar 8, 2010 at 15:09
  • 5
    It's not necessarily about "length". No one is suggesting that word or char count be used as the sort criterion. As for "this is sufficient, therefore it should win", you're assuming that every answer that reaches some threshold of completeness is just as good. I completely disagree.
    – allyourcode
    Commented Mar 15, 2010 at 21:11
  • 8
    re Pollyanna's second point: I do not doubt that people look at all answers to old questions occasionally, and upvote the better ones, but I think that's rare. One of the points that I'm trying to make is that there's alot of room for improvement when the desired outcome is requires a whole series of rare occurrences.
    – allyourcode
    Commented Mar 15, 2010 at 21:16
  • 12
    Even though I encourage quick and dirty answers, better (and later) answers should triumph over quick and dirty ones...in the long run. But "in the long run" is not good enough for right now. The underlying problem is that questions with one up-voted post are considered "answered". Early upvoted posts build momentum by being shown first and allot the question as "answered", discouraging delayed higher-quality posts which would provide better information to all future viewers, but which can never build the same momentum as the first quick and dirty answer.
    – Petrus Theron
    Commented Feb 11, 2011 at 18:29
  • 5
    @Tom - but a brilliant answer released at 11am is worth more than a bad answer at 10:45am.
    – Martin Capodici
    Commented Jul 31, 2014 at 22:11
  • 6
    I would even say a good answer posted after 4 minutes is worth more than a placeholder posted after 3 minutes (which got edited to a mediocre answer after 11 minutes and accepted as the fastest after 15 minutes.)
    – Robert
    Commented Aug 5, 2014 at 6:56
  • 3
    Part of the problem here is that the placeholder answers appear within the grace period as the "first legitimate answer". The logical reasoning of the questioner would dictate that given two roughly similar answers, the first one posted should be accepted. However, often times, the first one posted wasn't the first to actually answer, and either edited later or even plagiarized another answer. There's nothing wrong with quick & dirty answers - if they actually answer the question.
    – crush
    Commented Aug 27, 2014 at 18:37
  • 2
    "Quick and dirty answers" often serve the needs of the asker all the more specifically because they're quick, definitely we don't want to discourage them. But more complete answers often do better for the "long term information store" purpose of stack exchange, because later viewers rarely have exactly the same problem as the original asker. A more rounded answer is more likely to help people with related-but-not-identical problems.
    – Ben
    Commented May 10, 2017 at 4:55
315

Grace period idea (currently editing...)

One solution would be to have a grace period where votes are hidden:

  • The grace period begins when the question is posted and lasts for some set length of time.
  • During the grace period the following is true:
    • Sorting by votes is disabled (questions are displayed in a random order if 'votes' is chosen)
    • Votes from other users are invisible meaning...
      • At first all answers appear to have a vote of 0
      • Any answer that you vote on will display your vote only (+1 or -1)
  • When the grace period ends, all votes which were hidden will be revealed.

Maybe the grace period could be the same as the grace period for editing the question.

@Jeff Atwood Sounds complicated

No more complicated than...

  • A grace period in which your edits don't count as real edits.
  • Hiding the number of Offensive flags from some users and not others.

All you need is a simple flag that switches between two modes of the question: "Grace Period Mode" and "Regular Mode". You must already have this because of the grace period for counting edits on questions.

Will people understand what is happening?

Yes, if you explain it to them. Let's say we name this grace period "Secret Voting". All you have to do is put a banner of some sort above all the answers that says "The Voting is Currently Secret". When the grace period expires, the banner would be removed.

Also during the grace period, the Votes sort tab would not be present, but instead a Random sort tab would be present. Again, people will understand it, because the interface explains it to them and doesn't make them think.

Improve this answer
19
  • 62
    I really like this idea - partly because I only upvote when the answer seems lower than it should be. With votes not visible for the question's first appearance on hte homepage (until it falls off, maybe?) you'll get a much more interesting, and possibly truthier, voting.
    – Pollyanna
    Commented Sep 12, 2008 at 2:20
  • sounds really complicated -- will people understand what's happening?
    – Jeff Atwood
    Commented Sep 12, 2008 at 8:40
  • 3
    Jeff, I think if you add some visual clues or simply descriptive text on question page, people will understand. IMO it's not that complicated.
    – aku
    Commented Sep 12, 2008 at 8:50
  • Agreed. See my updates to the post addressing Jeff's objections.
    – Justin Standard
    Commented Sep 12, 2008 at 18:02
  • 5
    I like this idea - except. It doesn't yet address the problem where a good answer is added a long time after the question is no longer hot.
    – Richard Corden
    Commented Sep 19, 2008 at 13:28
  • 13
    Disallow marking an answer as "correct" during the grace period as well. This increases the chance that the "correct" answer isn't just "the best answer to arrive in the first 10 minutes." I envision a secret-voting period of, say, an hour, but the editing grace period is 10 minutes -- too short.
    – Rob Kennedy
    Commented Dec 10, 2008 at 17:00
  • @Rob I like your idea. It encourages the questioner to return back to the site and accept an answer. But it also have a risk of questions with no accepted answers.
    – blntechie
    Commented May 28, 2010 at 6:49
  • 6
    @blntechie we already have lots of questions with no accepted answers
    – jcolebrand
    Commented Dec 22, 2010 at 14:27
  • 6
    I think this is a great idea. @Jeff any chance of this being considered?
    – David Tang
    Commented Feb 3, 2011 at 23:55
  • 2
    @box9 we already show answers randomly by vote, so the important part of this is implemented. The rest, well, it's been 2 years.. so.. :)
    – Jeff Atwood
    Commented Feb 4, 2011 at 0:13
  • 45
    @Jeff, true, but I don't see the random sorting feature really getting to the meat of the problem. The secret voting idea (with the banner), however, will really encourage people to vote more and more objectively.
    – David Tang
    Commented Feb 4, 2011 at 0:18
  • 7
    Another +1 for hiding votes for a period. The clock should start when the first answer is posted, not when the question is posed. The problem with random display of equal weight votes is that the first and likely incomplete answer will get an up vote and then subsequent ones will all randomly sort after the first one. Then the first one, with it's one vote keeping it first, can be edited into a better answer.
    – Josh Heitzman
    Commented Nov 14, 2012 at 4:58
  • 6
    Why hasn't this been implemented?
    – canadianer
    Commented Jan 1, 2015 at 19:55
  • 7
    This would make people look at votes without the huge bias of say +7 or -3 next to it. They would evaluate the answers themselves for quality. Perhaps in general there could be an option to hide votes by default and show them on a click.
    – intcreator
    Commented Aug 1, 2015 at 1:21
  • 4
    I like this idea, but I have a few suggestions to improve it. First of all, I think that users should always be able to see all votes on their own answers. Second of all, during the grace period, I suggest that answers can be sorted from newest to oldest, so that each answer gets a lot of attention right after it's posted, and therefore all answers get about as much attention.
    – Donald Duck
    Commented Oct 2, 2017 at 14:48
136

I've noticed myself reacting to this too, although I try to mitigate against it as much as possible.

I sometimes try and post a short correct answer as quickly as possible to kind of "mark" the question with my answer to discourage other similar answers. I then I would edit my answer to improve it, using the 5 minutes grace period to edit without it appearing as an edit (although I'll still carry on editing after then if I can improve it, I just use the grace period to edit without worrying about writing edit descriptions).

Conversely, if I find someone has answered the question while I was writing my answer, and I don't feel my answer adds anything, then I've deleted my answer on occasion, although if I provide a link and the previous answer doesn't I'd leave mine. Once I can edit I'd be inclined to remove my answer and add the link to the other answer. Since comments were added I've also been adding to answers by commenting on them, again, once I've got edit rights I might edit directly for simple improvements (adding links, spelling, grammar, formatting, etc.) and just use comments for actual comments.

I've also learnt a few of the keyboard shortcuts for the edit box (hover over the buttons to see them on the tooltips.) The shortcut I use the most is CTRL+L to add a link to selected text. That makes it much quicker to write a quick answer with a few useful links in it.

I've also taken the attitude that there is also a long game to be played. If you can provide a better answer then the initial rush of points that the first answers got is possibly just the tip of the iceberg and there's probably much more points "underwater" to be made once the site is indexed by Google and we start finding answers to questions in Google search results. (although the fact that new users can't vote straight away - you need +15 rep to upvote - might mitigate against this.)

Update: for a really good description of how the long game will play out and why the fastest gun answers aren't a problem see Adam Davis's Answer.

Improve this answer
7
  • 4
    That's exactly what I do, too. However, once you hit submit the 5-minute clock starts ticking, so your short answer only gets as good as you can make it in 5 minutes.
    – Kyle Cronin
    Commented Sep 11, 2008 at 14:13
  • I just carry on editing, take a look at the edit history for this answer.
    – Sam Hasler
    Commented Sep 11, 2008 at 14:21
  • 10
    the link is broken in here.
    – Rosinante
    Commented Dec 14, 2009 at 18:55
  • 1
    Hmm, it doesn't do a good job moving posts between stackoverflow.com and meta, you lose the edit history and it doesn't preserve inter-question links.
    – Sam Hasler
    Commented Dec 14, 2009 at 20:21
  • I answer a lot from my mobile device. This does not allow for a lot of citing or code examples. I often will have to revisit the answer once I get to a pc so I can elaborate or cite sources.
    – rlemon
    Commented May 24, 2012 at 17:33
  • 1
    The problem with the long game is that it requires a questioner who isn't so impatient as to accept the first answer, and abandon monitoring the question.
    – crush
    Commented Aug 27, 2014 at 18:47
  • “I sometimes try and post a short correct answer as quickly as possible to kind of "mark" the question with my answer to discourage other similar answers.”—The fastest territory marker (in the West).
    – Guildenstern
    Commented Sep 29, 2023 at 12:42
103

The reputation system is valuable, and I find myself emotionally validated by it a bit, but I also have to keep reminding myself that it is an illusion.

There are likely more points to be gained in the first 24 hours of a question than in the whole rest of its lifetime, because a lot of fast guns will be searching the unanswered questions, looking to build rep. And because the person who asked the question is likely to choose an answer and mark it "correct" within this fairly short time, while the question still matters to him. So, for those who see the Stack Overflow site as a game where building reputation is how you 'win' ... yeah. Fast guns will 'win'.

But a lot of Stack Overflow users will be playing a different game. For them the 'win' happens when they get an answer to the problem they have right now. The people playing this game are probably much less likely to even notice the reputation game. Many of them will probably not even up-vote the answer which works for them. Long term, I wonder what the stats will show: which percentage of asked questions never get an answer chosen by the question-asker?

And stats can't really reflect this, but how many people will get an answer without ever posting a question? They may find their answer via Google, or by starting to type a question and the site's own search engine flags a similar question/answer - one that's good enough. Will they bother to vote? Heck, a lot of them won't even be able to - because they haven't gained enough rep to vote. They never started playing the rep game.

So, an answer which never even gets rated might be the one that's really solving the most issues for this (potentially huge) group of people who never play the rep game, or never even notice it. That's the illusion: the Illusion Of Perfect Feedback. You're never going to get perfect feedback. So in this sense, the ordering of the answers will never be perfect. It is possible that it will never even be very good. Because it cannot be.

So, fast guns, slow guns, medium guns: write the best answer you can, no matter how long it takes, and realize that even if you never get a single point on the Stack Overflow Rep-O-Meter, you may be helping a lot of people.

I'm not dissing the rep system or the fast guns. They kickstart the process, and that's valuable!

Improve this answer
4
  • 3
    i think the fact the question searching appears to be so unreliable compounds the first point you make - questions seem to get lost in the melee...
    – flesh
    Commented Dec 10, 2008 at 18:59
  • 2
    Maybe asker should be able to vote on answers.
    – Joshua
    Commented Apr 17, 2009 at 22:30
  • 7
    Actually rep is not an illusion. Rep determines what you can do on the site, it determines not only how much influence you can exert, but also how much you can contribute at all. I cannot even disagree with your answer via downvote here on meta, because of this alleged illusion, simply because I posted something unpopular and on meta "downvotes mean something different", although the fact that I cannot disagree with your answer shows that to be false as well.
    – Joshua Drake
    Commented May 30, 2012 at 13:38
  • 3
    If you search and find the question and the answer to it you want you should upvote both of them
    – CashCow
    Commented Oct 2, 2014 at 15:37
52

I think that this site has to become more like a wiki where many people add small contributions to existing answers instead of creating their own new answer. Unfortunately the reputation system as it is now doesn't really encourage this yet.

I understand from listening to the podcast that this is the direction the creators of Stack Overflow want to go too. Planned features like making questions and answers community editable would be a big step in the right direction.

Improve this answer
4
  • Questions and answers are community editable, but you have to have significantly contributed to the community before you can so so (3,000 reputation) -Adam
    – Pollyanna
    Commented Sep 11, 2008 at 13:22
  • It only requires 2000 rep to edit answers, not 3000.
    – Chris Upchurch
    Commented Sep 11, 2008 at 16:10
  • 5
    You're right. Although I ran into interesting interaction today - I deleted comments between I and another user that had no further use in the topic, and they tracked down my email and called me a censor - expecting that our off-topic conversation would last on Sof forever. People won't like edits.
    – Pollyanna
    Commented Sep 11, 2008 at 20:15
  • 3
    People don't like other people who judge, decide upon or have the authority to change/delete what they have done.
    – Bora
    Commented May 18, 2011 at 5:59
43

Remove visible scores.

Yes, you hear me right! It feels horrible knocking someone down, especially if you take them into the -1 or less realm! If the scoring system is hidden and you just say "Yup like it" or "Nup not good" and the actual effect that it has is hidden then people will be more likely to rate answers (especially down) and the chaff may remove itself more effectively.

A second solution would be to not allow ranking until a question had been open for a while (an hour say) then all questions have an equal chance (if you display them in a random order to different users all the better).

Improve this answer
4
  • 8
    I really like the random ordering among equally rated answers.
    – Caspin
    Commented Jan 5, 2009 at 20:17
  • 1
    Hmm... I like the idea. This is going into my Greasemonkey cache.
    – new123456
    Commented Nov 5, 2010 at 22:22
  • 16
    But the whole point of visible scores is so that the Google-searchers know which answers to trust...
    – Kate Gregory
    Commented May 12, 2011 at 13:11
  • 7
    @KateGregory So just hide them during the first 24 hours while the dust is settling.
    – endolith
    Commented Oct 25, 2017 at 14:04
33

A related problem is the fact that when a question already had 10+ answers, an answer with new information given at this time tends to get lost in the clutter in the bottom and not get the exposure it deserves.

One possibility would be to position an answer based on both its rating and its "age" (for lack of a better word), giving recent but already voted answers a place before older answers with a similar rating.

Improve this answer
5
  • choose "newest" instead of "votes" ordering to view answers
    – David Schmitt
    Commented Sep 11, 2008 at 10:01
  • 1
    But it's not the default, nor - necessarily - the 'best' way of viewing answers. The types of answers referred to will still suffer bias.
    – Bobby Jack
    Commented Sep 11, 2008 at 10:09
  • With "votes" ordering the secondary sort is newest first; so newer questions appear above questions with the same number of votes.
    – Sam Hasler
    Commented Sep 11, 2008 at 11:20
  • @ Sam - is the secondary sort really the newest? Or can you set it up to be?
    – Richard Corden
    Commented Sep 19, 2008 at 13:25
  • Or have one answer highlighted below the accepted answer, chosen at random from the low-voted answers to get it more up or down votes.
    – endolith
    Commented Oct 25, 2017 at 14:04
22

I think macbirdie is on to something. I would recommend a slight enhancement, though:

  • for questions that have no answer marked as "the answer" by the question-asker, sort by "oldest" as the default
  • for question that do have at least one answer marked as "the answer," sort first by "the-answer?" then by "votes", then by "newest" (the current scheme)
Improve this answer
19

I agree, and fully admit to being guilty of this myself, on occasion. I have, thankfully, seen such 'answers' being modded down, but the level of this occurring varies hugely from question to question - some 'bad' answers are fairly quickly modded down, others are left, possibly, indefinitely.

I think the point about wiki-editing is significant. If the barrier was much lower, we'd get better quality answers, and people would also be less keen to run around, scrabbling for rep. I think Jeff's reservations about this are slightly too great - wikipedia has proven to work without such draconian restrictions.

I also think it would help a lot if up-voting also cost, just like down-voting does. Currently, there is nothing to discourage people from rushing around, blindly up-voting everything they come across.

UPDATE: I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ...

UPDATE: To clarify the delay concept: this would only apply to answers (but all answers), and could either be a 'reasonable' time to mask 'first posts' (e.g. 10 minutes), or could be dynamic based on number of answers, number of views, etc. "InSciTek Jeff" had a great idea relating to 'blind' votes (see question comments).

Improve this answer
3
  • "Currently, there is nothing to discourage people from rushing around, blindly up-voting everything they come across." other than the 30 votes per day limit.
    – Sam Hasler
    Commented Sep 11, 2008 at 10:00
  • Yeah, I must admit I rarely run into that. It's still quite a lot of potentially 'random' up-votes for all but the most obsessive ;)
    – Bobby Jack
    Commented Sep 11, 2008 at 10:08
  • 4
    I like the "I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ..." Jeff nuked an idea i posted to uservoice regarding a delay period for vote to be shown. "A pathology of its own", he said.
    – Stu Thompson
    Commented Sep 14, 2008 at 15:12
19

Many comments here refer to the wikification of the answering process: Merging comments, editing each other's comments, etc.

As far as I see it, we're talking about two different types of sites here: Collaborative and Competitive.

  • a Collaborative site is one where the strive is to reach the correct answer for the sake of the correct answer, without regard for selfish advancement. Everyone selflessly adds input, comments on each other's input, etc.
  • a Competitive site is one in which each contributor's work is measured and compared to others'. This is the approach which brings more user involvement (who doesn't want another badge?), but has some other downsides to it too.

SO is a competitive site, rewarding each of its members for their individual effort, so I'm looking for answers that go more along those lines.

I don't pretend to try and change the whole perspective of how the site operates to being a collaborative site.

Improve this answer
3
  • 2
    It's both. So users can get whatever intrinsic or extrinsic reward they want for answering questions but the askers still get their answers.
    – Sam Hasler
    Commented Sep 11, 2008 at 12:03
  • 4
    +1 Excellent point. It cannot be collaborative because we do not have joint goals beyond are own; and the points proces is clearly about personal points only,there are no points for collaboration,for getting to the most correct answer.
    – PerformanceDBA
    Commented Dec 12, 2010 at 13:55
  • @PerformanceDBA There are points for collaboration. Maybe there should be more.
    – endolith
    Commented Oct 25, 2017 at 14:06
18

You could let old answers "fade away", unless someone votes on them and they are not one of the top n answers. Hacker News uses this technique to gray out downvoted comments so you can ignore the bad answers and focus on the good ones.

But I guess the better way would be to change the culture of Stack Overflow:

  • If you ask a question, you should own the answering process. Vote, comment and clarify heavy on the answers. People will see that you care and keep answering even on older questions.
  • As a reader, use voting more deliberately. Especially the downvote-button is useful to push new answers above inaccurate old ones.
  • Too many rules are confusing and rarely the solution to more than one specific problem. Provide a framework with few, simple rules and let the community do the rest.

Yes, I know: if we don't have all these safeguards in place, the barbarians will burn down the town, once we open up the gates. But a lot of the rules are there to make things nicer, not safer.

Improve this answer
3
  • 6
    Just a preference, but I think downvotes should only be used for incorrect or misleading answers. Upvotes should be used to push the cream up, and ownvotes to push the rotten items down, but I wouldn't want people to downvote a question when it applies just because it's not as good as another...
    – Pollyanna
    Commented Sep 11, 2008 at 14:51
  • True, that is probably what jeff had in mind, but if downvoting had no negative effect on your reputation you could use it that way. Btw: 800 views and only ~60 votes on the answers? Are they all bad, or does the general user only vote on really good stuff? Why not encourage them to vote a lot?
    – sdfx
    Commented Sep 11, 2008 at 15:02
  • 4
    Once we get out of beta yo'll see far less than 10% of the viewers voting. It would be great if everyone voted often for all good answers, but it's either human nature or culture that we really only vote up the best one if it's currently an underdog, or really exceptional - votes are scarce. -Adam
    – Pollyanna
    Commented Sep 11, 2008 at 15:22
15

I completely agree. The pattern of a lot of questions and answers almost appear to look like reputation trading rather than providing quality information. Obviously, this is an anecdotal opinion.

The system does appear to reward incomplete but quick answers.

Improve this answer
2
  • 6
    ...and does not reward those who think an answer out completely.
    – Stu Thompson
    Commented Sep 14, 2008 at 15:14
  • You're right, and I've flagged this as a "quick and incomplete answer" (that should have been a comment instead).
    – ivan_pozdeev
    Commented Aug 30, 2016 at 18:16
14

One thing that could possibly help in this case would be to force the "Oldest" sorting mode for the question owner so he goes through all the answers.

Improve this answer
1
  • 5
    i raised a user voice for exactly this, much like a forum, there is value in being forced (as much as you can force anyone) to read the answers people have provided rather than lazily upvoting the top answer because it's there..
    – flesh
    Commented Dec 10, 2008 at 19:01
14

Following on from Omer's distinction between a Collaborative site and a Competitive site...

It would seem that we need the best of both worlds. There's nothing wrong with a bit of healthy competition so that people strive to produce timely, high quality answers. However, we don't want the answer to be lost between several people's contributions. Certainly marking answers up and down helps the good stuff filter through to the top, but if there's lots of good answers you really need a combined view.

I suggest the question owner take on (at least initially) driving the collaborative element and start the 'wikificiation' process of combining the best bits of several answers into (drum roll) the answer.

The question owner can do this today by adding another answer, but really it's an additional feature that should be put into the tool.

I'd say that most (90%?) of questions wouldn't need this extra step, but where there are several high quality answers having a single point that draws them together would be excellent.

Improve this answer
14

Get a grip. The idea of the site is to get answers to straightforward questions, and then get back to work. If one asks a question, one is going to be looking for what one thinks is the right answer, so one is going to read all of them.

  • Before Stack Overflow: I'm looking at the entire internet full of content that isn't remotely relevant to my problem.

  • After Stack Overflow: I'm looking at an unordered set of answers to my specific question, written by real people, and one of them is probably a nudge in the right direction.

Improve this answer
11

I agree to a point, although I have to guess that the site has not even come close to settling down. Users are still in a mad panic, scrabbling for reputation points and badges of any shade - what you are describing is bound to happen... initially.

It will calm down.

Having said that, I've noticed a pleasant tendency for people to vote for the actual best answer, most of the time.

Despite all the bilge posts, it seems the cream does often rise to the top!

Improve this answer
11

I think that a time buffer is a solution, but the buffer should, instead of buffer the appearance time of the answers, decide when people can vote. Answers will appear as they are being written, but votes can be issued after a half hour or ten minutes, whatever.

Given that you won't be upvoted in that period of time you know you can take 10 minutes or a half hour to prepare the answer.

The saddest part of it all is: What is reputation actually good for? I mean, besides a little ego trip? I'm as guilty as the next guy, but it's still sad.

Improve this answer
3
  • 2
    Yes, rep is definitely a head trip. But it's also tied to rights, and there are obviously a lot of users who are eager to contribute and make the site better. I was so excited when I could retag! Yes, it is sad.
    – harpo
    Commented Sep 11, 2008 at 14:29
  • 1
    @harpo +1 people on meta often seem to forget that rep is more than gold stars.
    – Joshua Drake
    Commented May 30, 2012 at 13:34
  • 1
    -1 You should be able to vote as you read the answers, or else you'll forget to vote. But for a certain time period, the votes shouldn't be visible to anyone else, and shouldn't affect the order of the answers, to make sure they get a fair amount of votes.
    – endolith
    Commented Oct 25, 2017 at 14:09
11

I think that it works ok the way it is. What is the problem of 10 short answers BEFORE yours if yours is good? In the end, people will hopefully vote yours up.

Out of my mind a few things thay may help are:

The asker may have the ability to "hide" certain useless answers, or the ability to MERGE two (or more) answers into one single text, making it easier for the reader to follow the answers. Like "The Answer" is conformed with text from answer1,2,3,4 and 5. Preserving the author's name for each "part" of "The Answer". (Sorta like SVN Blame) ;)

Some sort of visual cue indicators that "help" the reader to see that there are more good answers "below". And stuff like that.

Maybe all this makes no sense at all. :)

Improve this answer
3
  • 3
    If there are 10 short answers before a good and correct one, then that good and correct one will be on the second page that no one reads.
    – Josef
    Commented Sep 11, 2008 at 13:37
  • "No one" apparently includes you and me.
    – harpo
    Commented Sep 11, 2008 at 14:38
  • 2
    While I like the idea of letting the OP hide or merge (or otherwise define the sorting/visibility) of answers, I think you are neglecting the lemming mentality we seem to see with new questions and their respective answers. The OP should not have to actively manage posts in thread for the first day
    – Stu Thompson
    Commented Sep 14, 2008 at 15:23
11

@Wedge,
I would disagree. It would be too idealistic to believe in collaborative intelligence. Ivan Hamilton has some interesting thoughts on this problem. Most people wouldn’t spend their time to check correctness of someone's answer. In many cases they don’t have enough expertise to make correct judgment. Most of the time people vote because they like some answer or feel that it's correct. That’s why trivial answers have huge vote count.

It’s a basic psychology. Herd instinct makes people to believe in quality of up-voted answers. Here is a typical situation: Someone posts a short answer that looks nice (but might be incorrect or just off-topic) In few moments it gains some votes. After a while popular answer becomes covered with votes like a snowball. Even if someone posts an answer that is really correct but doesn’t look so nice, it will be literally buried under a mass of other answers.

Of course there are good examples when SO works as expected – stupid answers get down voted, good answers bubble up. But it’s too naive to suppose that this model works in most cases.

Some people talk about long-term perspective: “when site goes public a lot of people will vote and answer author will gain scores. Good answers go up, bad answers go down”. When site goes public most people will come just for a quick answer for their problem, it’s the same way search engines work. Highest ranked question will be chosen.

Now try to recall how often you scroll to next page of search engine result. How often do you scroll to the end of the first page? When question polluted with tons of answers it’s really hard to choose, unless you’re proficient in given field.

In general case vote count is measurement of popularity, not quality. To mitigate this problem I propose following scenario:

For N minutes since question posted, people can’t vote on any answer.

If question gathered some feedback in N minutes it must be shown on a main page. Answers must be sorted in random order for each viewer.

Now competition begins. There is N equal answers to choose (displayed in random order)

That’s way most answers will have equal chance. Since there will no way to gather reputation in first N minutes, people will spend more time on their answers. Now when question reappears on main page, it will contain N equal answers sorted in random order. People will not be affected by influence of first-the-most-voted answer; they will see N answers and will be forced to read most of them to make a choice. Since questions are sorted in random order, there is equal chance for all answers to be selected.

Key point is to split process into 2 stages:

  • accumulation of answers
  • selection of best answer
Improve this answer
8

I do my best to answer with the detail that I find is appropriate. In my limited experience, I have found that the "fastest draw" does get some up votes and initially my post falls beneath them. However, I find that if your answer truly has more value than any of the other answers, yours will quickly get voted much higher than the previous ones.

Here's an example of this in action. Eric's post was correct, but little more than a code snippet. I decided it's better to teach someone to fish rather than to catch it for them so I explained why the code snippet worked. By the time my explanation was finished, Eric had 3 or 4 up-votes. Notice how over time, my answer got more votes than Eric.

I think SO corrects this itself. I browse answered questions from time to time and up-vote the good responses, particularly when the post teaches me something I didn't know. There's not really a solution to this problem, other than hiding the reputation number so people won't be carefully calculating their every move to maximize their reputation boost. If you decide that the up-vote is more important than providing the best answer, then you are part of the problem. If instead you'd rather be known for always providing a detailed and correct answer, I think you'll find that the "problem" evaporates.

I'm here to help other people and get help myself; reputation is just an arbitrary score that indicates how many times I've had the most interesting/correct/accepted answer to a question.

Improve this answer
8

I think allowing multiple answers to be selected as correct allows the questioner to acknowledge caveats, different opinions and nuances. Often an answer does not cover all sides to the question, but two or three might.

Improve this answer
8

@Adam: I appreciate your argument, but I respectfully disagree.

I find that now there's a flood of answers within minutes. Yes, it used to be one fast and several later, but now it's 2–5 immediately, often with very similar information, often with shallow, trivial information. I picked a random question current at the time of this comment, and you can see this effect for yourself.

These linger around forever. Usually people don't down-vote, perhaps because of the combination of losing rep and having just 30 votes per day.

I also disagree that "if someone is looking for the information later, they'll read all the answers." I don't think people carefully look through all the answers, and again I don't think they'll be down/up-voting as they go.

Improve this answer
7
  • We'll just have to disagree. If i'm working on a project and need an answer, I search Sof. If a question is already here, then I do read through all the answers - so the purpose of Sof is preserved - those looking for information will still get it.
    – Pollyanna
    Commented Sep 11, 2008 at 15:33
  • For general Sof users, who aren't terribly interested in the question, yes, they often won't read all the answers, and they won't vote unless there's a compelling reason to (ie, if the answers look well ordered, why mess with it?). But for people who need the answer, every answer is often used.
    – Pollyanna
    Commented Sep 11, 2008 at 15:34
  • Fair enough. I've now been keeping my eye out, and I think I see both phenomena. Perhaps you're right that, even with all the dups and crap (which undoubtedly is there), there's enough of the effect you're describing that votes happen and good ones bubble up.
    – Jason Cohen
    Commented Sep 11, 2008 at 16:27
  • Well, we can certainly hope so! What we're seeing now might get better or worse once out of beta. This is essentially a control system with a feedback loop, but control systems with humans in the loop are notoriously finicky to adjust. One of the reasons I generally take a wait and see attitude.
    – Pollyanna
    Commented Sep 11, 2008 at 16:47
  • I'm don't see the problem from the example you give. There's a simple question and several highly up voted correct answers. If you require that every answer to a question be prefaced by an extensive, informative thesis on the subject at hand, then I think your requirements are excessive.
    – Wedge
    Commented Sep 11, 2008 at 20:51
  • @Wedge: At the time of writing that question didn't have those properties. However, now it does. This does support your argument, so yes I think I'm coming around to your/Adam's point of view. Thanks!
    – Jason Cohen
    Commented Sep 12, 2008 at 16:11
  • I always read through all the answers and select the one that teaches me the most. If there is a tie, I choose the first one. I up-vote all posts that contain a usable resolution, but I cannot down-vote bad answers because my score is too low. This is probably another reason that people do not use their down-votes.
    – user245368
    Commented Dec 19, 2013 at 16:40
6

Here's what I have noticed and tried as well:

Put a one-liner real quick, and then edit the answer and add more details. But this is not a solution to the problem. Once again, the only reason I would do this is because if I am spending time to put out a well-constructed answer, I would like it to be read.

For a solution the (programmer's) mind suggests that you should take a token to begin an answer, and then the answers are posted in the order the token was taken regardless of who submits first. Of course, there can be many problems around this.

Improve this answer
5

A few ideas, mostly independent/combinable:

(1) Have a buffer time for upvotes. You upvote an answer, but it doesn't count on the rankings for a little while.

(2) Show different rankings to different people randomly, weighted by upvotes and newness. This way newer answers still get a chance, but don't drown upvoted ones (or vice versa).

(3) Temporarily place new answers at the top. Give them N/5 views at that position, where N = # views so far.

Improve this answer
4

I agree too. I've done the same thing. Been waiting for a page to load so I can verify what I'm writing, but it's taking too long so I just post without the verification.

It does reduce the effectiveness of the site I think. Rather than have a few well written responses, you end up with 10 short half-baked answers.

I'm not sure how you'd fix this - what about a delay between the time you post? So it's not like you're all rushing to answer the question, because if there is a 30 minute delay before the post is viewed, you can safely assume someone else will have already answered the question.

Improve this answer
4

I like the idea of hiding the scores on the question page. Knowing an answer's number of votes somehow affects my voting. Clearly we are more liberal with upmodding than with downmodding.

I also like the idea of showing answers in a random order for, say, the first day or two (except the chosen answer). As a result, "bad" answers will sometimes show at the top during this period. This "undue" notice will actually cause the truly bad answers to receive more downvotes, and hence improve the eventual sorting by rank.

Improve this answer
4

Fastest gun isn't always a bad thing, it only gets annoying when people start counting their position in the post. Like many issues which are solved in development, they can be fixed twice. The initial to fix the immediate problem followed by more thorough fix so it never happens again. As a responder to a question, you can give the quick and dirty response that you are thinking of, then research it and follow up with a more complete, thorough answer.

Since this site has been designed to allow you to re-edit your posts, and others posts (once you have enough reputation) it is perfect for a quick answer followed by a more thorough answer. You never know, somebody's initial thought may help you with your approach.

Improve this answer
4

This may or may not have already been said, and as of right now, there are 39 answers. some very long-winded. I chose to be lazy and not read through them all.

One way to prevent duplicate answers from appearing right after one another is when someone goes to post, let them know if there have been any new posts since they started writing theirs. This would prevent something that's happened to me where someone was quicker to the draw in saying the exact same thing i just did.

Improve this answer
2
  • 1
    It's exactly what I proposed in one of my posts. It's not hard to implement, but will reduce number of dupes. It's already implemented for topic creation - you can see in real time similar questions. I will put this request on uservoice as soon as I get spare votes there.
    – aku
    Commented Sep 12, 2008 at 5:52
  • I absolutely second this. Seeing whether somebody else (possibly even somebody with considerable reputation, increasing the probability of a high quality answer) has started answering the same question would reduce stress, unnecessary duplicate work and redundant research made my two or more people at the same time.
    – Pekka
    Commented Nov 1, 2009 at 19:34
4

Sometimes I even see users giving replies like: "change this line of code with this one", not caring about giving an explanation of what they say. "Just do this, so I get my reputation".

And you could see low quality answers being edited and becoming better and better again in the time, but they figure first in the list of all answers, even if who gave it was slower than a user who gave a high quality and complete answer without editing it.

Improve this answer
1
2

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .