What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 149 Stack Exchange communities.

I feel like there's a problem with Stack Overflow, as the number of people prowling it increases.

Each question's answers are sorted by descending score and then by descending time of posting. This means that if a person sits down and answers a question in a long, thorough way, going through every nook and cranny, once they post their answer, it will already be one of about seven different ones, some of which have already been upmodded. This wouldn't be a problem if those answers were as thorough as the one this guy's posting, but they usually aren't. Some of them are downright wrong, some aren't even answers to the question asked because their poster didn't bother to read the question all the way through.

This causes a problem I like to call Stack Overflow's Fastest Gun in the West Problem. I've come to a point where I'd rather just send a short, simple, correct explanation, than to go and do some proper research, write a whole blog post about it or even make sure the code I post even compiles, just so it will be noticed, as opposed to the incorrect ones.

I'm sure I'm not the only one doing this and that it despairs many people from even trying to answer questions.

I've long ago learned to try to always raise a problem with a solution in hand, rather than just say "This is a problem, handle it," so my question, after this long tirade, is:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

I promise to vote on answers I like, even if it takes you a long while to post them. :)

share|improve this question

migrated from stackoverflow.com Jul 26 '09 at 13:52

This question came from our site for professional and enthusiast programmers.

create a quick answer, and then edit the detail in steps?? –  Jonathan. May 24 '11 at 19:39
@MortenBergfall he is talking about this edit –  ajax333221 May 24 '12 at 17:01
@Jonathan. That's still too bad, we're always in a hurry. –  Ramy Al Zuhouri Jan 9 '13 at 3:33

50 Answers 50

Remove visible scores.

Yes, you hear me right! It feels horrible knocking someone down, especially if you take them into the -1 or less realm! If the scoring system is hidden and you just say "Yup like it" or "Nup not good" and the actual effect that it has is hidden then people will be more likely to rate answers (especially down) and the chaff may remove itself more effectively.

A second solution would be to not allow ranking until a question had been open for a while (an hour say) then all questions have an equal chance (if you display them in a random order to different users all the better). Compare to this post, another problem with being slow :D.

share|improve this answer
I really like the random ordering among equally rated answers. –  Caspin Jan 5 '09 at 20:17
Hmm... I like the idea. This is going into my Greasemonkey cache. –  new123456 Nov 5 '10 at 22:22
But the whole point of visible scores is so that the Google-searchers know which answers to trust... –  Kate Gregory May 12 '11 at 13:11

This problem was solved 80 years ago. (See here for XKCD-author Randall's explanation).

Of course, this answer is late, so it will never get upvoted :)

[Edit] Layman summary: we want to determine what the upvote-percentage would be if everyone voted. But since we only have a small sampling of votes, we use fancy statistics to determine a range of percentages we can be fairly certain the real percentage falls under. We take the lower end of that range, to err on the side of caution.

Here is the output of the equation. You'll notice that when there are many votes, the output is close-ish to positiveVotes/totalVotes, but when there are few votes it's much smaller.
This is exactly what we want.

Here is some code:

///Returns a rating for the given post.  Larger is better.
///Based on the equation found at http://www.evanmiller.org/how-not-to-sort-by-average-rating.html
public double GetPostRating(int numPositiveVotes, int numNegativeVotes)
    int totalVotes = numPositiveVotes + numNegativeVotes;
    if(totalVotes == 0)
        return 0;

    const double z = 1.96; //Constant used for 95% confidence interval in a p-distribution
    double positiveRatio = ((double)numPositiveVotes)/totalVotes;

    //Crazy equation to find the "Lower bound of Wilson score confidence interval for a Bernoulli parameter"
    //Again, see the above webpage
    return (positiveRatio + z*z/(2*totalVotes) - z * Math.sqrt((positiveRatio*(1-positiveRatio)+z*z/(4*totalVotes))/totalVotes))/(1+z*z/totalVotes);

Note that the above equation assumes upvotes and downvotes have the same frequency. Since upvotes are way more common, downvotes should ideally be weighted more harshly (in other words, 3 downvotes says a lot more about an answer than 3 upvotes).

Also, I believe newer answers should be given preferential treatment, at least for a few minutes (see my comment below).

But even without these, this is a neat improvement.

share|improve this answer
+1 infinity, although the formula would have to be modified to take into account the pinned "correct answer" somehow. Even Randal of XKCD fame is behind on this one! –  Gleno Aug 2 '11 at 22:36
"newer answers should be given preferential treatment" - I should explain. When people are racing to answer a brand new question, the answers posted a few minutes later tend to be slightly longer but higher quality; but, by then the early answers have gained too much first-five-minutes-momentum. If you add a factor of say, someConstant/secondsSinceAnswerWasPosted to the above rating, this would give the later answers a precious minute or two at top to gather some votes of their own. And usually when someone an older question it's because none of the existing answers were satisfactory. –  BlueRaja Dec 10 '11 at 0:19
@BlueRaja: mostly right, but I've seen questions with accepted answers still getting answers a year later, often very poor answers from new users who don't yet know The Ways Of Our People. –  sarnold Dec 10 '11 at 0:28
@sarnold: In that case, the new answer will be conveniently at top for us to flag (or upvote). It will only be there a few minutes though, so it will likely be only the people who spend all day hitting F5 on the "active questions" page who see it at top, not the people who actually need to find an answer to that question. –  BlueRaja Dec 10 '11 at 0:38
Statistics FTW! However, this would require that the cost of downvoting is more or less the same as the cost of upvoting. Currently you get points for upvoting, and lose points for downvoting. That will seriously screw up the statistics (although there might be a way to adjust for it). –  naught101 Feb 6 '12 at 6:58
@naught101: This is completely unrelated to a user's reputation, it's only for sorting answers on a page. –  BlueRaja Feb 6 '12 at 17:03
Yes, but the facts that a) you require more reputation points to downvote, and b) you lose reputation points for downvoting (and gain them for upvoting), means that down votes are much less likely, and are therefore more meaningful than upvotes (people probably put more consideration into their down votes than their upvotes - I know I do). The above equation doesn't take this into account at all, although it could, if you weighted the downvotes higher than the upvotes (eg. by total votes on all sites/down votes) –  naught101 Feb 7 '12 at 7:06
This would be a perfect solution, but unfortunately I feel people are put off downvoting due to the negative reputation it gives - especially new users with low reputation. –  James Donnelly Mar 22 '13 at 10:04
I think that "negative" votes should be number of view of that answer. In long run every answer will have similar number of views. This will create 3rd option for voting: Great answer +1, ok answer 0, Wrong answer -1. Down vote should have bigger weight, and numbers of views less weight. –  Yankes Jul 25 '14 at 15:27
upvoted even if your maths is poor. 1927 was 84 years ago at the time you posted (presumably 2011). Now 87.. –  CashCow Oct 2 '14 at 15:33
The fact that you've so many upvotes proves that your answer is WRONG! :p –  Mrchief Oct 27 '14 at 19:36
A problem with this formula is that, for posts with no upvotes, it always returns (or should return, if there were no round-off errors) a zero score regardless of the number of downvotes. (In fact, numerical errors may cause downvotes to slightly increase the score.) A better choice could be to use the midpoint of the 95% Wilson score interval instead of the lower bound, which also leads to a simpler formula: score = (upVotes + z*z/2) / (totalVotes + z*z). As a bonus, for high vote counts, this actually favors uncontroversial posts a bit more than using the lower bound would. –  Ilmari Karonen Mar 12 at 19:53
I have cast thousands of upvotes, but not a single downvote. I've thought about it a few times but usually the answer had been downvoted already so I didn't feel good about downvoting it even more. What you could do instead of comparing up and down votes, is taking into account how many people have upvoted an answer, in comparison to how many people have read it. I wrote a blog post about that, debiki.com/-9qb49/… –  KajMagnus Jun 15 at 11:04

Early in my experience on Stack Overflow my gut reaction to this issue was that it was a problem. I also hadn't asked any questions at that time. I soon changed my mind. I believe over time that the most correct answers do bubble up to the top.

If I ask the question I will be reading all the answers no matter what order they are in. As someone looking for an answer I will most likely be doing the same thing. To me what order they are in and how much they have been upvoted doesn't really matter if I need the information.

share|improve this answer


I created a request on UserVoice:

Provide live monitoring of answers being posted

To help us fight against tons of duplicate answers, provide following functionality:

At the time person is writing his answer show excerpts from answers being posted. Color code similar answers.

It can be implemented in the same way it's done for new question posting - i.e. author can see that similar posts already exist.

If it's what you mentioned, please vote for it.

share|improve this answer

Here's what I have noticed and tried as well:

Put a one-liner real quick, and then edit the answer and add more details. But this is not a solution to the problem. Once again, the only reason I would do this is because if I am spending time to put out a well-constructed answer, I would like it to be read.

For a solution the (programmer's) mind suggests that you should take a token to begin an answer, and then the answers are posted in the order the token was taken regardless of who submits first. Of course, there can be many problems around this.

share|improve this answer

You could let old answers "fade away", unless someone votes on them and they are not one of the top n answers. Hacker News uses this technique to gray out downvoted comments so you can ignore the bad answers and focus on the good ones.

But I guess the better way would be to change the culture of Stack Overflow:

  • If you ask a question, you should own the answering process. Vote, comment and clarify heavy on the answers. People will see that you care and keep answering even on older questions.
  • As a reader, use voting more deliberately. Especially the downvote-button is useful to push new answers above inaccurate old ones.
  • Too many rules are confusing and rarely the solution to more than one specific problem. Provide a framework with few, simple rules and let the community do the rest.

Yes, I know: if we don't have all these safeguards in place, the barbarians will burn down the town, once we open up the gates. But a lot of the rules are there to make things nicer, not safer.

share|improve this answer
Just a preference, but I think downvotes should only be used for incorrect or misleading answers. Upvotes should be used to push the cream up, and ownvotes to push the rotten items down, but I wouldn't want people to downvote a question when it applies just because it's not as good as another... –  Adam Davis Sep 11 '08 at 14:51
Once we get out of beta yo'll see far less than 10% of the viewers voting. It would be great if everyone voted often for all good answers, but it's either human nature or culture that we really only vote up the best one if it's currently an underdog, or really exceptional - votes are scarce. -Adam –  Adam Davis Sep 11 '08 at 15:22

I think that this site has to become more like a wiki where many people add small contributions to existing answers instead of creating their own new answer. Unfortunately the reputation system as it is now doesn't really encourage this yet.

I understand from listening to the podcast that this is the direction the creators of Stack Overflow want to go too. Planned features like making questions and answers community editable would be a big step in the right direction.

share|improve this answer
You're right. Although I ran into interesting interaction today - I deleted comments between I and another user that had no further use in the topic, and they tracked down my email and called me a censor - expecting that our off-topic conversation would last on Sof forever. People won't like edits. –  Adam Davis Sep 11 '08 at 20:15
People don't like other people who judge, decide upon or have the authority to change/delete what they have done. –  Bora May 18 '11 at 5:59

The reputation system is valuable, and I find myself emotionally validated by it a bit, but I also have to keep reminding myself that it is an illusion.

There are likely more points to be gained in the first 24 hours of a question than in the whole rest of its lifetime, because a lot of fast guns will be searching the unanswered questions, looking to build rep. And because the person who asked the question is likely to choose an answer and mark it "correct" within this fairly short time, while the question still matters to him. So, for those who see the Stack Overflow site as a game where building reputation is how you 'win' ... yeah. Fast guns will 'win'.

But a lot of Stack Overflow users will be playing a different game. For them the 'win' happens when they get an answer to the problem they have right now. The people playing this game are probably much less likely to even notice the reputation game. Many of them will probably not even up-vote the answer which works for them. Long term, I wonder what the stats will show: which percentage of asked questions never get an answer chosen by the question-asker?

And stats can't really reflect this, but how many people will get an answer without ever posting a question? They may find their answer via Google, or by starting to type a question and the site's own search engine flags a similar question/answer - one that's good enough. Will they bother to vote? Heck, a lot of them won't even be able to - because they haven't gained enough rep to vote. They never started playing the rep game.

So, an answer which never even gets rated might be the one that's really solving the most issues for this (potentially huge) group of people who never play the rep game, or never even notice it. That's the illusion: the Illusion Of Perfect Feedback. You're never going to get perfect feedback. So in this sense, the ordering of the answers will never be perfect. It is possible that it will never even be very good. Because it cannot be.

So, fast guns, slow guns, medium guns: write the best answer you can, no matter how long it takes, and realize that even if you never get a single point on the Stack Overflow Rep-O-Meter, you may be helping a lot of people.

I'm not dissing the rep system or the fast guns. They kickstart the process, and that's valuable!

share|improve this answer
i think the fact the question searching appears to be so unreliable compounds the first point you make - questions seem to get lost in the melee... –  flesh Dec 10 '08 at 18:59
Maybe asker should be able to vote on answers. –  Joshua Apr 17 '09 at 22:30
Actually rep is not an illusion. Rep determines what you can do on the site, it determines not only how much influence you can exert, but also how much you can contribute at all. I cannot even disagree with your answer via downvote here on meta, because of this alleged illusion, simply because I posted something unpopular and on meta "downvotes mean something different", although the fact that I cannot disagree with your answer shows that to be false as well. –  Joshua Drake May 30 '12 at 13:38
If you search and find the question and the answer to it you want you should upvote both of them –  CashCow Oct 2 '14 at 15:37

Grace period idea (currently editing...)

One solution would be to have a grace period where votes are hidden:

  • The grace period begins when the question is posted and lasts for some set length of time.
  • During the grace period the following is true:
    • Sorting by votes is disabled (questions are displayed in a random order if 'votes' is chosen)
    • Votes from other users are invisible meaning...
      • At first all answers appear to have a vote of 0
      • Any answer that you vote on will display your vote only (+1 or -1)
  • When the grace period ends, all votes which were hidden will be revealed.

Maybe the grace period could be the same as the grace period for editing the question.

@Jeff Atwood

Sounds complicated

No more complicated than...

  • A grace period in which your edits don't count as real edits.
  • Hiding the number of Offensive flags from some users and not others.

All you need is a simple flag that switches between two modes of the question: "Grace Period Mode" and "Regular Mode". You must already have this because of the grace period for counting edits on questions.

Will people understand what is happening?

Yes, if you explain it to them. Let's say we name this grace period "Secret Voting". All you have to do is put a banner of some sort above all the answers that says "The Voting is Currently Secret". When the grace period expires, the banner would be removed.

Also during the grace period, the Votes sort tab would not be present, but instead a Random sort tab would be present. Again, people will understand it, because the interface explains it to them and doesn't make them think.

share|improve this answer
I really like this idea - partly because I only upvote when the answer seems lower than it should be. With votes not visible for the question's first appearance on hte homepage (until it falls off, maybe?) you'll get a much more interesting, and possibly truthier, voting. –  Adam Davis Sep 12 '08 at 2:20
Jeff, I think if you add some visual clues or simply descriptive text on question page, people will understand. IMO it's not that complicated. –  aku Sep 12 '08 at 8:50
I like this idea - except. It doesn't yet address the problem where a good answer is added a long time after the question is no longer hot. –  Richard Corden Sep 19 '08 at 13:28
Disallow marking an answer as "correct" during the grace period as well. This increases the chance that the "correct" answer isn't just "the best answer to arrive in the first 10 minutes." I envision a secret-voting period of, say, an hour, but the editing grace period is 10 minutes -- too short. –  Rob Kennedy Dec 10 '08 at 17:00
@blntechie we already have lots of questions with no accepted answers –  jcolebrand Dec 22 '10 at 14:27
I think this is a great idea. @Jeff any chance of this being considered? –  Box9 Feb 3 '11 at 23:55
@box9 we already show answers randomly by vote, so the important part of this is implemented. The rest, well, it's been 2 years.. so.. :) –  Jeff Atwood Feb 4 '11 at 0:13
@Jeff, true, but I don't see the random sorting feature really getting to the meat of the problem. The secret voting idea (with the banner), however, will really encourage people to vote more and more objectively. –  Box9 Feb 4 '11 at 0:18
Man, I love this idea... –  Gleno Aug 2 '11 at 22:18
+1 Hiding votes for a period seems like a swell idea –  Miserable Variable Oct 2 '11 at 17:23
Another +1 for hiding votes for a period. The clock should start when the first answer is posted, not when the question is posed. The problem with random display of equal weight votes is that the first and likely incomplete answer will get an up vote and then subsequent ones will all randomly sort after the first one. Then the first one, with it's one vote keeping it first, can be edited into a better answer. –  Josh Heitzman Nov 14 '12 at 4:58

Sometimes I even see users giving replies like: "change this line of code with this one", not caring about giving an explanation of what they say. "Just do this, so I get my reputation".

And you could see low quality answers being edited and becoming better and better again in the time, but they figure first in the list of all answers, even if who gave it was slower than a user who gave a high quality and complete answer without editing it.

share|improve this answer

I know, I am quite late in reacting to this question.

So my answer will suffer with the problem what this post is trying to state.

This answer would be the last one in the list, a lot below the highly upvoted answers.

Now the solution I would like to suggest for the problem is:

  1. Give priority to new suggestions and answers. I am not omitting the usefulness of the old ones, but we should be soft enough to welcome the new suggestions.

  2. If upvoted questions are given priority over other ones, then downvoted questions should be given minority over others.

  3. The mark-it-as-a-useful answer trick should be used to decide the priority of a good answer and the mark-it-as-spam trick should be used to decide the location of the answer down the hill.

share|improve this answer
If you order the answers by "active" then your answer will appear first. –  Lix Sep 15 '12 at 10:21

Good answers will rise up. Don't worry, post a helpful answer that does not exceed half a page. Instead of "read this" or "link to that", be objective and comprehensive. Also, the first/last line (bold) as fast as possible should be the direct answer to the OP's question. These kind of answers will be rewarded with votes.

As for not reading all the way down, well it's your loss if you don't read all the answers to find the gem. Someone else definitely will. So post and help away!

Hey I made my first 1000, so post! post! post! Just kidding. 'The work is its own reward' as Jeff likes to quote.

share|improve this answer

Get a grip. The idea of the site is to get answers to straightforward questions, and then get back to work. If one asks a question, one is going to be looking for what one thinks is the right answer, so one is going to read all of them.

  • Before Stack Overflow: I'm looking at the entire internet full of content that isn't remotely relevant to my problem.

  • After Stack Overflow: I'm looking at an unordered set of answers to my specific question, written by real people, and one of them is probably a nudge in the right direction.

share|improve this answer

I do NOT want to, in any way, discourage the quick and dirty answer.

I've asked questions that have received an immediate answer with enough information to get me past my block, but not served on a platter as you propose, with all the information I might need.

This is extraordinarily helpful - I know I can post something on here, day or night, and get an answer within minutes, often seconds, that will be better than searching through books, online, etc, even though it's a throw-away post that took someone 35 seconds, who may have only answered for the reputation. If it's at all helpful, I upvote it.

Later on I'll review my question again and select the answer that is what I consider the best of the bunch (I try and wait a day or so for long answers like you provide).

But I DO NOT want to 'adjust' the system to discourage short quick answers, or make long thoughtful answers that come a day late so highly valued that no one posts the short ones anymore.


Let's look at it from the perspective of what the site is supposed to be:

A long term repository of searchable, free, and freely editable information.

In the long run, it doesn't matter if the long, good answers don't get the initial rush of votes. They will ultimately go to the top because

  • There are generally few answers per question, so future searchers will read the entire post and may upvote the better answer. The special case of the question with tons of answers is just that - a special case, and it will be gamed and mined for rep a dozen ways from Tuesday, so it's not worth fixing - it's less than 1% of total content - don't design the system around the special case, or else all your cases will become special.

  • Once someone posts a new answer, the post gets a new chance to be on the front page, which will entice some more people to see it that haven't - they'll upvote the good, long answer at the bottom of the page if they feel it's better than what was initially offered.

  • The person who initially asked the question will often come back later and review it, and if your post is truly the better answer they'll select it as the new accepted answer.

A key point, though, is that if someone is looking for the information later, they'll read all the answers unless it's obvious that the first one is good enough for their purposes.

So that completely guts your argument that it's important that the longer, complete answers should get a boost to the top, somehow. The only reason left to complain about the tradeoff between short posting and long posting, then, is because you may not get as much of the initial rep as you would otherwise.

share|improve this answer
I disagree too. Even in beta there are enough new questions that old answered questions get rotated off the front page. Most of your argument assumes that the person who asked the question performed due diligence to maintain that question. I don't think that happens as much as it should do. –  tonylo Sep 11 '08 at 17:49
tonylo, the main problem being discussed here is that there's a lot of initial activity on a question, most of which makes spending time on a great answer less advantageous. So the issue is really only with questions that will stay on the front page and deciding whether to make a great post or not. –  Adam Davis Sep 11 '08 at 20:13
Taking your stance as valid, I think there is still an issue whereby most question-posters will not come back a day+ later to upvote/select the best answer. If there was some [more obvious / greater] incentive to do that then I could agree with you more completely. –  Sparr Sep 20 '08 at 2:07
The bounty system is working very well to give the detailed long answers a chance for great rep increases... –  Adam Davis Jan 28 '09 at 17:19
I disagree. As to your key point, I don't think people who come much later read through all the answers and upvote the good ones. A more likely scenario is that they go through the first couple of answers, because in the mind of that person, the first couple of answers probably a) are the best b) contain 80% of the information one would get by reading all of the answers, which might take considerable time. That person is not likely to review all the answers, even if the first couple do not contain all the details he cares about. Would you start from the middle or end of the list of answers?? –  allyourcode Mar 8 '10 at 2:12
In that case, the quick and dirty answers that start at the top tend stay there, and the carefully thought-out ones tend to stay near the bottom. You can't dismiss this by saying "when that happens, it means the quick and dirty answers are deserve to be on top". The problem is that quick and dirty answers benefit from a positive feedback loop: the higher an answer is in the list, the more likely someone is to view and upvote it. Long and thorough answers, on the other hand, have to fight an uphill battle in order to gain attention, a prerequisite for upvotes. –  allyourcode Mar 8 '10 at 2:21
@allyourcode - but if the quick and dirty answer is right and sufficient to resolve the issue, then who is to say that the longer, more ponderous message is righter or sufficienter than the quick and dirty answer? Why do people hold so tightly to the idea that a longer, perhaps even more complete answer, is better than the shorter answer which also resolves the problem, and has come earlier to boot? Keep in mind that we're not talking about wrong answers. Both are right. –  Adam Davis Mar 8 '10 at 15:02
@allyourcode - further, my own experience shows that longer posts which come later still receive significant upvotes over time. Otherwise the necromancer badge would never be awarded. People still vote on posts I made 2 years ago. Your assumption that people don't go back and upvote posts, even shown way down on the page, is flat out wrong. But don't take my word for it - examine the data dump and draw your own conclusions. –  Adam Davis Mar 8 '10 at 15:09
It's not necessarily about "length". No one is suggesting that word or char count be used as the sort criterion. As for "this is sufficient, therefore it should win", you're assuming that every answer that reaches some threshold of completeness is just as good. I completely disagree. –  allyourcode Mar 15 '10 at 21:11
re Pollyanna's second point: I do not doubt that people look at all answers to old questions occasionally, and upvote the better ones, but I think that's rare. One of the points that I'm trying to make is that there's alot of room for improvement when the desired outcome is requires a whole series of rare occurrences. –  allyourcode Mar 15 '10 at 21:16
Even though I encourage quick and dirty answers, better (and later) answers should triumph over quick and dirty ones...in the long run. But "in the long run" is not good enough for right now. The underlying problem is that questions with one up-voted post are considered "answered". Early upvoted posts build momentum by being shown first and allot the question as "answered", discouraging delayed higher-quality posts which would provide better information to all future viewers, but which can never build the same momentum as the first quick and dirty answer. –  pate Feb 11 '11 at 18:29
@Tom - but a brilliant answer released at 11am is worth more than a bad answer at 10:45am. –  Martin Capodici Jul 31 '14 at 22:11
I would even say a good answer posted after 4 minutes is worth more than a placeholder posted after 3 minutes (which got edited to a mediocre answer after 11 minutes and accepted as the fastest after 15 minutes.) –  Robert Aug 5 '14 at 6:56

I've noticed myself reacting to this too, although I try to mitigate against it as much as possible.

I sometimes try and post a short correct answer as quickly as possible to kind of "mark" the question with my answer to discourage other similar answers. I then I would edit my answer to improve it, using the 5 minutes grace period to edit without it appearing as an edit (although I'll still carry on editing after then if I can improve it, I just use the grace period to edit without worrying about writing edit descriptions).

Conversely, if I find someone has answered the question while I was writing my answer, and I don't feel my answer adds anything, then I've deleted my answer on occasion, although if I provide a link and the previous answer doesn't I'd leave mine. Once I can edit I'd be inclined to remove my answer and add the link to the other answer. Since comments were added I've also been adding to answers by commenting on them, again, once I've got edit rights I might edit directly for simple improvements (adding links, spelling, grammar, formatting, etc.) and just use comments for actual comments.

I've also learnt a few of the keyboard shortcuts for the edit box (hover over the buttons to see them on the tooltips.) The shortcut I use the most is CTRL+L to add a link to selected text. That makes it much quicker to write a quick answer with a few useful links in it.

I've also taken the attitude that there is also a long game to be played. If you can provide a better answer then the initial rush of points that the first answers got is possibly just the tip of the iceberg and there's probably much more points "underwater" to be made once the site is indexed by Google and we start finding answers to questions in Google search results. (although the fact that new users can't vote straight away - you need +15 rep to upvote - might mitigate against this.)

Update: for a really good description of how the long game will play out and why the fastest gun answers aren't a problem see Adam Davis's Answer.

share|improve this answer
That's exactly what I do, too. However, once you hit submit the 5-minute clock starts ticking, so your short answer only gets as good as you can make it in 5 minutes. –  Kyle Cronin Sep 11 '08 at 14:13
the link is broken in here. –  Rosinante Dec 14 '09 at 18:55

I think it's got to be down to the community to a large extent - and to the question owner.

Perhaps the question owner should be given more power to change the order of answers to a greater degree?

The only problem with that is that a lot of people asking questions will be 'drive-by' users, just wanting a quick answer to their problem. They're not the sort of user who will hang around to collate and manage responses.

share|improve this answer

The problem

Fast and correct, or comprehensive and insightful. (Scroll down to see suggestion)

I do not think that either type of answer is inherently better. A "good enough" answer will solve the problem and get you back on track. An insightful answer will teach you something new, make you more skilled.

The problem is that if you focus only on rewarding fast answers, many of the comprehensive answers will never get posted at all.

As it is now, the quick answers float to the top, and stay there. People do not always bother to check all the answers, and they might not consider the depth of the question, the way the asker would. If a top voted answer is good enough, chances are it will recieve more votes than it deserves, merely because it is voted highest. That is an error of conformity of the reviewing audience.

Blind votes

Unless votes are made in the blind, you are always getting a systematic error. Simply put, here's the three biggest culprits:

  1. How many upvoted it?
  2. How high up on the answer page is it?
  3. Who made the post?

That is: People will tend to conform to the majority's opinion. People will only read answers to a certain thread depth. People will tend to agree with "authorities", e.g. people with high rep, many badges, etc.

These are all basic concepts in psychological testing. There are many kind of errors that prevent you from getting correct answers. Here is a wikipedia entry on bias, take a look at Bandwagon effect and Primacy Effect.

The correct way to do it, is blind tests

The suggestion

Note: This only affects people who:

  • Are eligible to vote (due to rep or other constraints)
  • Still hold the option to vote for the question.
  • Are not the person posting the question

Anyone who just want to see the voting results can do so by clicking "finished voting".

Blind votes:

  • The author's name & rep is hidden.
  • Other people's votes are hidden until you are finished voting.
  • Answers are presented in semi-random order (1).

(1): A clearly poor answer should be nudged towards the bottom, a clearly good answer should be nudged to the top. Otherwise we are not using the expertise of our users. However, to avoid tainting by position, order is determined by vote intervals, with random order inside the intervals.

Those who do not vote

What about users who are only after answers? If you cannot or choose not to vote on a question, all the information becomes visible to you, and you can use this additional information to evaluate the quality of the answer.

The effect

By having users only vote on the merits of an answer, the validity of the vote tally will be maximized. In other words, we will know that the votes cast reflect the quality of the answer.

As it is now, the votes reflect something more than a good answer. Two identical answers may get a completely different score. Not because one person was faster in solving the problem, but because they shoved a foot in the door fastest.

How will this affect quick and dirty answers? It will still allow users to get rep for quick answers, but it will reduce the exponential effect of "being first", and also disperse some of the rep towards those who feel they have something to add, despite not being first to reply.

In short, it will be less of an "all-or-nothing" effect to being first.

Note: Please give feedback in the comments. The feedback so far has really been helpful.

share|improve this answer
But this ignores the population of people who look at votes not to decide how to vote (they may not even have an account) but only to know which answer to trust. That is the larger community. –  Kate Gregory May 21 '11 at 19:48
Consider stackoverflow.com/questions/6076717/where-to-put-dllimport as a single example (I just grabbed a multi answer question from one of my tags that newbies ask in a lot) - the top vote getter is clearly better than the other two. And the whole point of this site is to serve those who cannot tell which answer is better, and to help them. –  Kate Gregory May 21 '11 at 20:15
I have to say I strongly disagree with this. Yes, you can look at data which shows that higher rep users get more upvotes for more-or-less equivalent answers. However, generally speaking users with higher rep make answers that are simply better -- that's what gets them the rep in the first place. Furthermore, being able to point at an answer and say "I wrote this" is useful for employers as well as for employees showing off what they're capable of. This suggestion is completely contrary to the function of the sites which is why I downvoted. –  Billy ONeal May 22 '11 at 18:24
-1 Appeal to authority fallacy. That you have a university degree in psychology is quite irrelevant. This needs to be substantiated not with philosophical argument, but rather with empirical proof that people actually do vote based on a user's rep. I don't think that exists, in fact I've seen quite the opposite. –  Cody Gray May 22 '11 at 18:29
@TLP: Well, then we can agree to disagree. I think it's a horrible idea. However, this answer would still get a -1 from me if nothing else that it does not talk at all about the FGITW problem (the topic of this question). –  Billy ONeal May 22 '11 at 18:34
Not necessarily. Reputation is a direct indicator of the quality of answers you've posted on this site. A university degree isn't necessarily the same thing. Although that part of the comment was kind of a joke, contrasting your bristling at downvotes and justification of your theories with the very same underlying logic that you appear to criticize. –  Cody Gray May 22 '11 at 18:38
@Cody I am not saying votes are primarily based on rep, I am saying that they boost the effect of a good answer. But I do not think rep is the biggest factor in this case. More likely a high score and top position causes more bias than rep. –  TLP May 22 '11 at 18:58
"Fast and correct, or comprehensive and insightful." In my experience, this is very much a false choice. The FGITW problem is generally understood to mean that someone will post a fast and correct answer to start with, and then (usually within the 5 minute window) edit it to be comprehensive and insightful. In the cases where I've used my fast gun, this is what I do. In the cases I've seen Jon Skeet et. al. use his fast gun, this is what he does. It's not necessary to pick one or the other. It generally ends up that the answer that gets the upvotes deserves most, if not all, of those. –  Cody Gray May 23 '11 at 3:28

I realize now that the FGITW problem is actually the reason that Stack Overflow works well.

If you don't know enough about a topic, then you won't be able to post a quick answer.

Those who post a good quick answer, tend to be those who have more expertise in that area. This gives a higher probability of getting good expert answers.

share|improve this answer
The problem is not good quick answers, but bad quick answers. It generally takes longer to explain why an obvious solution that a non-expert might suggest is not a good one, than it does for a novice to post that flawed solution. –  Peter Boughton Oct 6 '10 at 17:14
"tend to be" is an incorrect deduction here. Those who post first, simply post quickly. They can later edit their answer to include anything they missed, and wrongly recieve credit for providing a "good fast answer", when in fact they merely provided a "fast answer". –  TLP May 30 '11 at 7:52

I've put together a different but as it happens related question (Expiration of answers for questions with novel solutions), and a proposed solution, that I think helps resolve (or at least significantly advances discussion about) the "Fastest Gun in the West Problem" by having the "fastest answers" replaced by "better answers" as old and perhaps deprecated answer-sets are revolved by the community (i.e. wiped, to-be replaced by new sets of answers).

share|improve this answer

Following on from Omer's distinction between a Collaborative site and a Competitive site...

It would seem that we need the best of both worlds. There's nothing wrong with a bit of healthy competition so that people strive to produce timely, high quality answers. However, we don't want the answer to be lost between several people's contributions. Certainly marking answers up and down helps the good stuff filter through to the top, but if there's lots of good answers you really need a combined view.

I suggest the question owner take on (at least initially) driving the collaborative element and start the 'wikificiation' process of combining the best bits of several answers into (drum roll) the answer.

The question owner can do this today by adding another answer, but really it's an additional feature that should be put into the tool.

I'd say that most (90%?) of questions wouldn't need this extra step, but where there are several high quality answers having a single point that draws them together would be excellent.

share|improve this answer

I think allowing multiple answers to be selected as correct allows the questioner to acknowledge caveats, different opinions and nuances. Often an answer does not cover all sides to the question, but two or three might.

share|improve this answer

Just allow the OP to select "how long before the first answer gets displayed", say, 10 minutes. If T is the original post time, then we'll call T + 10 minutes "Answer Open Time". Or we could call it Business Time, whatever you prefer.

This makes every question like a mini-contest, where people have 10 minutes or so to submit their answers. Answers are then displayed, but "early" answers are timestamped with Business Time, not the time the server originally received them. This effectively removes the fastest gun problem.

share|improve this answer
I don't like this at all - you'd end up with a bunch of identical answers. It would result in a lot of wasted (redundant) effort when people put time into a question that is already well answered. Besides, FGITW was largely replaced with SCITE. Any method, including this one, will be gamed just as heavily as the method that it's 'fixing'. But I believe the unintended consequences of a 10 minute wait time (redundant redundancy) greatly outweigh any benefit it might provide... –  Adam Davis Dec 14 '09 at 17:16

I agree to a point, although I have to guess that the site has not even come close to settling down. Users are still in a mad panic, scrabbling for reputation points and badges of any shade - what you are describing is bound to happen... initially.

It will calm down.

Having said that, I've noticed a pleasant tendency for people to vote for the actual best answer, most of the time.

Despite all the bilge posts, it seems the cream does often rise to the top!

share|improve this answer

I agree, and fully admit to being guilty of this myself, on occasion. I have, thankfully, seen such 'answers' being modded down, but the level of this occurring varies hugely from question to question - some 'bad' answers are fairly quickly modded down, others are left, possibly, indefinitely.

I think the point about wiki-editing is significant. If the barrier was much lower, we'd get better quality answers, and people would also be less keen to run around, scrabbling for rep. I think Jeff's reservations about this are slightly too great - wikipedia has proven to work without such draconian restrictions.

I also think it would help a lot if up-voting also cost, just like down-voting does. Currently, there is nothing to discourage people from rushing around, blindly up-voting everything they come across.

UPDATE: I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ...

UPDATE: To clarify the delay concept: this would only apply to answers (but all answers), and could either be a 'reasonable' time to mask 'first posts' (e.g. 10 minutes), or could be dynamic based on number of answers, number of views, etc. "InSciTek Jeff" had a great idea relating to 'blind' votes (see question comments).

share|improve this answer
I like the "I wonder if there should be a limit - either time, or number of views / votes / answers - before answers can be voted on at all ..." Jeff nuked an idea i posted to uservoice regarding a delay period for vote to be shown. "A pathology of its own", he said. –  Stu Thompson Sep 14 '08 at 15:12

I agree too. I've done the same thing. Been waiting for a page to load so I can verify what I'm writing, but it's taking too long so I just post without the verification.

It does reduce the effectiveness of the site I think. Rather than have a few well written responses, you end up with 10 short half-baked answers.

I'm not sure how you'd fix this - what about a delay between the time you post? So it's not like you're all rushing to answer the question, because if there is a 30 minute delay before the post is viewed, you can safely assume someone else will have already answered the question.

share|improve this answer

I completely agree. The pattern of a lot of questions and answers almost appear to look like reputation trading rather than providing quality information. Obviously this is an annectdotal opinion.

The system does appear to reward incomplete but quick answers.

share|improve this answer
...and does not reward those who think an answer out completely. –  Stu Thompson Sep 14 '08 at 15:14

I think that a time buffer is a solution, but the buffer should, instead of buffer the appearance time of the answers, decide when people can vote. Answers will appear as they are being written, but votes can be issued after a half hour or ten minutes, whatever.

Given that you won't be upvoted in that period of time you know you can take 10 minutes or a half hour to prepare the answer.

The saddest part of it all is: What is reputation actually good for? I mean, besides a little ego trip? I'm as guilty as the next guy, but it's still sad.

share|improve this answer
Yes, rep is definitely a head trip. But it's also tied to rights, and there are obviously a lot of users who are eager to contribute and make the site better. I was so excited when I could retag! Yes, it is sad. –  harpo Sep 11 '08 at 14:29

A related problem is the fact that when a question already had 10+ answers, an answer with new information given at this time tends to get lost in the clutter in the bottom and not get the exposure it deserves.

One possibility would be to position an answer based on both its rating and its "age" (for lack of a better word), giving recent but already voted answers a place before older answers with a similar rating.

share|improve this answer
But it's not the default, nor - necessarily - the 'best' way of viewing answers. The types of answers referred to will still suffer bias. –  Bobby Jack Sep 11 '08 at 10:09

I feel that it works fairly well as it is. Using the default sort of newest first, the long thorough answer will show up first for the answers with the same vote. I do think that the 'best' answer will in most cases end up at the top of the list. If a user browsing the site really care about finding a good solution to a problem then they will look at all the answers and vote up the one(s) that prove useful.

share|improve this answer

Another possibility that might help is to give users a better view into what changed recently around their votes. I haven't yet found a good way to see all Q&As I voted on that have changed. Having that, it'd be easier to review one's votes, which in turn might lead to long-term improvements in vote quality.

share|improve this answer

You must log in to answer this question.

protected by Jeff Atwood Aug 8 '11 at 5:13

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged .