1291

I feel like there's a problem with Stack Overflow, as the number of people prowling it increases.

Each question's answers are sorted by descending score and then by descending time of posting. This means that if a person sits down and answers a question in a long, thorough way, going through every nook and cranny, once they post their answer, it will already be one of about seven different ones, some of which have already been upvoted. This wouldn't be a problem if those answers were as thorough as the one this guy's posting, but they usually aren't. Some of them are downright wrong, some aren't even answers to the question asked because their poster didn't bother to read the question all the way through.

This causes a problem I like to call Stack Overflow's Fastest Gun in the West Problem. I've come to a point where I'd rather just send a short, simple, correct explanation, than to go and do some proper research, write a whole blog post about it or even make sure the code I post even compiles, just so it will be noticed, as opposed to the incorrect ones.

I'm sure I'm not the only one doing this and that it despairs many people from even trying to answer questions.

I've long ago learned to try to always raise a problem with a solution in hand, rather than just say "This is a problem, handle it," so my question, after this long tirade, is:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

I promise to vote on answers I like, even if it takes you a long while to post them. :)

migrated from stackoverflow.com Jul 26 '09 at 13:52

This question came from our site for professional and enthusiast programmers.

  • 192
    create a quick answer, and then edit the detail in steps?? – Jonathan. May 24 '11 at 19:39
  • 18
    @Jonathan. That's still too bad, we're always in a hurry. – Ramy Al Zuhouri Jan 9 '13 at 3:33
  • 2
    @RamyAlZuhouri, the time to click post and edit is insignificant compared to writing an answer that is long enough and thorough enough to need to click post and edit. (in other words if your in a hurry the answer your writing will be shorter and less thorough so you won't have to edit in the detail in steps anyway) – Jonathan. Jan 9 '13 at 13:28
  • 1
    @Jonathan. But the answers should be sorted considering also the edits then. – Ramy Al Zuhouri Jan 9 '13 at 13:46
  • 4
    I read the first two answers in descending score order (and their comments), then skimmed a bit more and grew weary of the topic. Are there great answers among the other 48 (as of now)? I don't expect I'll ever know. It's not in my nature to go hunting for under-appreciated answers, even though I've felt the sting of under-appreciation on S.O. – Griffin Oct 2 '14 at 14:14
  • 23
    This answer was practically identical to the accepted answer to the same question, but posted 10 seconds later. The accepted answer got 838 upvotes; the linked answer got 8 upvotes. There's no way in hell that the 8300 rep difference between what those answerers earned is fair or represents the value of their contributions. – Mark Amery Nov 8 '15 at 21:51
  • 1
    I posted a possible solution here meta.stackoverflow.com/questions/337187/… – Brad Thomas Oct 31 '16 at 19:47
  • The rush to answer is because current scoring incentives are not temporally aware. The resulting dependence on the chronology of answers gives those who slight the question and rush to answer a statistical advantage. The skewing of reputation points based on temporal placement of events is a challenge in learning, the study of it, and the design of learning algorithms. The goal should be (to the degree possible) chronology independence. I tried to introduce that term here: meta.stackexchange.com/questions/291364/… – Douglas Daseeco Mar 14 '17 at 17:27
  • The problem with quick answer followed by edits that flesh out and elaborate the answer is that people are also quick to downvote an answer resulting in a quick, incomplete answer getting downvoted. – Richard Chambers Oct 10 '17 at 3:19
  • It's an interesting problem. On the one hand people coming back to the site to understand a problem will upvote the best answer. So the best answer will get more votes overall. However the person asking a question is often >< close to missing a deadline and getting sacked, so for that chap, he wants the quick and dirty solution, not a protracted discourse of the whole life story and history of C++. So I don't think anything needs fixing here. There are necromancer medals on stackoverflow for a reason! – Owl Oct 25 '18 at 16:06

50 Answers 50

2

This may or may not have already been said, and as of right now, there are 39 answers. some very long-winded. I chose to be lazy and not read through them all.

One way to prevent duplicate answers from appearing right after one another is when someone goes to post, let them know if there have been any new posts since they started writing theirs. This would prevent something that's happened to me where someone was quicker to the draw in saying the exact same thing i just did.

  • 1
    It's exactly what I proposed in one of my posts. It's not hard to implement, but will reduce number of dupes. It's already implemented for topic creation - you can see in real time similar questions. I will put this request on uservoice as soon as I get spare votes there. – aku Sep 12 '08 at 5:52
  • I absolutely second this. Seeing whether somebody else (possibly even somebody with considerable reputation, increasing the probability of a high quality answer) has started answering the same question would reduce stress, unnecessary duplicate work and redundant research made my two or more people at the same time. – Pëkka Nov 1 '09 at 19:34
2

Sometimes I even see users giving replies like: "change this line of code with this one", not caring about giving an explanation of what they say. "Just do this, so I get my reputation".

And you could see low quality answers being edited and becoming better and better again in the time, but they figure first in the list of all answers, even if who gave it was slower than a user who gave a high quality and complete answer without editing it.

1

I feel that it works fairly well as it is. Using the default sort of newest first, the long thorough answer will show up first for the answers with the same vote. I do think that the 'best' answer will in most cases end up at the top of the list. If a user browsing the site really care about finding a good solution to a problem then they will look at all the answers and vote up the one(s) that prove useful.

1

Another possibility that might help is to give users a better view into what changed recently around their votes. I haven't yet found a good way to see all Q&As I voted on that have changed. Having that, it'd be easier to review one's votes, which in turn might lead to long-term improvements in vote quality.

1

I have been thinking about this problem for some time but didn't find good solution.

Since reputation system exists people want to get some reward for their answers. As number of users growth, number of duplicate answers will grow as well.

One of the useful features would be a real-time update on question's answers while you're writing your own answer. It can be implemented in the same way as search, but "search result" should be updated every N seconds/minutes. That way I can clearly see that some one else post similar answer.

Another feature that I find to be useful is merging - as @Unsliced suggested. There must be a way to merge similar answers. However it's not clear who should do it and how to divide reputation points between answering people.

1

I think SO is not only competitive. The competition is just an incentive to -in the end- provide a good (if not the best) source of information for programmers, written by programmers. The reputation feeds two things: you own ego and the confidence of the one who's reading an answer when it sees that it comes from a "guy who's not a newbie in the site and knows what he's talking about". Ot at least, you could change all that quote to: "a guy who has been hanging aroung a lot of time in SO". (which doesn't mean he's good). Badges kind of help with that. You could have 10.000 rep points, and little good badges, or 1000 rep and a few silver…

In either case, time will tell. I think that this is both competitive and collaborative with strong and weak points in each.

1

Could an answer's length be factored into the sorting algorithm somehow? How about letting the reader choose how they want their answers sorted? Much like I can go to Best Buy or Circuit City's sites and sort my search answers by brand, price, or alphabetically, I might be inclined to sort my answers descending by length if given the option... at least as a short-term experiment, although if I find the quality of answers to be better that way, I might just leave it like that forever.

1

This is a very valid problem, which I have been thinking a lot too. I have some ideas for this.

Redesign ranking functionality a bit. People can upvote (good ones can range from 1-10) or downvote (bad ones will always show as -1) but users will NOT be able to see their actual rep scores.

Use only Badges (or Grades) for display. Ranking can be used internally to determine the badges but the user will never see his rep points.

This has a psychological effect. Instead of posting a quick reply and refreshing the screen for the rep points to shoot up, users will try to write a more useful and conservative reply with code samples and will know they will get more points in the long run.

Allow users who posted the question to only Accept the answer after a day. If you want to get too draconian, allow him to accept only after upvoting or downvoting all the existing answers. :)

To eliminate fastest gun replies, answers will show up only after 5 minutes. That way, even if 10 quick guns are composing replies and posting, all those answers will show up at the same time. This will negate people to post quick replies and allow them to think at least for 5 minutes.

1

@Mel,

You have some interesting point. It will be nice to differentiate answers by various criteria, not only by votes count. Length of the answer can matter for me. Event if it's wrong, it serves as an indication that person spent some time thinking on his answer.

For example, when I browse through answers I'm interesting not only in "common opinion" but in experts decision. I don't care how much "reputation" answered person has. But I'm interested in his skill in given technology. For example there might be a little statistics attached to answer - total votes, how much people with average score per answer (with same\similar tags) > N voted, etc. Of course there might be a case when someone with tiny "reputation" provides best answer. But it's really a matter of statistics. Probability that quality of answer given by "experts" would be good is quite high. I prefer to not see "reputation" of answering person. In my opinion "reputation" doesn't matter, but specific skill does.

1

Maybe there could be a "challenge" to the selected answer. So the person asking a question gets a response and marks that as their answer, even though it may, for whatever reason, be an invalid answer. Part of it could be by adding a comment to the "selected answer" but those are somewhat buried to anyone reading the answer.

If there were a way to show that there is some dispute/challenge from people other than the questioner that the selected answer is the best answer, it should foster some discussion and perhaps even cause the questioner to change their selected answer.

Of course, this way lay flame wars, perhaps, but I'm just using this as a starting point.

1

@Justin Standard,

I think some kind of grace period might be the best thing we can come up with. Also secrecy of the ballot is the way to avoid biased opinions. But your proposition is not solving following problem:

Boundary (first/last) answers receive more attention (and votes).

It means that when votes will be revealed, there is a high probability that mostly first\last answers will be up-voted like it happens in current implementation.

Since there are "oldest" & "newest" sort modes, some answers will be less noticeable than other. To solve this issue (first->most popular answer) I proposed to disable voting for some time and pop question to main page after a while to indicate that it's ready for answer selection.

However disabling voting will make it harder for question author to select some answer in the first few minutes after grace period. So your idea might be better.

  • As some other posters have pointed out, FAST answers are a GOOD thing -- we don't want to discourage them. However, we still want BEST answers to get enough exposure to win out in the end. If a fast answer gets a lot of votes, maybe it is the intersection of FAST and BEST. – Justin Standard Sep 12 '08 at 17:49
1

Just allow the OP to select "how long before the first answer gets displayed", say, 10 minutes. If T is the original post time, then we'll call T + 10 minutes "Answer Open Time". Or we could call it Business Time, whatever you prefer.

This makes every question like a mini-contest, where people have 10 minutes or so to submit their answers. Answers are then displayed, but "early" answers are timestamped with Business Time, not the time the server originally received them. This effectively removes the fastest gun problem.

  • Interesting idea. Would require the OP to remember to do this though. – ChrisF Dec 14 '09 at 16:27
  • 6
    I don't like this at all - you'd end up with a bunch of identical answers. It would result in a lot of wasted (redundant) effort when people put time into a question that is already well answered. Besides, FGITW was largely replaced with SCITE. Any method, including this one, will be gamed just as heavily as the method that it's 'fixing'. But I believe the unintended consequences of a 10 minute wait time (redundant redundancy) greatly outweigh any benefit it might provide... – Adam Davis Dec 14 '09 at 17:16
1

I've put together a different but as it happens related question (Expiration of answers for questions with novel solutions), and a proposed solution, that I think helps resolve (or at least significantly advances discussion about) the "Fastest Gun in the West Problem" by having the "fastest answers" replaced by "better answers" as old and perhaps deprecated answer-sets are revolved by the community (i.e. wiped, to-be replaced by new sets of answers).

  • Seems to me that what you're proposing is essentially a much more naïve version of Reddit's comment sorting algorithm. – Aarobot Dec 13 '10 at 18:06
  • @Aarobot: That's an interesting reference, but I'm not sure my idea has much in common with Wilson Weighting (WW), does it? WW is a form of decay/normalization such as what I've proposed (albeit convolutedly) elsewhere: meta.stackexchange.com/questions/6662/8101#8101. The 'revolving' idea is different because (1) it's methods are different: qualified user decisions, not an automatic weighting (2) it has a different effect: revolving answer sets, not prioritizing existing answers, and (3) its purpose is different: revitalizing questions for current answers, not weighing existing answers. – Brian M. Hunt Dec 13 '10 at 20:28
  • Yeah, I know the actual implementation doesn't have much in common, but you've posted this as a solution to the FGW, and Reddit's algorithm solves it in a simple, elegant, and effective way; no offense but you're proposing something that's far more complicated (for the users) and also flimsier (since it relies on community users digging up really old questions and uses a shotgun approach to promote the best answers). The SO team has demonstrated that they can do sophisticated sorting, vis-a-vis the new "Interesting" tab on SO's front page; I'd rather see the same sort of thing for answers. – Aarobot Dec 13 '10 at 21:28
  • @Aarobot: Thanks. I'm not confident I've effectively communicated my proposal to you, which proposal resolves more issues than the FGW problem, and it obviates the need for WW-style sorting. Hence my suggestion about how the site can be changed, without hurting it, to make the FGW problem go away. :) – Brian M. Hunt Dec 14 '10 at 20:02
1

I realize now that the FGITW problem is actually the reason that Stack Overflow works well.

If you don't know enough about a topic, then you won't be able to post a quick answer.

Those who post a good quick answer, tend to be those who have more expertise in that area. This gives a higher probability of getting good expert answers.

  • 10
    The problem is not good quick answers, but bad quick answers. It generally takes longer to explain why an obvious solution that a non-expert might suggest is not a good one, than it does for a novice to post that flawed solution. – Peter Boughton Oct 6 '10 at 17:14
  • @Peter, Yes, but I'm saying that the probability will be that you get more of the good expert answers, than of the bad answers, because the non-experts won't be able to come up with those answers quick enough. So overall you'll get a strong tendency (as has been seen) for good quick expert answers to dominate. – Lance Roberts Oct 6 '10 at 17:18
  • 4
    "tend to be" is an incorrect deduction here. Those who post first, simply post quickly. They can later edit their answer to include anything they missed, and wrongly recieve credit for providing a "good fast answer", when in fact they merely provided a "fast answer". – TLP May 30 '11 at 7:52
  • 1
    FGITW being a good solution assumes that the best experts on the topic were the fastest guns, i.e. saw the question first and did all the magic it takes to bubble to the top (deploy first, then iterate until perfection). It even incentives strategic down-voting of answers that are Ok, but are stealing the show of even better answers (given, you are an awful or overly rational human being). FGITW is inherently greedy, and greedy is often sub-optimal. – Domi Nov 28 '14 at 5:40
1

Early in my experience on Stack Overflow my gut reaction to this issue was that it was a problem. I also hadn't asked any questions at that time. I soon changed my mind. I believe over time that the most correct answers do bubble up to the top.

If I ask the question I will be reading all the answers no matter what order they are in. As someone looking for an answer I will most likely be doing the same thing. To me what order they are in and how much they have been upvoted doesn't really matter if I need the information.

0

The central problem here is that while there might be several answers to a given question, in an ideal world there would only be one representative of each answer, rather than a whole muddle of similar duplicates.

Obviously we want our answer to be that representative. Our fear is that if we spend too long composing the definitive response someone else will get there first - and if they 'win' they'll get the rep (and, of course, the rep will be going to another answer that can't be as good as ours).

I like the ideal of good answers rising, but what we actually need here is probably some concept of answer merging.

It's probably not for the questioner to make the decision, they can accept an answer but there should be some mechanism for respondees to be able to merge another's answer into theirs and the product would be owned by the community wiki.

0

I think it's got to be down to the community to a large extent - and to the question owner.

Perhaps the question owner should be given more power to change the order of answers to a greater degree?

The only problem with that is that a lot of people asking questions will be 'drive-by' users, just wanting a quick answer to their problem. They're not the sort of user who will hang around to collate and manage responses.

0

Good answers will rise up. Don't worry, post a helpful answer that does not exceed half a page. Instead of "read this" or "link to that", be objective and comprehensive. Also, the first/last line (bold) as fast as possible should be the direct answer to the OP's question. These kind of answers will be rewarded with votes.

As for not reading all the way down, well it's your loss if you don't read all the answers to find the gem. Someone else definitely will. So post and help away!

Hey I made my first 1000, so post! post! post! Just kidding. 'The work is its own reward' as Jeff likes to quote.

  • Good answers will rise up. But how fast they are, years? Someone will read all the answers. But how many? Many people are very busy and they depends on the system to sorting good answers to top logically wise as soon as possible. – Luo Jiong Hui Feb 17 '16 at 18:23
  • @Luo - I was naive-r in 2008. See meta.stackoverflow.com/questions/310698 - I have changed my mind on this – Gishu Feb 23 '16 at 8:42
-5

The problem

Fast and correct, or comprehensive and insightful. (Scroll down to see suggestion)

I do not think that either type of answer is inherently better. A "good enough" answer will solve the problem and get you back on track. An insightful answer will teach you something new, make you more skilled.

The problem is that if you focus only on rewarding fast answers, many of the comprehensive answers will never get posted at all.

As it is now, the quick answers float to the top, and stay there. People do not always bother to check all the answers, and they might not consider the depth of the question, the way the asker would. If a top voted answer is good enough, chances are it will recieve more votes than it deserves, merely because it is voted highest. That is an error of conformity of the reviewing audience.

Blind votes

Unless votes are made in the blind, you are always getting a systematic error. Simply put, here's the three biggest culprits:

  1. How many upvoted it?
  2. How high up on the answer page is it?
  3. Who made the post?

That is: People will tend to conform to the majority's opinion. People will only read answers to a certain thread depth. People will tend to agree with "authorities", e.g. people with high rep, many badges, etc.

These are all basic concepts in psychological testing. There are many kind of errors that prevent you from getting correct answers. Here is a wikipedia entry on bias, take a look at Bandwagon effect and Primacy Effect.

The correct way to do it, is blind tests

The suggestion

Note: This only affects people who:

  • Are eligible to vote (due to rep or other constraints)
  • Still hold the option to vote for the question.
  • Are not the person posting the question

Anyone who just want to see the voting results can do so by clicking "finished voting".

Blind votes:

  • The author's name & rep is hidden.
  • Other people's votes are hidden until you are finished voting.
  • Answers are presented in semi-random order (1).

(1): A clearly poor answer should be nudged towards the bottom, a clearly good answer should be nudged to the top. Otherwise we are not using the expertise of our users. However, to avoid tainting by position, order is determined by vote intervals, with random order inside the intervals.

Those who do not vote

What about users who are only after answers? If you cannot or choose not to vote on a question, all the information becomes visible to you, and you can use this additional information to evaluate the quality of the answer.

The effect

By having users only vote on the merits of an answer, the validity of the vote tally will be maximized. In other words, we will know that the votes cast reflect the quality of the answer.

As it is now, the votes reflect something more than a good answer. Two identical answers may get a completely different score. Not because one person was faster in solving the problem, but because they shoved a foot in the door fastest.

How will this affect quick and dirty answers? It will still allow users to get rep for quick answers, but it will reduce the exponential effect of "being first", and also disperse some of the rep towards those who feel they have something to add, despite not being first to reply.

In short, it will be less of an "all-or-nothing" effect to being first.

Note: Please give feedback in the comments. The feedback so far has really been helpful.

  • 3
    But this ignores the population of people who look at votes not to decide how to vote (they may not even have an account) but only to know which answer to trust. That is the larger community. – Kate Gregory May 21 '11 at 19:48
  • @Kate Gregory Partially true. If you do not know which answer to trust, then I doubt that answer would benefit you much anyway. You must have some comprehension on the subject matter to understand the answers. How about if you a) cannot vote due to rep, or b) forgo voting on this question, then you can see the other votes. – TLP May 21 '11 at 20:00
  • 1
    Consider stackoverflow.com/questions/6076717/where-to-put-dllimport as a single example (I just grabbed a multi answer question from one of my tags that newbies ask in a lot) - the top vote getter is clearly better than the other two. And the whole point of this site is to serve those who cannot tell which answer is better, and to help them. – Kate Gregory May 21 '11 at 20:15
  • @Kate Gregory Well, with my suggestion that a) those who can't vote, and b) those who don't want to vote, can see the votes, that is not an issue. MY point is that if you wish to vote, then you don't get to see how everyone else voted, then you have to actually read the answers and judge them on their merit. – TLP May 21 '11 at 20:22
  • 1
    I have to say I strongly disagree with this. Yes, you can look at data which shows that higher rep users get more upvotes for more-or-less equivalent answers. However, generally speaking users with higher rep make answers that are simply better -- that's what gets them the rep in the first place. Furthermore, being able to point at an answer and say "I wrote this" is useful for employers as well as for employees showing off what they're capable of. This suggestion is completely contrary to the function of the sites which is why I downvoted. – Billy ONeal May 22 '11 at 18:24
  • E.g. I think this "cure" is worse than the disease. Furthermore it makes it impossible to identify problem users, because answers are anonymized, and moderators and people with moderator tools can't simply go to a user's profile and check their answer history to see if they're consistently a problem. Moreover, it "solves" a problem which really isn't a problem in the first place -- there are cases where this happens, but it's certainly not widespread enough to require that huge of an engine change. Finally, this answer does not answer the original question about the FGITW problem. – Billy ONeal May 22 '11 at 18:27
  • @Billy ONeal Then you have not understood my suggestion. You CAN point to an answer and say "I wrote this" because your name is only hidden to those who are still eligible to vote. The point is simply to avoid any bias that takes place before a user votes, nothing else. – TLP May 22 '11 at 18:28
  • 3
    -1 Appeal to authority fallacy. That you have a university degree in psychology is quite irrelevant. This needs to be substantiated not with philosophical argument, but rather with empirical proof that people actually do vote based on a user's rep. I don't think that exists, in fact I've seen quite the opposite. – Cody Gray May 22 '11 at 18:29
  • @TLP: Not true. Sure, if I'm logged in I can show an employer something, but what if I'm just posting a link? The employer is an "eligible voter" and therefore would not be able to see who wrote the answer. – Billy ONeal May 22 '11 at 18:29
  • @Billy ONeal The employer can then simply click the "done voting" button and see the results. Assuming he had enough rep to vote to begin with. I do not see the problem. – TLP May 22 '11 at 18:32
  • 1
    @TLP: Well, then we can agree to disagree. I think it's a horrible idea. However, this answer would still get a -1 from me if nothing else that it does not talk at all about the FGITW problem (the topic of this question). – Billy ONeal May 22 '11 at 18:34
  • @Cody Then you would say that Billy ONeal also made this mistake by saying that higher rep people generally post better answers? And as such, you do agree that showing rep before votes are cast is an "appeal to authority fallacy". – TLP May 22 '11 at 18:34
  • 1
    Not necessarily. Reputation is a direct indicator of the quality of answers you've posted on this site. A university degree isn't necessarily the same thing. Although that part of the comment was kind of a joke, contrasting your bristling at downvotes and justification of your theories with the very same underlying logic that you appear to criticize. – Cody Gray May 22 '11 at 18:38
  • 1
    @Cody I am not saying votes are primarily based on rep, I am saying that they boost the effect of a good answer. But I do not think rep is the biggest factor in this case. More likely a high score and top position causes more bias than rep. – TLP May 22 '11 at 18:58
  • 2
    "Fast and correct, or comprehensive and insightful." In my experience, this is very much a false choice. The FGITW problem is generally understood to mean that someone will post a fast and correct answer to start with, and then (usually within the 5 minute window) edit it to be comprehensive and insightful. In the cases where I've used my fast gun, this is what I do. In the cases I've seen Jon Skeet et. al. use his fast gun, this is what he does. It's not necessary to pick one or the other. It generally ends up that the answer that gets the upvotes deserves most, if not all, of those. – Cody Gray May 23 '11 at 3:28
-10

I know, I am quite late in reacting to this question.

So my answer will suffer with the problem what this post is trying to state.

This answer would be the last one in the list, a lot below the highly upvoted answers.

Now the solution I would like to suggest for the problem is:

  1. Give priority to new suggestions and answers. I am not omitting the usefulness of the old ones, but we should be soft enough to welcome the new suggestions.

  2. If upvoted questions are given priority over other ones, then downvoted questions should be given minority over others.

  3. The mark-it-as-a-useful answer trick should be used to decide the priority of a good answer and the mark-it-as-spam trick should be used to decide the location of the answer down the hill.

  • 11
    If you order the answers by "active" then your answer will appear first. – Lix Sep 15 '12 at 10:21

You must log in to answer this question.

protected by Jeff Atwood Aug 8 '11 at 5:13

Thank you for your interest in this question. Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged .