Fastest Gun in the West Problem

Question

I feel like there's a problem with Stack Overflow, as the number of people prowling it increases.

Each question's answers are sorted by descending score and then by descending time of posting. This means that if a person sits down and answers a question in a long, thorough way, going through every nook and cranny, once they post their answer, it will already be one of about seven different ones, some of which have already been upvoted. This wouldn't be a problem if those answers were as thorough as the one this guy's posting, but they usually aren't. Some of them are downright wrong, some aren't even answers to the question asked because their poster didn't bother to read the question all the way through.

This causes a problem I like to call Stack Overflow's Fastest Gun in the West Problem. I've come to a point where I'd rather just send a short, simple, correct explanation, than to go and do some proper research, write a whole blog post about it or even make sure the code I post even compiles, just so it will be noticed, as opposed to the incorrect ones.

I'm sure I'm not the only one doing this and that it despairs many people from even trying to answer questions.

I've long ago learned to try to always raise a problem with a solution in hand, rather than just say "This is a problem, handle it," so my question, after this long tirade, is:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

I promise to vote on answers I like, even if it takes you a long while to post them. :)

Answer 1

Another possibility that might help is to give users a better view into what changed recently around their votes. I haven't yet found a good way to see all Q&As I voted on that have changed. Having that, it'd be easier to review one's votes, which in turn might lead to long-term improvements in vote quality.

Answer 2

I think that to the degree this is a problem it's mostly a problem with the beta. The limited user pool means that sometimes answers may languish a while before they get attention. When the user pool grows by several orders of magnitude I don't think this issue will be as important.

Answer 3

I've put together a different but as it happens related question (Expiration of answers for questions with novel solutions), and a proposed solution, that I think helps resolve (or at least significantly advances discussion about) the "Fastest Gun in the West Problem" by having the "fastest answers" replaced by "better answers" as old and perhaps deprecated answer-sets are revolved by the community (i.e. wiped, to-be replaced by new sets of answers).

Answer 4

@contagious,

I created a request on UserVoice:

Provide live monitoring of answers being posted

To help us fight against tons of duplicate answers, provide following functionality:

At the time person is writing his answer show excerpts from answers being posted. Color code similar answers.

It can be implemented in the same way it's done for new question posting - i.e. author can see that similar posts already exist.

If it's what you mentioned, please vote for it.

Answer 5

The central problem here is that while there might be several answers to a given question, in an ideal world there would only be one representative of each answer, rather than a whole muddle of similar duplicates.

Obviously we want our answer to be that representative. Our fear is that if we spend too long composing the definitive response someone else will get there first - and if they 'win' they'll get the rep (and, of course, the rep will be going to another answer that can't be as good as ours).

I like the ideal of good answers rising, but what we actually need here is probably some concept of answer merging.

It's probably not for the questioner to make the decision, they can accept an answer but there should be some mechanism for respondees to be able to merge another's answer into theirs and the product would be owned by the community wiki.

Answer 6

Maybe there could be a "challenge" to the selected answer. So the person asking a question gets a response and marks that as their answer, even though it may, for whatever reason, be an invalid answer. Part of it could be by adding a comment to the "selected answer" but those are somewhat buried to anyone reading the answer.

If there were a way to show that there is some dispute/challenge from people other than the questioner that the selected answer is the best answer, it should foster some discussion and perhaps even cause the questioner to change their selected answer.

Of course, this way lay flame wars, perhaps, but I'm just using this as a starting point.

Answer 7

I realize now that the FGITW problem is actually the reason that Stack Overflow works well.

If you don't know enough about a topic, then you won't be able to post a quick answer.

Those who post a good quick answer, tend to be those who have more expertise in that area. This gives a higher probability of getting good expert answers.

Answer 8

I think SO is not only competitive. The competition is just an incentive to -in the end- provide a good (if not the best) source of information for programmers, written by programmers. The reputation feeds two things: your own ego and the confidence of the one who's reading an answer when it sees that it comes from a "person who's not a newbie in the site and knows what they are talking about". Or at least, you could change all that quote to: "a person who has been hanging around a lot of time on SO". (Which doesn't mean they are good). Badges kind of help with that. You could have 10.000 rep points, and little good badges, or 1000 rep and a few silver ones.

In either case, time will tell. I think that this is both competitive and collaborative with strong and weak points in each.

Answer 9

@Mel,

You have some interesting point. It will be nice to differentiate answers by various criteria, not only by votes count. Length of the answer can matter for me. Event if it's wrong, it serves as an indication that person spent some time thinking on his answer.

For example, when I browse through answers I'm interesting not only in "common opinion" but in experts decision. I don't care how much "reputation" answered person has. But I'm interested in his skill in given technology. For example there might be a little statistics attached to answer - total votes, how much people with average score per answer (with same\similar tags) > N voted, etc. Of course there might be a case when someone with tiny "reputation" provides best answer. But it's really a matter of statistics. Probability that quality of answer given by "experts" would be good is quite high. I prefer to not see "reputation" of answering person. In my opinion "reputation" doesn't matter, but specific skill does.

Answer 10

I have been thinking about this problem for some time but didn't find good solution.

Since reputation system exists people want to get some reward for their answers. As number of users growth, number of duplicate answers will grow as well.

One of the useful features would be a real-time update on question's answers while you're writing your own answer. It can be implemented in the same way as search, but "search result" should be updated every N seconds/minutes. That way I can clearly see that some one else post similar answer.

Another feature that I find to be useful is merging - as @Unsliced suggested. There must be a way to merge similar answers. However it's not clear who should do it and how to divide reputation points between answering people.

Answer 11

I feel that it works fairly well as it is. Using the default sort of newest first, the long thorough answer will show up first for the answers with the same vote. I do think that the 'best' answer will in most cases end up at the top of the list. If a user browsing the site really care about finding a good solution to a problem then they will look at all the answers and vote up the one(s) that prove useful.

Answer 12

Could an answer's length be factored into the sorting algorithm somehow? How about letting the reader choose how they want their answers sorted? Much like I can go to Best Buy or Circuit City's sites and sort my search answers by brand, price, or alphabetically, I might be inclined to sort my answers descending by length if given the option... at least as a short-term experiment, although if I find the quality of answers to be better that way, I might just leave it like that forever.

Answer 13

This is a very valid problem, which I have been thinking a lot too. I have some ideas for this.

Redesign ranking functionality a bit. People can upvote (good ones can range from 1-10) or downvote (bad ones will always show as -1) but users will NOT be able to see their actual rep scores.

Use only Badges (or Grades) for display. Ranking can be used internally to determine the badges but the user will never see his rep points.

This has a psychological effect. Instead of posting a quick reply and refreshing the screen for the rep points to shoot up, users will try to write a more useful and conservative reply with code samples and will know they will get more points in the long run.

Allow users who posted the question to only Accept the answer after a day. If you want to get too draconian, allow him to accept only after upvoting or downvoting all the existing answers. :)

To eliminate fastest gun replies, answers will show up only after 5 minutes. That way, even if 10 quick guns are composing replies and posting, all those answers will show up at the same time. This will negate people to post quick replies and allow them to think at least for 5 minutes.

Answer 14

Just allow the OP to select "how long before the first answer gets displayed", say, 10 minutes. If T is the original post time, then we'll call T + 10 minutes "Answer Open Time". Or we could call it Business Time, whatever you prefer.

This makes every question like a mini-contest, where people have 10 minutes or so to submit their answers. Answers are then displayed, but "early" answers are timestamped with Business Time, not the time the server originally received them. This effectively removes the fastest gun problem.

Answer 15

I think it's got to be down to the community to a large extent - and to the question owner.

Perhaps the question owner should be given more power to change the order of answers to a greater degree?

The only problem with that is that a lot of people asking questions will be 'drive-by' users, just wanting a quick answer to their problem. They're not the sort of user who will hang around to collate and manage responses.

Answer 16

Early in my experience on Stack Overflow my gut reaction to this issue was that it was a problem. I also hadn't asked any questions at that time. I soon changed my mind. I believe over time that the most correct answers do bubble up to the top.

If I ask the question I will be reading all the answers no matter what order they are in. As someone looking for an answer I will most likely be doing the same thing. To me what order they are in and how much they have been upvoted doesn't really matter if I need the information.

Answer 17

In response to @Justin Standard's answer

I think some kind of grace period might be the best thing we can come up with. Also secrecy of the ballot is the way to avoid biased opinions. But your proposition is not solving the following problem:

Boundary (first/last) answers receive more attention (and votes).

It means that when votes will be revealed, there is a high probability that mostly first\last answers will be upvoted like it happens in the current implementation.

Since there are "oldest" & "newest" sort modes, some answers will be less noticeable than others. To solve this issue (first->most popular answer), I proposed to disable voting for some time and pop the question to the main page after a while to indicate that it's ready for answer selection.

However, disabling voting will make it harder for the question author to select some answer in the first few minutes after the grace period. So your idea might be better.

Answer 18

Good answers will rise up. Don't worry, post a helpful answer that does not exceed half a page. Instead of "read this" or "link to that", be objective and comprehensive. Also, the first/last line (bold) as fast as possible should be the direct answer to the OP's question. These kind of answers will be rewarded with votes.

As for not reading all the way down, well it's your loss if you don't read all the answers to find the gem. Someone else definitely will. So post and help away!

Hey I made my first 1000, so post! post! post! Just kidding. 'The work is its own reward' as Jeff likes to quote.

Answer 19

The problem

Fast and correct, or comprehensive and insightful. (Scroll down to see suggestion)

I do not think that either type of answer is inherently better. A "good enough" answer will solve the problem and get you back on track. An insightful answer will teach you something new, make you more skilled.

The problem is that if you focus only on rewarding fast answers, many of the comprehensive answers will never get posted at all.

As it is now, the quick answers float to the top, and stay there. People do not always bother to check all the answers, and they might not consider the depth of the question, the way the asker would. If a top voted answer is good enough, chances are it will recieve more votes than it deserves, merely because it is voted highest. That is an error of conformity of the reviewing audience.

Blind votes

Unless votes are made in the blind, you are always getting a systematic error. Simply put, here's the three biggest culprits:

1. How many upvoted it?
2. How high up on the answer page is it?
3. Who made the post?

That is: People will tend to conform to the majority's opinion. People will only read answers to a certain thread depth. People will tend to agree with "authorities", e.g. people with high rep, many badges, etc.

These are all basic concepts in psychological testing. There are many kind of errors that prevent you from getting correct answers. Here is a wikipedia entry on bias, take a look at Bandwagon effect and Primacy Effect.

The correct way to do it, is blind tests

The suggestion

Note: This only affects people who:

• Are eligible to vote (due to rep or other constraints)
• Still hold the option to vote for the question.
• Are not the person posting the question

Anyone who just want to see the voting results can do so by clicking "finished voting".

Blind votes:

• The author's name & rep is hidden.
• Other people's votes are hidden until you are finished voting.
• Answers are presented in semi-random order (1).

(1): A clearly poor answer should be nudged towards the bottom, a clearly good answer should be nudged to the top. Otherwise we are not using the expertise of our users. However, to avoid tainting by position, order is determined by vote intervals, with random order inside the intervals.

Those who do not vote

What about users who are only after answers? If you cannot or choose not to vote on a question, all the information becomes visible to you, and you can use this additional information to evaluate the quality of the answer.

The effect

By having users only vote on the merits of an answer, the validity of the vote tally will be maximized. In other words, we will know that the votes cast reflect the quality of the answer.

As it is now, the votes reflect something more than a good answer. Two identical answers may get a completely different score. Not because one person was faster in solving the problem, but because they shoved a foot in the door fastest.

How will this affect quick and dirty answers? It will still allow users to get rep for quick answers, but it will reduce the exponential effect of "being first", and also disperse some of the rep towards those who feel they have something to add, despite not being first to reply.

In short, it will be less of an "all-or-nothing" effect to being first.

Note: Please give feedback in the comments. The feedback so far has really been helpful.

Answer 20

I know, I am quite late in reacting to this question.

So my answer will suffer with the problem what this post is trying to state.

This answer would be the last one in the list, a lot below the highly upvoted answers.

Now the solution I would like to suggest for the problem is:

1. Give priority to new suggestions and answers. I am not omitting the usefulness of the old ones, but we should be soft enough to welcome the new suggestions.

2. If upvoted questions are given priority over other ones, then downvoted questions should be given minority over others.

3. The mark-it-as-a-useful answer trick should be used to decide the priority of a good answer and the mark-it-as-spam trick should be used to decide the location of the answer down the hill.

Answer 21

Holy hell! How did this transform from:

How do you think this can be changed? What would you change in Stack Overflow to make this problem go away without hurting the site?

to

Competitive vs Collaborative, Hiding vs Showing votes, and a bunch of dead links.

This just shows me that SO/SE can't hold a topic to save its life.

Well, this question has been asked 10 years ago. So I would try to answer it with opinions and suggested changes that apply to modern SO/SE:

To try and give an concise answer to your first paragraph after the introduction:

It seems that same topic holding mechanism that occurred on your question happens on every other one, even to this day. People have ADHD... or something. It seems that most of users tend to not reserve part of their attention to the questions at hand and to push answers on skimming. This happens a lot in day to day conversation: people tend to focus on part of the conversation, and often that part either proves their point or people are plain out against it. I am afraid that there isn't a way to battle this. Behaviour like this first needs to change in our everyday IRL lives than to transgress into online communication.

"Stack Overflow's Fastest Gun in the West Problem" if we govern ourselves by short attention span, it is logical that we have policy "shoot first - ask questions later". Thorough answers you mentioned, are high in quality, but low in value. I've mentioned this in previous questions/answers. But what does it matter if the future does not present sustainability? By giving short non-elaborative answers it reduces to meta keywords answer/question that can be searched for which it generates an probability for dupes. Because if an answer isn't clear to everyone, a new user (or old) will ask it again.

How do you think this can be changed?

Well, we can't change human communication IRL. But we can change the visibility of communication. If a topic is interesting enough for the user, it is logical that the same user will come back to revisit new answers. So far only pings are intentional, either by comment or answer to the original poster.

What would you change in Stack Overflow to make this problem go away without hurting the site?

I think we can give more value to the topic if we ping every time a new answer is posted to the question we have already given answer. So in this case every person that has posted an answer to this question should get an notification that I have added another answer to it. They might choose to ignore it, vote on it, downvote it or comment on it. But in the end it is the users choice.

That way thorough late answers could get attention they deserve, and they would become long investment instead of short. This should in turn incentivise users to grant a tad bit more attention to the answers instead of skimming through to post the quickest one.