Is the fastest gun in the west problem solved? Is it really a problem?
-
6darnit, I was going to ask the same question, missed it by 3 hours I guess. I was going to call it the "Sundance Kid" problem. :-)– Jason SCommented Jun 28, 2009 at 15:12
-
13I don't mind seeing the early votes. I just hate it when an answer is immediately accepted. I feel like better answers aren't even attempted when that happens.– NosrednaCommented Jun 29, 2009 at 4:42
-
3Starting bounty because I really like Greg's solution– PekkaCommented Sep 18, 2011 at 12:40
-
Is it just me? I have no frakking idea what's going on here!– Lightness Races in OrbitCommented Sep 18, 2011 at 22:19
-
@pekka why are you dredging this up now? Random orderings of posts with the same score was, and is, the best solution. There is no FGITW problem that I am aware of after that change, and according to at least one academic study based on our data, there never was a FGITW problem to begin with..– Jeff AtwoodCommented Sep 19, 2011 at 5:49
-
1@Jeff I wasn't focusing on the FGITW problem at all, but on hiding vote counts on answer for some time (e.g. for as long as the question is on the front page) to avoid the bike shed effect. But this may be the wrong proposal to support in that vein, fair enough– PekkaCommented Sep 19, 2011 at 8:47
-
I really want to post an answer saying only 'NO.', but that doesn't have enough characters (nothing a few empty pictures can't remedy), and I would be a 'slow gun', not a fast one. (2+ years late.)– Mateen UlhaqCommented Nov 2, 2011 at 4:18
-
Surprisingly, the "mother" question was not linked to this one. I added the link.– Daniel DaranasCommented Dec 29, 2011 at 13:59
Add a comment
|
14 Answers
I had an idea for this that I submitted to uservoice a while ago, but I can't seem to find it now so I'll recreate it:
When casting a vote, the effect of that vote would be delayed for all other users for five minutes or something. You would be able to see the vote effect, so that you could easily confirm that your vote was actually cast. But everybody else would see the effect of that vote five minutes later.
This would allow each answer to be voted on its own merit, at least for the first few minutes. From the point of view of a user, the site would appear to work exactly the way it does now except it would look like everybody else was off having a coffee break whenever you cast a vote.
answered Jun 28, 2009 at 11:52
-
53
-
3But the first answer would still be at the top, and if we've learned anything from elections, its that being on top is a HUGE advantage. Commented Jun 28, 2009 at 18:57
-
28Perhaps the default sort could randomise the order of posts with the same number of votes. Then the only bias toward the first answer would be that it would be visible the longest. Commented Jun 28, 2009 at 19:29
-
3I really like this idea, and the mixed ordering idea, also perhaps, if you can only accept an answer after an hour would help encourage higher quality answers– wafflesCommented Jun 29, 2009 at 1:42
-
3What I don't like in this solution is that question view would be inconsistent for different users.– SergCommented Jul 1, 2009 at 9:56
-
2I disagree. This would make a great point of SO moot: Currently, garbage answers will get down immediately.– mmxCommented Jul 13, 2009 at 17:57
-
I'd +1 but that would change the votes away from forty-two... Commented Aug 2, 2010 at 11:52
-
10I see that despite Jeff liking this proposal, it's still not implemented. I'm assuming the problem is that an answer that may only deserve a +3 score gets a +10 score because nobody knew that it was already voted up before they got to it. If so, the answer is simple, just display all answers in a random order, regardless of score, for the first hour or so after a question is posted. Then everybody can see the scores and vote up/down accordingly, and there's no inherent bias towards early answers. If an answer is accepted, of course, that could go right to the top anyway. Commented Mar 7, 2011 at 22:20
-
@Greg: I like your proposal, though I don't understand why all votes have to be delayed for five minutes. I suggested a modified solution below (at the bottom of this list ;) Commented Sep 18, 2011 at 22:29
-
6@sskuce - No answer "deserves" a +3 score and not a +10 score. Upvote if it's useful, downvote if it's not, don't vote if you're not sure. Commented Oct 7, 2011 at 21:17
-
The Gregs proposal is great, however I'd extend the 5 minutes to 1 hour. 5 minutes is still too short if we want to get higher answer quality.– TomasCommented Oct 13, 2011 at 9:33
-
@sskuce, agree to the Kevin's comment, moreover, by displaying the vote count you totally negate the genial Gregs proposal. Of course there would be a bias, because the people would look at higher voted answers first, regardles of the order.– TomasCommented Oct 13, 2011 at 9:35
-
No matter what Jeff says, Fastest Gun in the West is still a massive problem:
If you are the first to post an answer you have an almost 50% chance of it getting accepted, if you post the 5th answer you only have a 2% chance of getting your answer accepted.
0 50477 46.305 1 26680 24.475 2 14516 13.316 3 7609 6.980
-
24OTOH, if you have four answers, at least one of them should be pretty good. People with taste are more likely to censor themselves. Fifth answers are often (but not always) redundant and clueless. Commented Jun 28, 2009 at 12:38
-
-
3Lots of people (and Jeff) don't like this statistic, on one hand, when an answer is posted you have no idea if 10 answers or 1 answer will be posted, so the stat is valid. It basically saying that if you are first its more likely your answer will be accepted, similarly being jon skeet or better still rq, makes it much more likely your answer will get accepted. It does not take into account the number of answers or if the question is a community wiki (can not do this yet)– wafflesCommented Jun 29, 2009 at 1:48
-
Hm, that's a good point about one-answer questions.. I've relayed that bit of your question into a comment on stackoverflow.com/questions/951056/… (yay, circular linking!)– dbrCommented Jun 29, 2009 at 1:58
-
How do these figures stack up for questions with exactly four answers? Commented Jul 15, 2009 at 16:06
-
Note to self: read rest of answers, as Jeff answers this (albeit using 5 as the magic number) Commented Jul 15, 2009 at 16:08
-
2Late to the show here, but this doesn't take into account the likelihood of the first answer being at least as good as any subsequent answers, which I would argue is also quite likely for most questions.– Rex MCommented Sep 21, 2009 at 4:34
-
I suspect that this understates the problem. Knowing the fastest-gun problem, people will only take the time to write a good answer if they think they can answer significantly better. So we should expect later answers to be better; and yet they are being accepted less. Commented Jul 16, 2012 at 1:48
I guess the question is really: is it a problem?
There are several different aims to the site:
- Get a specific answer to a question as quickly as possible for the sake of the questioner
- Get a full, detailed answer to a question over time for the sake of future visitors
- Allow developers to show off :)
Getting a quick answer in serves two purposes:
- It may serve the first site purpose in minutes (or less - I think my fastest was to have an answer posted and accepted while the question was still saying "1 minute ago").
- It shows other potential answerers that you're answering along a particular line of thought. For instance, if I see Marc Gravell answering about a particular topic, even with just a quick answer, I probably won't bother any more; he's likely to say everything I would, unless I've got a completely different idea.
The main problem I see is that of positive feedback - an earlier answer will often get more votes (and keep getting more votes) than a better, later answer. That's not always the case, but it does happen. (And I'm fully aware that I'm usually the beneficiary of that.)
answered Jun 28, 2009 at 11:53
-
3There's "as quickly as possible" and there's "quickly". I think a 5 min delay, as suggested, would not diminish the value of SO as a place where you can get quick answers. Commented Jun 28, 2009 at 15:29
-
17Delaying the effect of votes by five minutes would be reasonable - and may well reduce the problem. Not sure about delaying the actual answers themselves by five minutes - that removes the benefit of my bottom bullet, and may actually lead to more answer duplication. Commented Jun 28, 2009 at 15:42
-
I fully agree with Jon, and it's what I have been doing. I still think it's something that should be done automatically, though. A more dynamic site? However, I think Jon forgets a fourth important aim, which is central to me: give a good use to my time. I.e. not to spend 20 minutes or more writing a reasonable answer only to find out another person wrote almost exactly the same. I'd much rather prefer to use the time in perfecting that answer (if possible) or answering another question. Commented Jul 29, 2014 at 2:49
Let's look at whether it's really a problem. To find out, we need to focus on questions that:
- Have more than one submitted answer
- Have one accepted answer
- Have not been closed
Other answers haven't included these criteria, and I think that's a big problem. For example, if you don't filter out questions with only one answer, well then of course the first answer is going to be much more likely to be accepted - because duh, it's the ONLY answer.
If you're playing along at home with SQL Server, the query to find the questions I'm focusing on is:
SELECT COUNT(DISTINCT pQ.Id) AS Questions
FROM dbo.Posts pQ
WHERE pQ.PostTypeId = 1
AND (SELECT COUNT(*) FROM dbo.Posts pA WHERE pQ.Id = pA.ParentId) > 1
AND pQ.AcceptedAnswerId IS NOT NULL
AND pQ.ClosedDate IS NULL
Out of those 88,975 (Oct 19, 2014: 2,379,173) questions, how many of them had the first answer accepted? The query I used was:
SELECT COUNT(DISTINCT pQ.Id) AS Questions
FROM dbo.Posts pQ
INNER JOIN dbo.Posts pA1 ON pQ.AcceptedAnswerId = pA1.Id
WHERE pQ.PostTypeId = 1
AND (SELECT COUNT(*) FROM dbo.Posts pA WHERE pQ.Id = pA.ParentId) > 1
AND pQ.ClosedDate IS NULL
AND NOT EXISTS (SELECT Id FROM dbo.Posts pA WHERE pQ.Id = pA.ParentId AND pA1.CreationDate > pA.CreationDate)
Turns out 49,989 (Oct 19, 2014: 1,013,791) of the first answers were accepted, or about 56% (Oct 19, 2014: 42.6%) of the time.
The AND (SELECT COUNT(*) FROM dbo.Posts pA WHERE pQ.Id = pA.ParentId) > 1 was missing here.
If the SQL query as it was was really what Brent Ozar used, this number is likely to be in excess of the "correct" number.
The next question would be, "How often was a later answer scored higher by the viewers?" The query I used was:
SELECT COUNT(DISTINCT pQ.Id) AS Questions
FROM dbo.Posts pQ
INNER JOIN dbo.Posts pA1 ON pQ.AcceptedAnswerId = pA1.Id
INNER JOIN dbo.Posts pA2 ON pA2.ParentId = pQ.Id AND pA1.CreationDate < pA2.CreationDate AND pA2.Score > pA1.Score
WHERE pQ.PostTypeId = 1
AND pQ.ClosedDate IS NULL
AND NOT EXISTS (SELECT Id FROM dbo.Posts pA WHERE pQ.Id = pA.ParentId AND pA1.CreationDate > pA.CreationDate)
Turns out 3,167 (Oct 19, 2014: 128,257) times, a later answer has a higher score than the accepted (earlier) answer.
That's 6.34% of the first answer accepted had a later answer with a higher score in June 2009 and now, in October 2014, it's 12.65%.
With the June 2009 "first answer accepted" statistic suspect, comparing a later answer with higher score as a percentage of multiple answer questions: it was 3.56% in 2009 and 5.39% in 2014.
The SEDE queries.
-
The history of views by date isn't stored in the data dump, so I can't determine that. Commented Jun 28, 2009 at 13:22
-
1those stats say nothing about whether the first or later answers were the better answer. Commented Jun 30, 2009 at 14:30
-
1Sam - how do you propose to measure that statistically from the database? Commented Jun 30, 2009 at 15:02
-
1
The premise of the question is invalid: it is based on an assumption that the "fastest gun in the west" phenomenon is a problem, which I believe to be incorrect. One of the problems with forums is noone posts there because it takes a day or two (minimum) to get an answer. The fact that the system here encourages quick answers is a good thing. Usually if you have a programming related problem you need an answer now.
I'm brand new to Stack Overflow and think the speed thing is a problem. I'm enjoying answering questions and love the feedback from reputation. What I hate is writing a detailed, reasoned answer and suddenly getting an Ajax notification "there are 2 answers; reload to see them". I feel like I have to beat the timer. I'm already answering questions more quickly or leaving a short answer to go back and edit later.
I think the solution is to somehow delay votes and randomize the order for a few minutes until several answers come in. Then let the best answer win.
-
Nelson, the solution to this is to write an accurate, but not detailed, answer, post it, then come back and add detail. I'll often post an answer, come back and add links and post that, then come back and add the code sample. Commented Jul 31, 2009 at 16:01
-
6Which is awkward. I'd rather just write the answer and have done with it. Commented Jul 31, 2009 at 16:40
-
1
Here's some other data that Sam Saffron put together
Looking only at questions with 5 answers:
Answer # # of Qs Percentage of total 0 2524 26.196 1 1982 20.571 2 1918 19.907 3 1715 17.800 4 1496 15.527
where:
- Answer # is whether that answer was the first, second, third, etc to arrive, and
- # of Qs is the number of questions where that is the accepted answer.
Looking only at questions that received 2 answers within the first 5 minutes
Answer # # of Qs Percentage of total 0 8069 35.073 1 6508 28.288 2 3972 17.265 3 2094 9.102 4 1054 4.581 5 558 2.425
Personally, I think that looks totally reasonable
answered Jun 28, 2009 at 12:43
-
1It may look like being first is a slight advantage, but I'd add to these stats the warning that correlation does not imply causation. It is often the case that the first answer is the obvious answer to an easy question, and so the first answer does have a slightly higher likelihood of being the right answer.– Bill the Lizard ModCommented Jul 31, 2009 at 18:23
-
@Bill If that's not a form of causation, I don't know what else. Commented Aug 13, 2013 at 6:05
-
@MarioRossi The question here is whether being the first to answer gives an advantage in being upvoted. I'm saying that it doesn't, but being right does. The first answer isn't always right, but it often enough is that the two can be confused.– Bill the Lizard ModCommented Aug 13, 2013 at 11:03
-
@Bill On the contrary, the question here is FGITW. From that point of view, we should consider only the second table, which is saying that answering a question within the first 5 minutes gives you a chance of 63%. Nevertheless, my main interest here is not votes or reputation but the strong feeling of wasting my time. Commented Aug 14, 2013 at 0:07
I don't really think it's a problem. I'm rarely the first to answer, which means I have to write better (or at least longer ;)) answers in order to catch up. I'll never catch up with Jon Skeet rep-wise, but I've got 22k so far, and almost all of it has been earned on questions where I was not the first to answer.
It doesn't really bother me. The questions where "first answer wins" are usually the ones where only a few upvotes are going to be given anyway. In the longer-running questions, where all the real upvotes are given, there's plenty of time to jockey for position.
-
As I have said several times before, the point is not reputation. It is how many times you have felt you have wasted part of your time by repeating things already said, and how that time could have been used in answering (or perfecting the answer of) other questions. Commented Aug 13, 2013 at 6:07
Two-minute initial no-voting period?
Three-minute prohibition on edits for answers posted very quickly; say, within a minute of question posting?
Here's a metrics question:
Of the questions that have the first 2 answers within 10 minutes, what's the upvote difference between the first and second answers, and is that upvote difference statistically significant?
I've had several occasions where I've submitted a nearly-identical answer within 60 seconds of the first one, and there were a huge difference in votes. Weird... first because I would have expected this forum to be more merit based (as long as answers come in quickly, who cares whether one arrives in 4 minutes and the other 4 minutes 30 seconds), and secondly because why should I care about votes anyway? I have this strange urge to be less rigorous, go ahead and post my quick answer first, then go back and edit it to have the right links & stuff, especially since there seems to be a time period after the initial post when corrections/additions are accepted without it being called an "edit".
I'd suggest deciding what qualities of answers you want first (correct & complete answers that are reasonably quick) and coming up with social rules to encourage those qualities. SO's pretty good but issues like this make me wonder.
-
It's a slow query, but here it is. The answer, for the 155117 questions on SO with at least 2 answers within ten minutes to 30/5/2010, the first answer has a score of 0.69 more than the second answer. (I know you asked for up-votes only, but the query took 104 seconds as is!) Commented Jun 10, 2010 at 10:59
-
Upvotes and downvotes are NOT the point!!! The point is the best usage of our collective time. A proper statistic would be how different the responses during the first 5 or 10 minutes are. Difficult to measure, but in my experience you will see a lot of repetition and thus waste of time. Commented Aug 13, 2013 at 6:13
My suggestion
is similar to Greg's. But instead of delaying every vote for 5 minutes, I would:
- show the answers immediately,
- but show no votes at all (except your own) for the first 10 minutes or so after the question has been posted. (For viewers, instead of
0the vote field could displayvotes show in 3 min). - Then after the 10 minutes have passed, all votes will be displayed at once, and the usual process takes place, as it is right now (no more delays).
The part of my old answer
for which I got some down-votes and the bashing comments below:
I really feel this being a big problem. That's why I try to commit my answers quickly, only with the main aspects. Afterwards, I'm re-editing it to fill in details (like source code examples) to improve its quality. But this process is:
- not really comfortable, since I have to go over my answer at least twice and I find it kind of stressfull to be one of the fastest guns in the west (especially when some "popups" show how many others have answered in the mean time).
- Sometimes it results in weird comments that are only valid for a couple of minutes, until I have filled in detailed examples.
- Here is an example where I even got a down-vote that was not corrected after I re-edited my answer.
answered Sep 18, 2011 at 12:36
-
2Reading the above, I feel you are part of the problem, not suffering it?– ArjanCommented Sep 18, 2011 at 16:43
-
I'd say the problem is the system and the voters. Furthermore, I do improve my answers quite quickly, and when I vote answers up, I look through other answers and upvote similar ones as well, even if they don't have upvotes yet. Commented Sep 18, 2011 at 16:50
-
2I disagree. If you're getting "weird comments" and even downvotes, then I'd say your first answer was just not ready for posting. So, you're just trying to get to be the first answerer, as you're aware that first answers are likely to get more votes? I dislike that, a lot.– ArjanCommented Sep 18, 2011 at 17:00
-
I try to post only when the first answer does make sense already. I checked 10 of your answers, 9 of them edited by yourself afterwards. Maybe your first answers were just not ready for posting ;) Commented Sep 18, 2011 at 18:02
-
Sure I edit after posting too, sometimes even in the 5 minute grace period. But I've never received weird comments or downvotes for any first revision. And I surely don't post partial answers knowing that I am ging to improve it right away. How many of those 10 were first answers?– ArjanCommented Sep 18, 2011 at 18:07
-
I haven't looked for that. In fact I do like it when people improve their answers... And I think the same applies for me: I used to write very thorough answers that subsume several others and therefore were committed quite late. The others all got a lot of upvotes, mine maybe 1 or 2 by "late viewers". Only after several such situations I changed my behavior and started to list all the aspects, then commit the answer and later-on edit the answer if I want to add details to the aspects. Hope with this explanation you agree that the root cause of the problem is the voting behavior/system itself. Commented Sep 18, 2011 at 18:17
-
I don't see how my comments could be interpreted as bashing, but they surely were not intended that way. Sorry about that. That aside: I wonder if the quick non-revoked downvote on your example is related to this. To me, it surely is an example of FGITW: a single-line answer posted within 3 minutes after the question was asked, and then enhanced a lot 7 to 12 minutes later. But why couldn't the downvoter simply disagree with both the short and long answer?– ArjanCommented Sep 19, 2011 at 16:27
-
Sure, maybe he disagreed and did not comment on it. I did feel a little bashed here because I mainly want to give good answers, only secondly fast ones. Furthermore, many others have mentioned in this thread that they are using the same technique, without getting down-votes. That no longer seemed objective to me. Commented Sep 19, 2011 at 17:50
I'll answer from my personal point of view, which is not upvotes, downvotes, selection of the best answer or even reputation. My main concern is: wasting my time, or if you prefer, not being collectivelly more efficient. How many times I have spent 5 or more minutes writing an answer only to discover that another person was producing / had produced a very similar if not exact same one?
Some strategies that I have tried:
- Answer only questions that have been unanswered for a couple of hours. Challenges: To skip questions with only wrong answers (usually just one or two). Not providing additional points of view.
- Answer only questions that after a couple of hours have only 3 answers. Challenges: In many cases the 3 answers are duplicate (and possibly wrong) ones. Very few questions to answer at all; usually only those very badly stated. Additionally, I have noticed that duplicate answers are produced even after much more time than writing it would have normally required. A bit off-topic, but this would mean: a) We are extremely slow typists, b) Many of us abandons SOf for important periods, only to come back and press "submit", c) We answer questions without reading previous answers, or d) We answer question after reading previous answers, probably trying to synthesize the best of them.
- Be the fastest gun in town and try to announce to others my intention of producing an answer with a one-liner, leaving up to them to judge if my sketched answer sounds good enough or not. Challenges: Other contributors percieving that that's all the answer I'm going to produce and elaborating on it (precisely what I'm trying to avoid).
- (Current) Like 3, but being explicit in my intentions of ellaboration. Challenges: Wishfully none.
None of them has worked very well so far.
My proposition is to have the site informing the user that other people is already working on answers as soon as they enter the first character, and not as late as when they press "submit" (way too late, for everybody). Indication of the number of other people, names, reputation, maximum reputation (within the group), response length, etc. could also be very useful to me in deciding if to continue answering or not. Using the language of economics: make it a perfect market, where everything is a personal choice, but as well informed as possible.
Also useful: the ability to queue questions that I have chosen not to answer for later review.
Another alternative is to lock the question for a while. And/or after certain number of answers have started to be produced. However, somehow I'm more inclined for what you could call the liberal model, which also seems consistent with both the philosophy of the site and many opinions expressed here.
If this information is already available, please let me know how to access it.
-
-
1No, please don't "solve" this problem by doing 3. That's precisely the problem.– Aaron Bertrand StaffCommented Aug 13, 2013 at 1:22
-
My proposition is to have the site informing the user that other people is already working on answers as soon as they enter the first character. While interesting, new users may find this pretty demotivating. If someone were to try to answer for the first time, seeing 3 others with 10k, 40k or 100k rep answering may make the person stop. That would basically be the market working against new competition (going by your analogy).– asheeshrCommented Aug 13, 2013 at 2:37
-
Moreover, although I understand why you are saying this, I dont think the SE model is equivalent to a "market". The points system is a reputation based system, and is not a virtual economy type system. This makes a huge difference to the type of behavior we should encourage and the type of environment that is present.– asheeshrCommented Aug 13, 2013 at 2:38
-
@AsheeshR I think you are overlooking an extremely important part of my proposition, which is to be reminded about the question once interest has subsided, either to review the answers and learn something new, complement the answers with comments, or provide completely new answers and perspectives in case everybody missed them. Commented Aug 13, 2013 at 6:01
-
@AsheeshR I beg to disagree. You can find a pretty complete economic system here. Questions= demand. Answers= offer. Reputation/Votes= currency. Bounties= Stock Exchange (capital for capital) . What question to answer= what to produce (at the micro-economic level). Research on my own or ask a question= what to consume. Etc, etc, etc. I have even seen behaviors proper of very aggressive (aka "dynamic") markets. Disclaimer: I do agree that analogies are always limited and very frequently misleading. But I'd say this is precisely what makes SE more sucessful than other question and answer sites. Commented Aug 14, 2013 at 0:00
-
Stack Overflow because of its huge size appears like a virtual economy. However, if you look at the smaller sites, you will see that the system behaves like an almost perfect reputation points system. I wasn't specifically referring to SO, but to the entire network in my comments above. However, I overlooked the fact that FGITW occurs only on SO, so this discussion is limited to it. In which case, yes. SO does behave more like a virtual economy, than a rep system.– asheeshrCommented Aug 14, 2013 at 0:35
Maybe it has been suggested before, and it may not be popular, but why not set the timestamp of the answer to the last edit, and sort the answers with equal votes accordingly?
-
3then you'd have people game the system by editing it repeatedly to get the top position.– ChiiCommented Sep 22, 2009 at 14:19
To break up your jQuery code into functions, do you do it the same way as in JavaScript?
I would argue that there are still plenty of examples where the first or second to answer are not the answers that end up getting selected or with the most votes...
