Why do incorrect answers keep getting "accepted"?

Question

Seeing incorrect answers get "accepted" makes me seriously doubt the value of Stack Overflow. For example, this answer about Perl's symbolic references was accepted - but any competent Perl programmer knows that it's a horribly poor approach, and the answer that says as much outvoted the accepted one by 15 to 0 (as at 12/11/2008).

Is this a technical site that values accuracy, or just a popularity contest? I was hoping for the former, but examples like the above make it look more like latter in my opinion.

A related question: Maybe a delay of 24 hours or so before any answer could be accepted would be helpful. I get the impression that many of these cases aren't "gaming" the system, they're simply newbies accepting the first answer they get that seems to help. A waiting period would allow them to see what everyone else thinks of the answer - maybe they'd be more cautious about accepting an answer that's voted down to oblivion and loaded with comments that say "no, that's wrong."

Related question: Etiquette for correcting old questions with incorrect answers

Here is a list of problem questions I compiled.

Answer 1

The very act of asking the question means the individual is likely to be the least knowledgeable person in the thread for that particular narrow topic. This same person accepts the answer, even though it's likely they are the least qualified to do so.

But it's okay. It should still (most of the time) indicate that the accepted answer was able to solve the problem at some level for at least one person. The voting system also ensures that any caveats not included with the answer can be readily visible.

Answer 2

Jon Skeet can't possibly answer every question, so the chance of an incorrect answer slipping in could happen.

Answer 3

This is probably something for which there is no good solution.

One of the early questions that I saw on Stack Overflow was phrased as "which-is-better-this-or-that" and the simple, obvious answer "this" was massively upvoted while a different non-answer with no votes was accepted. Yet the technically correct answer was "neither of those, here is a third way"

This illustrates Joel's point and the limitations of any site like Stack Overflow:

• the asker did not understand the problem well enough, and asked the question in a manner that made bad assumptions about the answer.
• the "wisdom of the masses" is not always so wise, as the incorrect answers received lots of votes and the correct answer received only one
• the asker then accepted a non-answer with no votes
• the asker later added a comment saying 'I guess there is no correct answer but thanks for the discussion', which is funny because actually implementing any of the proposed solutions (or reading the background material) would have clearly demonstrated the correct approach (correct in the sense that "it actually works and is robust")

So people are not perfect, thus the sites they collectively create and moderate will not be either.

There is a solution though: get rid of the 'accepted answer' designation (though I kind of like it, it provides closure without closing the question.)

Answer 4

I think the whole "accepted answer" needs to be abolished and replaced with a "thank you" scheme.

Every person asking a question should be able to distribute up to 30 points (in total) to people who assist them. SO should display a thank you section at the top of the question to all the people who helped. Note you should also be able to thank people who commented and did not reply to your question.

Answer 5

I really vote for a 24 hours delay before answers can be accepted. Great idea

Answer 6

Here is my completely subjective view of how the voting system works. Roughly speaking (correct me if you aren't seeing the same behavior - and note that advantage is cumulative - e.g. you can be both fast and pithy, etc.):

1. The fastest answer has the biggest advantage.
2. The first answer that comprises a pithy statement of conventional wisdom, without reservation or offsetting factors, gets the next biggest advantage (which is what you'd expect, given the definition of "conventional wisdom").
3. Third advantage (compensating for late arrival) is pointing out an obvious flaw in a type #1 or type #2 answer, with a correction.
4. An answer that references a canonical source, addressing the question, comes next.
5. Then comes an answer referencing a source no one has heard of (but addressing the question, right or wrong).
6. Next comes an answer referencing any source about some other topic.
7. Followed by a thoughtful, extended answer weighing various factors and suggesting further reflection.
8. Next, a response pointing out that insufficient information is provided, and without context it's dangerous to discuss the topic. (Interesting how easily this gets trumped).
9. Then come funny answers.
10. But finally, all credit accumulated from the above attributes is futile if you are below answer #8.

Another interesting dynamic is that if you edit your response to improve it (possibly incorporating information from other answers), your answer will drop to the bottom of the group with your score (sometimes, sadly, below #8).

Any answer suggesting new, popular technology trumps an answer suggesting that an old way is a better way (think ORM/DAL).

And note that this process is becoming somewhat recursive/incestuous. Googling for an answer is incrreasingly bringing forth SO as a highly referenced source.

Answer 7

Why not have an accept answer by the person who asked the question and an accepted answer from the community?

Either set a threshold based on the number of upvotes. The first answer to that threshold is accepted, with a margin of upvotes. It leads the second answer by x votes.

Or have a separate accepted button that a certain number of people have to click. You could restrict this by reputation. I am sure there are other ways to have the community select an accepted answer.

Answer 8

The best practice isn't always the answer to the question that was asked.

In your example Does Perl have PHP-like dynamic variables?, the question that was marked as accepted, regardless of if it should or should not be used, is the answer to the question he was asking. (I guess it is, I don't know PHP or Perl)

Of course, the fortunate thing about SO is that the dialog is available for others to debate if what is being done is the right way or not. I think someone reading that post will notice that the answer directly beneath has many upvotes and would consider it as well.

Answer 9

This topic got some recent spin again when this happened. Apparently some user accepted and used an answer which superficially "worked", but turned out to be wrong.

This emphasizes again, that the person asking the question might have little to no capacity to identify a "good" or "correct" answer. I "understand" that the intention of the accept option is to show that the answer solved the problem for the user asking, but since SO and others are also used by other users to answer their questions via googling, the solutions should be correct and general enough to not only appease the person asking. I also understand that it is still the responsibility of the programmer to ensure his code works, but as can be seen in this example his or her problems can turn into problems for others quite easily.

Another problem is that an accepted answer gives a lot of reputation, and since early answers are usually the "winning" answers, this incentivizes users to post quick, untested and possibly harmful answers.

The obvious solution is to remove the accept option, and rank answers solely based on votes. This is also flawed, since this will not necessarily flood the correct answers to the top, but it at least removes some incentive from quick answering, and it removes the clear visual signal in the UI that one answer is "better" than the rest.

Answer quality ranking can be improved further by introducing a second vote display. Show the total votes in large as-is, but also show the votes by the top-5%-percentiles of the domain(s) in question in small letters next to it. Those users should be usually able to distinguish good from bad answers more reliably. Of course, answers should be sortable by that field as well.

Answer 10

In any system that has a bunch of humans, someone is going to look to game the system.

We could have central authorities, popularity, validity of edits or any number of metrics. No matter what, a really motivated person can find a way to game a system. In the end, an exception is not usually the rule. Some folks may end up accepting really wrong answers, but good answers do make it to the top in most cases.

Answer 11

I agree.

It would be better if the answer the community feels is right (i.e. highest votes) appears above the accepted answer, with the accepted answer coming next.

Answer 12

On EL&U and on ELL I have witnessed a number of incorrect accepted answers being heavily downvoted by the community coupled with innumerable comments lambasting the author or the OP because the answer was either incomplete or totally wrong.

Sometimes the OP is convinced and changes their mind, but sometimes they accept the first answer posted, to then disappear and never return. Sometimes the OP stubbornly ignores the protests and agrees with a "bad" answer simply because it suits their narrative.

The responsibility of resolving the issue of inexperienced, naïve, prejudiced or mule-headed OPs who accept answers that are evidently wrong, lies with the community and no one else. If the number of active users who remain fully engaged does not significantly dip, then the better answers will always rise to the top. When a significant number of users flag an answer that clearly contradicts common wisdom, moderators can and should intervene; either via through comments, closure, or in exceptionally egregious cases, deletion.

However, when the number of avid and engaged users drops on any one site, so too will the quality. With the obvious consequence that the number of undetected low-quality or errant answers accepted (mistakenly or non) will inevitably rise.

Answer 13

Users with a high enough rep should be able to change which answer gets accepted.

Preferably without changing the author

Answer 14

Given the assumption that a user with high reputation is more likely to be correct than someone with low reputation, the following simple changes may help alleviate the problem:

• Give high-reputation users the power to veto a marked answer and choose another one. OR, differentiate between the answer chosen by the asker, and the answer chosen by the masses.
• Weight the vote of high-reputation users more highly

Answer 15

The time limit for accepting the answer would not change the people: they still use the first hint that seems to help. Hence they can’t accept the answer, and they would do nothing.

So many answered question would stay unaccepted (except when some high-ranking person feels responsible to close open issues).

Answer 16

Maybe we need there to be two answers, one selected by the person who asked the question, and a second selected by people with over a certain threshold of points, and only after the question had been open a few days. Only an experiment allowing this would show whether this improves the correctness of answers selected.

Answer 17

Ignore the 'accepted' answer and pay attention to the answer with the highest votes.

This is a much better indicator of answer quality.

Answer 18

I agree with you. I think the best solution is to add the possibility to flag the accepted answer as "controversial".

Answer 19

In my opinion:

1. Part of the problem is SO seems to expect you to accept a solution. Or at least suggests you accept one. Sometimes this leads to people accepting something that is 'almost' right. The same problem exists at most 'solution sites'.

2. The second problem is that programming is not exact and people have different styles. Something I think is perfect for my style or follows my methodologies may irate another person and vice versa. On many occasions I have seen code that executed perfectly but was basically useless in a team environment as no one could understand it without constant re analysis ... "who is right"?

3. Sometimes an answer is actually a solution to the question asked, i.e. it works. For that person the problem is solved. The answer might not however consider wider issues like security, again a working answer providing a less than perfect solution. The person asking the question is unlikely to be aware of these wider issues.

Now I am a new user on SO, but have been observing for a while.

The thing I like about SO is that accepting a solution doesn't end the discussion. Through comments or new answers you (we) have the opportunity to correct or add to an accepted solution.

None of what I have said is about "popularity", is more about human nature and the ironically imprecise industry we work in.

Answer 20

The problem with this site is the questions will remain, but once it’s past the front page they don't receive more answers or upvotes. So the question will have the old questions, but without new answers which are perhaps better. Or they will receive new better answers without any upvotes.

I’m thinking about questions for code examples with code example answers. Not the more subjective ones ("What framework is best for me?") or author specific ("Where is the bug in my code here?").

And I hope older questions may be tagged with "generalcodeexample" or similar will resurface with new answers, so new (hopefully better) answers can have a chance to gain some votes. Instead of old good-enough answers remain on top.

• "generalcodeexample" should be sorted by votes only IMO.
• new answers long after the question is asked should have a chance against old answers with accumulated upvotes over time. (How I don't know. Points are not as important as getting the best answer recognised and on top)

Answer 21

Voting and reputation are both bad measures.

• How is it possible that someone with ~800 points doesn’t know that they can refer to the documentation of String to find out how to find if a String contains a certain character?

• If most of your users don’t know anything and they all vote for an answer, does voting help? I’m sure there are many examples of bad answers being voted up by people with lots of points.

It’s unfortunate that someone who didn’t know something, then asks a question and gets a bad answer which leads them terribly astray. It’s this sort of advice that leads the industry to be full of very incompetent developers who really should not be trusted.

Should it be possible for people to lose points if they give bad answers?

I personally think it should. Otherwise the points currency have no value. In the real world, people who are successful make lots of money (aka points) and people with dingbat ideas lose money and go broke. Should giving or voting for the wrong answer cost points? I think it should.

Answer 22

You raise a valid point, but having come from Experts-Exchange (many years ago, before it become the corporate monstrosity it is now) Stack Overflow is so much better. Sure, the wrong answer is accepted every now and then, but the question doesn't drop off the radar straight away, which I think is a huge strength of Stack Overflow.

New answers can be added much later and upvotes improve the chances that other readers will find the correct solution. As a relatively new Stack Overflow user I enjoy going through old questions and upvoting what I think are good and correct answers.

I disagree with a 24-hour time limit only because I suspect that the number of times this happens is low in comparison with the number of times the correct answer is rewarded, and we want to encourage questioners to reward the correct answer. Hopefully after trying and testing it of course.

If an incorrect or suboptimal answer is rewarded, the downvote system should be sufficient to indicate that it is not necessarily the best answer. This will work as long as people are prepared to use the downvote system.

Answer 23

Most of this discussion is conducted from the point of view of expert users.

As a new student of computer science, I am having to learn several new domains of knowledge at once, and I consult Stack Overflow, because I don't know what I don't know.

There are two general types of answers that I find helpful. There is the answer that provides a solution with enough information that I can implement it. There is also the answer that provides helpful background information to help me fill in the blanks in my knowledge.

On the other hand, there are a lot questions on Stack Overflow without answers that do either. As a new user, I lack the points to be able to up vote or comment on the answers. I am not alone, so I expect that Stack Overflow's expert users are not getting the feedback they need to improve their answers to help those that need them most.

Answer 24

I think that the user that asked the question has the right to decide the answer... even if it's wrong. Besides, we have the point system to allow the community point out if there's a better answer.

Answer 25

Reputation means nothing in terms of a person’s competence or knowledge. In the end, if you post enough times you’ll get points. Reputation is a measure of activity, not quality in any form.

Answer 26

Most formal questions' answers are not formally proven

Given a question that is well defined enough to have a computable correct answer, as the question has more possible solutions given the methods of computation, there is a higher probability that an incorrect solution would be selected randomly out of the possible choices.

In Stack Exchange, the method of computation often does not follow from a finite set of assumptions and logical implications, so a solution to the question may not be the correct answer. Basically we know from Stack Exchange that we do not use full deductive reasoning to compute answers, otherwise anyone that even came up with a solution would automatically be the correct answer.

1. There are advantages as well as disadvantages to this (as well as reasons).

   Also given completely deductive methods, we have to except that there is a limited amount of resources available for computation, i.e., time. As a question becomes more difficult or simply requires more resources to compute, there simply is not enough time to have elapsed to arrive at a full deductive solution.

   Often we can realize the difficulty of a problem and resort to non deductive methods to try and solve a problem or answer a question in a shorter period of time. The trade-off is that the accepted solution or likely correct solution may indeed be incorrect and there is no way to tell.

   Often, for most problems, completely deductive methods are simply not necessary and a waste of time and resources. For further insight to the solvability of problems I recommend reading up on the deterministic Turing machine or for an equivalent and elegant summary of this, Church's lambda calculus.