What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 128 Stack Exchange communities.

I think the Eternal September is dooming Stack Overflow to have the same questions asked over and over again, and the people who don't want questions closed are winning.

Why don't people want questions closed? Because then those questions are a candidate for deletion? Why don't people want questions deleted? Because then they lose reputation with easily asked questions that are popular.

As an example, this question was asked today. It's already been viewed 208 times, and the question asker has 16 upvotes for his question. He has a fair amount of reputation, so it isn't as if he's new.

Yet, that same question has been asked at least half a dozen times:

Another user has suggested getting rid of the close button entirely. I don't agree with that, but evidence suggests that we're not doing enough to stem the tide of highly duplicated questions (duplicated 4 or more times).

Problems

Problem 1: Users can game the system by asking highly duplicated questions and receiving reputation for them.

Net effects:

  • User keeps reputation
  • User is rewarded for asking a duplicate
  • This user (and other users) have further incentive to do it again.

Problem 2: Questions get duplicated; content is splintered throughout the system.

Net effects:

  • Broken windows. No upkeep makes this like every other venue for Q & A out there.
  • "Expert Users" could start to leave as soon as another system comes out that gets this 'right'. It's been a problem since the Usenet days, but it's one that should be fixed.
  • Stack Overflow isn't living up to its core mission of having an authoritative source for each question.

Problem 3: Users that care about the system and want to keep it as an authoritative source get discouraged by the deluge of duplicate and the seeming lack of moderators stepping in.

Net effect:

  • Stack Overflow isn't maintained as well by the community that cares about it as something more than a better version of Reddit or Hacker News.

Solution:

  • Stop rewarding reputation for questions that are closed as duplicates. Revoke any reputation granted for those questions (This is a draconian solution, I know).
  • Implement the features I've suggested here. (Note that those features have been suggested in other places by other people as well.

Examples of questions with 4 or more source duplicates:

share|improve this question
1  
The feature proposal here is identical to what Popular Demanded here. My take is exactly my answer there, except less emphasis on "the problem is not that prevalent". –  Grace Note Jun 4 '10 at 15:37
2  
Yes, I saw that; And meant to link it in the question. I think it is a possible solution, but the difference between our questions is that I don't think he enumerated all the issues asking a duplicate can create. –  George Stocker Jun 4 '10 at 15:40
1  
You bring up more issues, but I'm not entirely feeling that the proposal really addresses 2 and 3, which I think are the major issues. It blocks answers to duplicate questions a lot better than it blocks the duplicate questions themselves. The impact in solving the overarching problem, which is the continual stream of duplicates, seems small to me. In my opinion, it will actually cause harm to the site by making people less inclined to answer genuine questions that simply "look" like they could be duplicates. –  Grace Note Jun 4 '10 at 16:04
11  
Related: Give an incentive for finding duplicate questions –  Ether Jun 4 '10 at 16:26
1  
And the hits just keep on coming: stackoverflow.com/questions/3309089/…, stackoverflow.com/questions/3288459/… –  gnostradamus Jul 22 '10 at 17:35
add comment

13 Answers 13

up vote 31 down vote accepted

The answer to a lot of your points, as has been brought up before, is better merging.

The sooner the posts are merged, the better, and we can just leave a locked stub to help the search engine out later.

share|improve this answer
1  
is it ok, if I see something that is closed that had decent answers on it, to flag it and say "please merge answers into the dupe" ? Because the merging doesn't seem to happen much of the time. –  Kate Gregory Mar 27 '11 at 18:49
    
@Kate, yes, you could flag and ask for merge. Merging does take a lot more time, so it's not done as much. –  Lance Roberts Mar 27 '11 at 18:50
add comment

+1 to removing duplicates and revising SO for quality

The problem with duplicates is, they get in the way of new questions that haven't been answered yet.

I'm frustrated with SO because I spend 5 hours searching through duplicates looking for that one edge-case answer that will resolve all my problems. Then, when I've exhausted all possibilities, I create a question and the only answer I get is something along the lines of "that's interesting, why'd you wanna do that." And worse, I don't have the rep to down-vote the answer as subjective. ::sigh::

What happened to the dream of 'a site to that contains canonical answers about tech questions.' I really liked that dream.

I've copied my answer from The bike shed problem and SO as an example of duplicate questions that look suspiciously like low-hanging-fruit that I kept seeing over and over while I was searching for an answer.

Here's a frob this widget question from me that is both, not a paint shed question, and something that I really need help to answer.

The problem is, there are 400 questions something along the lines of.

Importing best practices

Still best practices - importing in the middle of a file

How are 'import module' and 'from module import *'

Or, why is 'from module import *' bad

Pertaining to cyclic imports

These are the 5 base cases, now mix and match every permutation of these questions until you get 400 unique questions and you now have 400 bikesheds sitting between the highly experienced/talented SO users and my hard/impossible question to answer.

Whatever happened to the concept of SO containing canonical answers to programming related questions. In this case SO needs 'A canonical guide to using python imports' thread so they don't block out the really difficult edge cases like mine.

The only thing I've seen that's canonical on SO is threads about programming:

  • cartoons
  • jokes
  • quotes
  • wtfs

If the community wiki mode was supposed to kill the rep-whoring of useless questions, why are those questions still dominating SO?

Edit: So 400 is an obvious exaggeration. So, to demonstrate I added links. Welcome to bikeshed-land. If you don't see a common theme I'll give you a hint. It rhymes with rubjective. We're talking about low hanging fruit here and a lot, if not most of those question have 5+ upvotes.

Update: It appears as though this issue has been addressed. Duplicates aren't bad, they're just different ways of asking the same question and should be marked as duplicate and linked to where the answer is addressed. In other words, this answer doesn't really apply anymore.

share|improve this answer
    
there is no "canonical" question -- this isn't wikipedia. We're a hybrid engine (see stackoverflow.com/about diagram) so there are degrees of influence here. –  Jeff Atwood Jun 11 '10 at 7:30
5  
My mistake, the whole 'canonical' concept kinda stuck with me since reading "After all, for the next 20 years, this question will be the canonical place on the web where programmers will come to find out about enlarging fizzbars without overwriting snibbits." @ joelonsoftware.com/items/2008/09/15.html. The fizzbars and snibbits must've gone to my head. –  Evan Plaice Jun 11 '10 at 7:41
5  
@Evan @Jeff Atwood Yea; I had the same silly impression after listening to Joel on the Podcast. –  George Stocker Jun 11 '10 at 10:52
3  
@george Joel has some strange ideas about how things work here, but he's also not here very often is he? Perhaps those two things are related.. –  Jeff Atwood Jun 22 '10 at 23:49
    
@Jeff I was researching to answer a question on meta and looked up a presentation by Joel I had recently watched about SO that was given @ google talks a while back. He also referenced the whole 'canonical' concept in that too. Here's the link youtube.com/watch?v=NWHfY_lvKIQ –  Evan Plaice Jun 23 '10 at 0:21
    
@Jeff I hate to bring it up but. Joel has done it again blog.stackoverflow.com/2011/01/…. "Help us build a great library of canonical answers." and this instance is dated 01/05/2011. Seriously, this is beginning to look like a mother/father debate. :) –  Evan Plaice Feb 8 '11 at 11:35
    
@evan read the words I wrote here, please. Thanks! blog.stackoverflow.com/2010/11/… –  Jeff Atwood Feb 8 '11 at 15:52
    
@Jeff Thanks for clearing that up. The answer has been updated to reflect your position. –  Evan Plaice Feb 8 '11 at 23:58
    
@evan it depends is the answer; if there are "too many" duplicates I totally agree that is bad. But about 4-5 is not so bad, and even desirable, so long as they are sufficiently different in text. –  Jeff Atwood Feb 9 '11 at 4:57
add comment

Ironically, isn't this question itself a duplicate, thus illustrating why these kinds of unintentional duplicates aren't evil ... or even wrong?

What is with people who answer questions that are known to be dupes?

I agree they should be closed, but I'm having a hard time understanding why people are so upset that others get a trickle of rep for duplicates asked and answered in good faith.

The only time I get irritated with duplicates is when the asker or answerer knows about the duplicate and proceeds anyway, and is thus asking or answering to game the system but ... clearly you didn't know about the duplicate when you asked this question, right?

I just think it's amusing that everything you're complaining about w/r/t duplicates applies to the very post you created. Maybe the next time you check for broken windows, start with your own home?

share|improve this answer
9  
While researching this post, I saw that post. I even mentioned in the comments above that I knew about that post. His post is asking about 'why do people do this?' my post is enumerating the problems with people doing 'x'. There's a difference there, and not an exact duplicate by any stretch of the imagination. Although, I guess that doesn't make as good of a sound byte. It's not just the question askers that get reputation, the answerers do too. In this case, one answerer nearly hit the rep cap for answering a mega-duplicate! –  George Stocker Jun 4 '10 at 17:42
3  
@george I hear you, but I worry that a lot of these requests are of the "I don't like it that others are getting reputation in a way I consider too easy and unfair!" rather than legitimate duplication concerns. That said, I do support closing them as duplicates where appropriate. But I think I am more comfortable with a moderate level of duplication (as long as it's in good faith!) than some are. –  Jeff Atwood Jun 4 '10 at 17:51
3  
@george basically, the way I think of this is as follows: if it's a good answer or question, provided in good faith, it deserves reputation. even if it is a duplicate. –  Jeff Atwood Jun 4 '10 at 17:53
7  
@Jeff Atwood My chief concern is the effect this will have later on down the road. Joel even mentioned this in a podcast, where people would leave a forum (or usenet) because people ask the same questions over and over, and people get tired of answering the same questions over and over. In this case, you reward answering and asking questions, even if they've been asked and answered 6 times. It's going to have a negative effect. It already has. Reputation for it bothers me because it's an incentive for them to keep doing it. It also just adds another dimension to the problem. –  George Stocker Jun 4 '10 at 17:54
    
I downvoted your answer because it said something that was incorrect. I like your comments though. Though I think your stance is going to cause problems, I like how you said what you said. –  George Stocker Jun 4 '10 at 17:59
1  
@george you want a reputation denial tool, which will drive away new users just as effectively. Anyway, these questions already get closed (see your own example!), just not "closed enough" to your satisfaction? You want "nuke it from orbit" closings? –  Jeff Atwood Jun 4 '10 at 18:00
1  
@George You cite Eternal September in the start of your question. Which makes me assume that the larger concern is about duplicates coming primarily from newer users. The majority of these probably don't care about the reputation, if they even are aware of its presence. I question how much changing the reputation system is going to provide any significant deterrent to this. –  Grace Note Jun 4 '10 at 18:01
11  
@george also, the fact that you're so vehement that this isn't a duplicate, when I believe it is a duplicate is also illustrative of the core problem here. –  Jeff Atwood Jun 4 '10 at 18:01
1  
It brings up a fundamental question: What does reputation mean? If we allow things to stand as they are, then it's not an indication of how someone adds value to the system. –  George Stocker Jun 4 '10 at 18:03
7  
@george well, I feel this question adds value even though I consider it a duplicate. Thus I am OK with you, me, and others getting rep from it. Q.E.D> –  Jeff Atwood Jun 4 '10 at 18:23
1  
Game. Set. Match. Atwood. –  George Stocker Jun 4 '10 at 18:27
2  
@george I want to be clear that I agree without any hesitation whatseover that we absolutely should be closing stuff. I support you 110% on closing dupes! If dupes aren't getting closed as dupes, there is a definite problem. Where I disagree slightly in focus is how "harmful" these duplicates are. –  Jeff Atwood Jun 4 '10 at 22:21
    
FWIW I worded my question as "why do people do x" because it seemed to fit in better with common Meta mannerisms. –  Pops Jun 8 '10 at 14:31
2  
@Randolpho The act of bestowing reputation is. One vote for a duplicate is one less vote for a question that isn't a duplicate. Compound that by the frequency at which questions fall off the front page and you have a problem where people that deserve that reputation don't get it, and those that don't, do. –  George Stocker Jun 23 '10 at 14:51
1  
@George Stocker: but that ignores the underlying, outright wrong statement you're making that people who make duplicates and answer them do not deserve rep earned from the dupe. It's a particularly arrogant statement, and I take issue with it. I know we geeks tend toward such arrogance, but that's not a good thing. Chillax; your world won't come crumbling down just because some n00b didn't bother to check for dupes and earned a few points of rep. And with that, I'm going to take my own advice. Jeff; sorry to bother your inbox with this argument. Signing off. –  Randolpho Jun 23 '10 at 17:48
show 7 more comments

This relates to another point I've brought up in the past regarding how valuable duplicates are and if we should bother to keep them around. The stated purpose of allowing duplicates to linger is that it helps cover a wider range of search terms. But this purpose is somewhat moot if A) people aren't bothering to search first and B) they find the duplicates but are compelled to reask the question anyway because it's easy Rep and a greater part of the community seems to be (wrongly) OK with it.

In short, there doesn't seem to be anything in place that really discourages asking of duplicate questions, meaning that duplicates will likely keep uselessly piling up. As far as a solution, maybe having unlimited close votes for just the "duplicate" reason (while still limiting reopen votes to the daily cap) will help to keep duplicates closed or close them faster. I for one run out of close votes rather quickly, so I often have to just leave comments on duplicates I find.

share|improve this answer
1  
I wanna supervote. Just one will do, but I want to cast it here. –  dmckee Jun 5 '10 at 1:37
    
leads to endless "inclusionist vs. deletionist" wars. Everything's a "duplicate" depending on how broad your concept of that word is.. codinghorror.com/blog/2006/04/… –  Jeff Atwood Jun 22 '10 at 23:50
add comment

I think it would be interesting to do a bit of analysis to see if your interpretation of people's motives is true: i.e.,

  • is it low or high rep users asking duplicates?
  • are some guilty of repeatedly asking duplicate questions?

My own feeling is that duplicates have a lot LESS to do with gaming the system, and a lot MORE to do with friction. It's just easier to ask a new question than to find an existing one. Some of that friction is unavoidable (we're all lazy, and most of us don't know how to read)..

The obvious pro-active solution is to make search better. The SO search box is not great, when it's not totally useless (try the difference between looking for "F# books" and "f# books"). You can opt to search via Google, but again, that's just more friction.

@Jeff, it could be interesting to try and analyse the log files to see how many users actually search and/or click on proposed duplicates before asking questions.

The "post-active" solution is to make clearing up duplicates easier, which is where the question Ether linked to was going. I have also previously suggested added incentives for pruning one's own duplicates.

More carrot, less stick.

share|improve this answer
add comment

Minor Irk. Some of those examples aren't duplicates. Why is using a View bad with SELECT * is talking about views. Select * Except is talking about selecting everything except a column so shouldn't be closed as an exact duplicate (even if they have similar answers).


Users can game the system by asking highly duplicated questions and receiving reputation for them.

I haven't found (and you haven't provided) any example of this. I don't think it is intentional. The issue is users aren't closing soon enough. From the views on some of the examples it should have been closed far sooner.

Making closing as a duplicate easier would help with the issue a bit. It is a pain to close as duplicate at the moment. (Copy link paste link, wait for box to come up, select question etc)

I would suggest that we make possible duplicates more prominent. Maybe put the list of possible duplicates above the question and make it far easier for users to agree with the duplicate. For example just a (undo able) button to agree with the suggested duplicate (Next to the list above the question).

Another idea to complement the above might be to have a "duplicate" button next to all the "Related" posts.


Why don't want people want questions closed? Because then those questions are a candidate for deletion? Why don't people want questions deleted? Because then they lose Reputation with easily asked questions that are popular.

This has never occured to me, but to be honest a lot of questions are getting deleted that shouldn't be. I don't know the exact statistics but I imagine a lot of the hits this site gets comes from search engines. Deleting duplicated questions that aren't exact duplicates removes alternative ways of asking something from the search engine index.


Stop rewarding reputation for questions that are closed as duplicates. Revoke any reputation granted for those questions (This is a draconian solution, I know).

This doesn't help the fact that the questions aren't getting closed soon enough.

If it got closed within 3 or 4 minutes, it wouldn't get very many upvotes as it would fall off the main page fairly fast. Problem solved.

share|improve this answer
5  
that's the other issue. Many "duplicates" are in fact slightly different questions. Now, whether they're different enough to justify existing as a standalone question is debatable, but the idea that there are all these perfect, exact duplicates out there .. well, it's a bit of a myth. It does happen, but like the tawny, foam-flecked hide of the unicorn, it's rare. –  Jeff Atwood Jun 4 '10 at 17:41
    
The 'close as duplicate' usability has been improved greatly, so it' quicker now. thanks, @Jeff. –  Lance Roberts Jun 4 '10 at 18:55
add comment

Maybe this proposal should be its own question, but what about warning question askers if their question is very similar to a question that gets a lot of duplicates? It might sound impractical, but there's already metrics for question similarity, and it's easy enough to calculate how many duplicates an existing question has.

share|improve this answer
add comment

How would you want this to work as a new user, who is looking at this site for the first time.

We should look at this more like the internet. Many destinations, many start points and many ways to get there.

My use case would go some thing like this

Ask a Question.
Q: How do I ask a Question on Stack-exchange.

What should happen at this point.

The first thing a user should see is the input box to ask a question. As they type a list of answered questions is generated as it currently does. They can select which answers, answered their questions. If none of the question or answers that came up solved there problem then posting the question to the community should be the next step. I understand that you wanted to make it easy to ask questions, and you have done that excellently, may be the problem now is, making it easy to find answers.

I think we have confused the way thing currently work with the way we want them too work. Do we need to have a separate function or option for searching and asking a question. I would just combine this into one option. Ask a (question / search). Trying to keep things simple would go a long way to limiting the number of duplicate questions that keep getting asked.

Next keep track of the question as part of the users session. Now if they find an answer or question that works for them they should click the tick or up vote just like community members. Except the reputation should be handled differently for strangers as apposed to community members. Now the session which saved the question can be used to link their question to the questions and answers they find. You can start to build a meta index for questions and answers. Which should be used to improve search results.

Many people would ask the same question in many different ways. I don't think closing some thing as a duplicate, which is a high reputation problem needing 3000 rep to vote to close. I think it is a linking problem. Give people an easy intuitive way of linking similar questions to similar answers. If there are more then one way of asking a question, why limit it, keep it and use it.

It may seam like people are being stupid, asking the same questions in different ways. I don't feel it is their stupidity I think it is the interface has hidden the answers they are looking for. Maybe all they are looking for is clarity, but with out enough reputation, you cant comment on some one else's question, or answers. How can the find clarity other then asking the same question with may be different word order to emphasize the issue they are having.

Lastly it should be a lot easier to post ancillary question or comments when people need clarification. May be an "I don't understand" button which lets them ask a question on a question. This should to push the question back on to the hot question list. To expose it the community as needing clarification.

share|improve this answer
add comment

Just a note:

I've just started a new question with "Obsfuscating C# code" as the title and none of the potential duplicates showed up on the "Related Questions" list. The algorithm seems to have picked up on the word "code" rather than "obsfuscating" - which is understandable given that it's misspelt. Typing "Obfuscating C# code" does produce a list of questions about obfuscation.

So in this case we can perhaps let the OP off asking a duplicate.

However, it does raise a couple of points about the related questions search.

  1. Should there be a spell check on the title regardless of the users settings? If the title is correct then the search can work. This could be done behind the scenes in the actual related search code.

  2. Should the related questions search be run once the body of the text has been entered and use the body text in the search as well? Would this increase the likelihood of a duplicate being found and spotted by the questioner?

share|improve this answer
1  
I think: yes to both. –  Jacques Carette Jun 27 '10 at 19:25
add comment

I was thinking duplicate questions could be forwarded instead removed or closed.

Forwarding questions would leave the different wordings of similar or same questions indexed & searchable while keeping the answers all maintained under the original question.

That way if someone asked "[php] How do I open a connection to a MySQL server?", "[php] How do I use mysql_connect?" is already answered, I could "answer" the former with a "forward" to the latter. If my forward became the accepted answer, or got the most votes after some period of time, subsequent hits to the forwarded question's URL would bring searchers to the one that was already answered.

Sorta like on wikipedia, if I go to http://en.wikipedia.org/wiki/Tip_ring_sleeve , wikipedia will forward me to http://en.wikipedia.org/wiki/TRS_connector

Why scrap Tip_ring_sleeve when other wikipedia users might not know it's often abbreviated to TRS? Why stiff stackoverflow users and subsequent searchers/users who don't know the right question to be asking?

share|improve this answer
add comment

Provided that the duplicates are well linked to each other, duplicates are, in my opinion, a Good Thing(tm). They increase the overall search space for users who are trying to find an answer to their question, because they increase the total volume of word combinations that a questioner might search against, increasing the likeliness of a hit.

Closing duplicates fine. Deleting duplicates is a Bad Thing(tm).

Worrying about them incessantly is... well, it's probably bad for your ulcer.

share|improve this answer
1  
There are good reasons to worry about them. How many forums have you been a part of where people ask the same question dozens of times? Sooner or later, people leave. We don't want that to happen here. –  George Stocker Jun 4 '10 at 17:51
    
@George Stocker: Every forum I've ever been a member of has had duplicates dropped on it. Of the larger more famous ones... Slashdot, Reddit, Dig... dupes are a constant thing. Some people whine about it, but most just get on with things and of the people who've left those forums, duplicates were never (that I'm aware of) on the list of reasons why they left. –  Randolpho Jun 4 '10 at 18:01
1  
This is an appealing point of view but it completely misses the issue of incentives. I don't care a whit that other people might get some reputation for asking or answering duplicates. But I do care that the system as we no it actively encourages asking duplicates and discourages the use of the search box. This is why better merging is a much better solution. –  dmckee Jun 5 '10 at 1:35
    
@dmckee: I don't agree that merging is a better solution; merging gets answers that don't answer the original question. And, like it or not, there are always differences between "exact duplicate" questions. Merging makes things worse, not better. The best solution is to close a duplicate and link to the question(s) it duplicates. –  Randolpho Jun 5 '10 at 19:41
add comment

I find this thread rather amusing since my own feeble attempt at getting the SO crowd to start thinking about solving its own problem of duplication got moved to meta. I tried rather hard to make the question a programming one rather than a value judgement, but was very quickly judged to be NARQ.

Perhaps if there were much better tools for finding duplicates, and then tools for merging or cross-linking (or ...), the total level of duplication would go down?

I am absolutely sure that the reputation system can be used to reward those who find duplicates and also provide a (serious!) disincentive to those who ask questions that have already been asked before. Along the same vein, I believe that there should be a higher cost to voting to close a question - something like 20 reputation points. It certainly should not be free, it should be more expensive than a downvote.

share|improve this answer
    
A higher cost, or indeed any cost, to close will mean that duplicate stay open longer (or even forever), splitting answers from what should be the "one true question". –  ChrisF Jun 22 '10 at 20:31
    
@ChrisF: you misunderstand me. I do not view these reputation-based incentives (closing costs, finding duplicates) as independent but as part of a rebalancing strategy which can be done by the community. A merge proposal (if accepted) should be worth some reputation points. A pure NARQ on the other hand, should cost. –  Jacques Carette Jun 22 '10 at 21:52
    
Thanks for the clarification. Still not sure votes to close should cost though. –  ChrisF Jun 22 '10 at 22:05
add comment

Personally, I hate that we close duplicate questions because they are helpful.

First, the OP often doesn't understand why his question is a duplicate even if it is pointed out to him especially when he already read that question and it didn't help him. Closing the question only drives the people asking questions away. Sometimes, someone else rephrasing the answer in a new question changes it just enough that the lightbulb comes on. We are often missing those lightbulb moments where a different explantion might suddenly make the person understand.

Next, the one of the best ways for junior professionals to get the knowledge they need to understand their own technologies better is to answer questions. If we close all the easy ones too fast, these people don't get the chance to learn by looking for the answers for other people. This is a great deal of how I learned a lot of what I know about SQL server since I was, at the time I was learning, in a job with no peers in my specialty. We are taking that experience away from the people who want to get better, but who are truly juniors right now.

Next, the answers change over time as the technology changes. By closing duplicates too fast, we may be encouraging people to use outdated methods.

Further as new people come onto the site, they may have a better answer that the orginal question had, but if the new question is closed too soon, they are less likely to provide that answer. Most people, I think, are less likely to provide a new answer to a question answered a while back than they might be to a new question.

By closing questions too fast, I think we are becoming a site more about the elitism of the advanced user and less about actually helping people who need help.

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .