Revamping the Duplicate question System

Question

Now that we have over 3 million questions on Stack Overflow, the "Duplicate Question" problem is becoming way more difficult to handle and address. We are seeing over 130 possibly duplicate questions on SO daily, handling them is always very tricky.

We have been kicking around some radical ideas to help improve this process and make it friendlier.

My current thinking is switching the focus when we deal with duplication. Instead of presenting the OP with a "duplicate question" and shutting them down, I think we should involve the OP and community at-large in the process by focusing on the "duplicate answer"

---

The way I see this working is allowing any user with more than X rep to select an exact duplicate answer from another question and "teleporting" it into the question she thinks is a dupe.

Once this "shadow community wiki answer" shows up it can be subject to voting and being accepted.

If it is accepted by the OP clearly the question is a duplicate, so the system can close it.

If it is downvoted by the community cause it does not answer the question, no loss, we can automatically remove it once it gets -1 or -2.

For these shadow answers, editing should be locked (and only accessible on the original question, so we do not create fragments)

---

I think that introducing such a system would be far more effective than what we have today, would heavily reduce mod workload, and would lead to better canonical answers. It would also eliminate the annoyingly long comment threads on possible duplicate questions.

We can also use this to give people "badges" for finding lots of great canonical answers and so on.

---

Thoughts?

Answer 1

Somewhat simpler variation on this:

1. First close-as-duplicate vote automatically creates an answer on the voter's behalf:

   Possible duplicate of: Revamping the Duplicate question System

   Please read the answers there to find the solution to your question.

   And marks it as Community Wiki.

2. Second close-as-duplicate voter selecting the same original adds automatic up-vote to that answer. If a different original is selected, a second answer is added instead.

3. Five close votes close the question as usual.

4. An answer that hits -3 or gets deleted removes the associated close vote(s) if the question is still open.

5. Accepting the answer triggers immediate closure.

Advantages

• Doesn't appear to duplicate information.

• Allows closers to provide a custom introduction to canonical questions if desired.

• Forces duplicate links to "compete" with fresh answers - explaining why a question is a duplicate becomes more important than just dragging up a link that looks sorta similar.

• Breaks out discussion of individual originals into separate comment threads.

Disadvantages

• Links less attractive than fresh answers.

• May inadvertently encourage link-only answers.

• Complicates the close-voting for duplicates.

• May move all discussion of dup-ness into answer comments instead of keeping it with the question.

Answer 2

If this was made an answer-but-not-an-answer I can sort of see this working; the problem with posting the dupe as a comment is:

1. It's sort of easy to be skipped over.
2. Only people of whatever privilege can react and vote-to-close on it.

... but the problem with posting it as an answer makes it:

1. The original poster of the answer can miss out on up-votes (as up-votes are on the CW rather than their own post).
2. Is there anything to distinguish a random answer from the duplicate answer?
3. A user finding and posting the dupe answer may have taken more time to find it than the FGITW, who could have posted a placeholder answer and attracted a few upvotes already; the dupe-answer is then lost in the sea of answers.

Could the dupe-answer be posted in the format of an answer, but in its own category, above the answers?

---

However, I think a much less radical idea would be to simply make the list of possible duplicates more visible to the OP when typing the question.

Currently the first list of duplicates is presented after they've entered a title; which is possibly almost entirely useless; the system has barely anything to go on; it doesn't even know which language the question is about (which is entered in tags at the end). Even on Meta, I've asked questions which have shown no duplicates in this list, only to be closed within seconds with a link to a spitting image of the question I was asking.

If we move the tag input above the body of the question and show suggestion after that, the system has a lot more information to draw a list of suggestions from.

---

The second list of duplicates is presented whilst the user is typing the body of a question, but:

1. Its feedback is very delayed. I'm not sure whether it's gauged by number of chars or time to type a message, but it always takes more than 1 minute to show when I've ever typed a question.

2. It's at the side, which is not where the user is looking.

I'd like to see the user finish typing their question click post, and then be presented with a page saying:

Hey, heres a list of duplicates, do any of them answer your question?

Yes, really that big.

If the user scrolls to the bottom and doesn't see a match, let them post their question.

Whether or not this replaces the list of suggestions along the side or is shown in tandem with the list of questions, I'm impartial. It could even be the same list of suggestions; let the people who didn't even see the list have their first scan, and let the people who've read the list scroll past it.

---

Some alternatives;

1. Make the vote-to-close barrier lower when the question is first posted (e.g. only 3 votes during the first hour, 5 after).

2. Give an incentive for finding duplicate questions or Reward finding duplicate questions - +10, +2, -5: People may be more inclined to find/ vote for a dupe if they get reputation for doing so (and ahem the top 2 feature-request's).

3. Make better use of poor peoples close-flags?

Answer 3

We need to a think a little out of the box here. Sometimes the duplicates have nuances which are threaded in the back and forth across the various answers and comments regarding the motivation of the question. It would be nice if this was able to be somehow synthesized into a definitive "answer".

I have mentioned before about the need for Stack Overflow to support a more didactic long form - i.e. articles or blog posts which examine a problem's facets more fully.

In many cases, the fact that there are duplicates suggests that:

1) the search is not finding the other possible solutions

2) the other solutions don't appear to meet the needs

3) it isn't clearly an organized definitive exploration of the entire problem (c.f. Jon Skeet's analysis of singleton or Erland Sommarskog's treatment of dynamic SQL)

For whatever reason, I find that Community Wiki isn't really rich enough for a real article - and long Community Wiki things are difficult to read here for some reason - not sure if it's the layout or just the way it fits into SO's existing Q&A format.

Answer 4

How about adding a second canonical form of answers which are used as duplicates, instead of copying the original?

One problem with shadowing an answer between questions is that the answer to one question may look confusing or incorrect when posted with a different question, even if the two questions are duplicates. For example, if the answer references code in the first question, those references wouldn't make sense in the context of the second. Another possibility is that original answer was a quick, oversimplified description of how to solve the problem, while the OP of the second question is a beginner and needs a more detailed explanation.

To make a single answer suitable for both questions (and possibly others in the future), it may have to be edited heavily. During this process, it is possible that some extraneous detail specific to the first question is removed, making the answer less helpful for that question. It is also possible that the original answerer may not like the edits, or not understand why they were made, and revert them, making the answer unsuitable again. Also, anyone who wants to edit that answer in the future would have to check every question it is shadowed onto to make sure it still makes sense in that context.

---

A possible solution to some of these problems is to create a canonical form of the answer which is connected to it, but not the same. When choosing an answer as a duplicate, the user would be presented with an editor pre-filled with the original answer for them to modify. The user creates a canonical answer which would work be suitable for both questions and submits it. The second question shows this new answer as a possible duplicate, with a link to the original added automatically. The first question continues to show the original answer, but adds a link at the bottom which can be used to view the canonical answer and a list of the questions it is used on, but the original answerer has the choice to replace it with the canonical version (hiding the original behind the link). If the same answer is used as a duplicate for a different question in the future, the canonical form of the answer is automatically used.

This would allow the first question to keep its specific answer and no information is lost, but there is a single canonical answer used for all other questions. The canonical answer could be edited from any of the questions it is posted on without modifying the original. The original answerer can keep their answer, so they won't be upset or revert the edits.

---

Obviously, the edits required to create a canonical form of an answer could be significant. Therefore, I would suggest giving reputation to the user if the answer meets the requirements to close the question. If the answer never meets the requirements, no reputation would be awarded. Perhaps 2 reputation per upvote for both the original answerer and the person who found the duplicate. The reputation for finding the duplicate could be awarded only if the finder provides significant edits to the post, to be more like the current system. This could prevent others from using the same answer for a different question since they couldn't get reputation from it, so perhaps the editing window should be provided with the canonical answer for future duplicates so that the second finder still has an opportunity to gain reputation for edits.

Answer 5

The only duplicate closes I agree with are questions asked by the same person on the same day.

Other duplicates never really are exact duplicates. The OP's context might be slightly different. Better solutions may have come up since the last time the question was asked. Problems with the old best answer might have surfaced. As a case in point, this question itself is also a duplicate, see here and here, among many others. But it's still useful.

The worst aspect of a duplicate close is how it alienate new users. A colleague of mine came to SO and asked a good question. It good closed as a duplicate. That might have been technically correct. But it also alienates fresh blood exactly like the RTFM comments on Usenet used to do.

So while I think your solution moves in the right direction, I'd go even further. The OP should be treated with respect and his question should not be closed as a duplicate without his agreement. The OP should be given a chance to respond to the duplicate suggestion. With maybe an exception for same-user same-day questions.