This question is another take on the duplicates problem. It steals is inspired by ideas from the various posts and s posted here on meta.

Disclaimer: In this post, I am referring to the "Users" or "The People" or "The Gang" or "Everyone". What I actually mean is the bunch of folks who post extremely worn down duplicates.

What's wrong with today's system?

Are duplicates hard to find? Not really, they aren't. The duplicate finder in the Mark as Duplicate rubric is usually spot on, as well as the duplicate finder under a question's title when asking a question.

This isn't a technical search problem. No sir. Given the fact that people can google 100% of all duplicates closed (because that's how I close almost all of my duplicates), we can understand that the problem exists between keyboard and chair. Users don't search. Users don't read

What else? The system rewards you for answering a duplicate but it does not reward you for marking one. That is a big problem. Because copy/pasting answer from the canonical is a hell-lot easier than marking the question as a duplicate of it. Not to mention you get fake internet points too!

What can be done?

My feature request contains multiple smaller ones. Each can be weighted and implemented individually, but I believe that all of them can be helpful.

Canonical Questions, Take who knows how many

Canonical question selection should be semi-automatic. Meaning, the system will choose canonicals based on various factors:

  • Votes the question has (A canonical is usually highly voted).
  • Votes the answers have (A canonical's answers, especially the top one, tend to be highly voted).
  • Question serves as a source for many duplicates. (If I close 50 questions as duplicates of the same question, it's a good candidate).
  • Question gets some sort of "Canonical Votes" or is made such by a moderator. (This is the "semi" part of "semi-automatic". If made by a moderator, it is binding, but user's votes should add towards the "canonical weight" of the question, adding up with the other factors).

The "canonical weight" of the question, can be used in the algorithms that find duplicates before the question is asked and when it is marked as a duplicate.

Above a certain threshold, the question becomes "canonical" (marked with a star, a heart, a unicorn, whatever). (This part is optional, but it may help to create a page that lists those canonicals. "The best info the site has to offer" kind of thing).

Aggressive Duplicate Prevention

Users don't search or read. Unless you shove it in their faces, you won't get anything.

If I try to ask a question, which was identified by the system as a high-enough probable duplicate of a "canonical-enough" question, the page should physically redirect to the duplicate page, with a message asking "Wait! Does this answer your question?" (While still allowing the user to read the question and answers obviously). The buttons should be disabled at first to prevent robo-clicking, and enabled automatically after a few seconds.

If the user answers "No", let his question be posted. If he answers "Yes" just close the message and keep him in the new question's page. Answering "Yes" should also raise the question's "canonical-weight".

Duplicate finding incentivization

One of the problems is that you get rewarded for answering duplicates, but not for finding them.

I propose a +5 rep bonus for successfully marking a question as a duplicate. As well as 3 badges (bronze, silver and gold) for accurately marking increasing amounts of duplicates.

Duplicate answering disincentivization

Votes on answers to duplicates as well as the question itself should be invalidated or at least earn no rep. A more aggressive version of the feature requests suggests a penalty on answerers who answered a question which got closed as a duplicate.

Smarter duplicate resolve

Let A, B and C be questions. If I close B as a duplicate of A, I should not be able to close C as a duplicate of B. In fact, if the algorithm found that C is a suitable duplicate for B, the system should offer A as the duplicate source.

There are questions with literal trees of duplicates, duplicates of duplicates of duplicates, I've seen such a chain with at least 5 levels, and I'm sure there are longer chains.

Wall of Text crits you for 9042!


phew that was long. If you got this far, congratulations! There are a lot of improvements that can be done (and probably I'll edit more as I remember/think about more things.

Would love any suggestions/editions/comments/arguments/etc.

  • I thought you couldn't create duplicate chains anymore, unless you were a moderator. There are a lot of existing chains since that restriction was added sometime in the last year (iirc), but there shouldn't be new ones. – Troyen Apr 25 '14 at 23:05
  • 1
    On the topic of "canonical" posts, though not duplicate prevention: over at ELL we've actually started intentionally crafting Canonical Posts. We're small so it's not a huge endeavor yet, but it's related to what you're talking about. – WendiKidd Apr 25 '14 at 23:05
  • @Troyen: I didn't know that, do you have a meta post that confirms it? – Madara's Ghost Apr 25 '14 at 23:06
  • 3
    "If the user answers "No", let his question be posted." - The average user who doesn't care will not look at all. They will just answer "No" to get it out of the way and their question posted ASAP. So I don't think that part will work. But I'm not against the idea of finding a way to punish users who say "No" to a duplicate that it eventually gets closed as a dupe of. – Mysticial Apr 25 '14 at 23:07
  • @WendiKidd: We have some canonical posts over at other sites too (this is most oriented towards Stack Overflow, but works anywhere). However the problem is that they are hard to find and not easy to close against. – Madara's Ghost Apr 25 '14 at 23:07
  • @Mysticial: Well then the buttons will be disabled for a few seconds like they do on the review queue, to ensure the user reads the nice big red message. – Madara's Ghost Apr 25 '14 at 23:08
  • Maybe I'm wrong. After searching, the best I found was a dev agreeing questions shouldn't be closed as a dupe of a dupe and detecting that should be automated but it may not have actually been done. Circular loops are blocked at least. – Troyen Apr 25 '14 at 23:23
  • @Mysticial any solution to this issue will always have problems but this might at the very least help. – terdon Apr 25 '14 at 23:54
  • On AU we talked about this for a while. If SE could implement something like that for some of the bigger sites and let the moderators set it up it might help a lot. Otherwise awesome stuff here. – ɥʇǝS Apr 26 '14 at 1:03
  • 4
    I'm a little concerned though, that 5 reputation per duplicate closure is a bit much. Just imagine the rush to close duplicate questions it would cause.. We already get TONS of people trying to pass simple/silly edits for two reputation... Maybe 5 reputation per every 5 or 10 questions closed properly? – ɥʇǝS Apr 26 '14 at 1:05
  • 1
    (wrt @ɥʇǝS) Like robo-reviewing, but without a review queue. Yep, 5 rep per closure is out of the question. – michaelb958--GoFundMonica Apr 26 '14 at 4:57
  • 1
    I have answered a question an mSO with regard to the process on creating canonical questions here: meta.stackoverflow.com/a/252046/578411 – rene Apr 26 '14 at 8:06
  • 1
    @ɥʇǝS: The small details can still be discussed. Note that this is the only solid metric I posted, and it already has arguments against. The amount can still be discussed. – Madara's Ghost Apr 26 '14 at 9:40
  • 1
    Who would get 5 points? I suppose the first person to flag or close-vote, yes? – brasofilo Apr 26 '14 at 15:02
  • 2
    Regarding "Duplicate answering disincentivization", in my opinion, if the question receives a good answer which lateron gets merged with the master question, there shouldn't be any disincentivization for it. – Aditya Apr 26 '14 at 15:53

Anything to help with finding and pointing to canonicals would be great. I've yet to understand why after all these years we still don't have a good canonical functionality.

With regards to your +5 rep for closing as a dupe:

  • too high
  • needs to be reversed if re-opened, questions are getting improperly closed of non-exact duplicates all the time

I like the idea of canonical weight for helping bring up relevant dupes.

  • 3
    Why is +5 too high? You potentially get +30 or more for a crap answer on a duplicate, how is +5 too high? If at all, it still might not provide enough incentive to close as duplicate. I can agree with the points being reversed in case of reopening though. – Madara's Ghost Jun 14 '14 at 15:25
  • @SecondRikudo meta.stackexchange.com/questions/230507/… – ɥʇǝS Jun 14 '14 at 16:02
  • You can buzz through dupes too quickly. You don't want to over-incentivize it, or you'll have too many focusing on that. – Lance Roberts Jun 14 '14 at 16:16

Selecting canonical Q&A is not a moderation action; it requires expertise.

Reputation isn't a good measure of expertise either. Reputation in tag (and tag combinations) is better, but really we'd probably need some sort of web-of-trust calculation that rewards people who have been repeatedly voted up by other people with a high expertise metric. And make downvotes really hurt the expertise metric, i.e. instead of 5 downvotes = 1 upvote, for expertise it should be 1 downvote = 50 upvotes.

  • 1
    No, because people will always disagree with me, or revenge downvote me, or whatever. I get downvoted, even when I don't serve it. As for reputation and tag badges, those are the best meters we've got. We use them for everything else, so.... – Madara's Ghost May 15 '14 at 6:27

I don't see this as a problem. If the dupes are attracting a lot of wasted duplicate answers, or the askers are not getting an answer because they're unaware of the canonical, we have a problem. But the mere fact of asking a dupe is not a problem. Askers don't read; we've developed a system to fix that for them by closing as dupe. Why layer complexity on that?

  • 1
    The idea is not to prevent duplicates altogether. Duplicates are awesome! The idea is to locate and mark duplicates easily and effectively. – Madara's Ghost Jun 15 '14 at 3:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .