Duplicates Mark II, Canonicals, more aggressive duplicate prevention

Question

This question is another take on the duplicates problem. It steals is inspired by ideas from the various posts and feature-requests posted here on meta.

_{Disclaimer: In this post, I am referring to the "Users" or "The People" or "The Gang" or "Everyone". What I actually mean is the bunch of folks who post extremely worn down duplicates.}

What's wrong with today's system?

Are duplicates hard to find? Not really, they aren't. The duplicate finder in the Mark as Duplicate rubric is usually spot on, as well as the duplicate finder under a question's title when asking a question.

This isn't a technical search problem. No sir. Given the fact that people can google 100% of all duplicates closed (because that's how I close almost all of my duplicates), we can understand that the problem exists between keyboard and chair. Users don't search. Users don't read

What else? The system rewards you for answering a duplicate but it does not reward you for marking one. That is a big problem. Because copy/pasting answer from the canonical is a hell-lot easier than marking the question as a duplicate of it. Not to mention you get fake internet points too!

What can be done?

My feature request contains multiple smaller ones. Each can be weighted and implemented individually, but I believe that all of them can be helpful.

Canonical Questions, Take _{who knows how many}

Canonical question selection should be semi-automatic. Meaning, the system will choose canonicals based on various factors:

• Votes the question has (A canonical is usually highly voted).
• Votes the answers have (A canonical's answers, especially the top one, tend to be highly voted).
• Question serves as a source for many duplicates. (If I close 50 questions as duplicates of the same question, it's a good candidate).
• Question gets some sort of "Canonical Votes" or is made such by a moderator. (This is the "semi" part of "semi-automatic". If made by a moderator, it is binding, but user's votes should add towards the "canonical weight" of the question, adding up with the other factors).

The "canonical weight" of the question, can be used in the algorithms that find duplicates before the question is asked and when it is marked as a duplicate.

Above a certain threshold, the question becomes "canonical" (marked with a star, a heart, a unicorn, whatever). (This part is optional, but it may help to create a page that lists those canonicals. "The best info the site has to offer" kind of thing).

Aggressive Duplicate Prevention

Users don't search or read. Unless you shove it in their faces, you won't get anything.

If I try to ask a question, which was identified by the system as a high-enough probable duplicate of a "canonical-enough" question, the page should physically redirect to the duplicate page, with a message asking "Wait! Does this answer your question?" (While still allowing the user to read the question and answers obviously). The buttons should be disabled at first to prevent robo-clicking, and enabled automatically after a few seconds.

If the user answers "No", let their question be posted. If they answer "Yes" just close the message and keep them on the new question's page. Answering "Yes" should also raise the question's "canonical-weight".

Duplicate finding incentivization

One of the problems is that you get rewarded for answering duplicates, but not for finding them.

I propose a +5 rep bonus for successfully marking a question as a duplicate. As well as 3 badges (bronze, silver and gold) for accurately marking increasing amounts of duplicates.

Duplicate answering disincentivization

Votes on answers to duplicates as well as the question itself should be invalidated or at least earn no rep. A more aggressive version of the feature requests suggests a penalty on answerers who answered a question which got closed as a duplicate.

Smarter duplicate resolve

Let A, B and C be questions. If I close B as a duplicate of A, I should not be able to close C as a duplicate of B. In fact, if the algorithm found that C is a suitable duplicate for B, the system should offer A as the duplicate source.

There are questions with literal trees of duplicates, duplicates of duplicates of duplicates, I've seen such a chain with at least 5 levels, and I'm sure there are longer chains.

Wall of Text crits you for 9042!

Conclusion

phew that was long. If you got this far, congratulations! There are a lot of improvements that can be done (and probably I'll edit more as I remember/think about more things.

Would love any suggestions/editions/comments/arguments/etc.

Answer 1

Anything to help with finding and pointing to canonicals would be great. I've yet to understand why after all these years we still don't have a good canonical functionality.

With regards to your +5 rep for closing as a dupe:

• too high
• needs to be reversed if re-opened, questions are getting improperly closed of non-exact duplicates all the time

I like the idea of canonical weight for helping bring up relevant dupes.

Answer 2

Selecting canonical Q&A is not a moderation action; it requires expertise.

Reputation isn't a good measure of expertise either. Reputation in tag (and tag combinations) is better, but really we'd probably need some sort of web-of-trust calculation that rewards people who have been repeatedly voted up by other people with a high expertise metric. And make downvotes really hurt the expertise metric, i.e. instead of 5 downvotes = 1 upvote, for expertise it should be 1 downvote = 50 upvotes.

Answer 3

While reversing upvotes on questions later found to be duplicates provides some disincentive there is also a potential for abuse, or equally poor duplicate finding; resulting in an unfair reputation loss.

As suggested in the Stack Overflow answer to: "How should you treat an answer on question that is a duplicate? [duplicate]", sometimes finding a duplicate is difficult, sometimes further clarification makes the duplicate easier to spot; we should assume good intentions, and not punish a great answer that may specifically address the question.

It like the "Fastest Gun in the West Problem" meets the "The fast and the furious [duplicate]", quantity over quality, and a hands (it) off approach - let someone else do the non-paying work of searching for duplicates; while the lazy (or unsuccessful) profit from answering.

What might work is something like the answer ban mechanism, after a duplicate is answered the user must do at least one search prior to answering, and an additional search for each additional duplicate answered.

After each answer on a question that isn't successfully closed as a duplicate within a week the number of required searches could be reduced by one, and for each successful answer that lasts a month a further reduction for each one. That would avoid the user digging themselves into a hole that they couldn't recover from; a problem with the post bans, and their lengthy timeouts.

This hardly imposes any punishment on legitimate answerers, since they would be searching for duplicates in any event.

Answer 4

I don't see this as a problem. If the dupes are attracting a lot of wasted duplicate answers, or the askers are not getting an answer because they're unaware of the canonical, we have a problem. But the mere fact of asking a dupe is not a problem. Askers don't read; we've developed a system to fix that for them by closing as dupe. Why layer complexity on that?