Project Reduplication of Deduplication Has Begun!

Question

While announcing the second iteration of the Stack Exchange quality project, I not-so-briefly alluded to a collaboration we're kicking off with the University Of Melbourne.

The project presents some very interesting possibilities for us if their model validates as well as is hoped:

• Knowing very quickly if a question is not a duplicate
• Being able to quickly surface the most appropriate (by criteria we provide) duplicate with a high degree of accuracy.
• Being able to surface high and low quality groups of questions that essentially ask the same thing that aren't currently being surfaced very well

Essentially, we might to be able to alleviate two of the most frequent pain points for new and experienced users alike:

• By showing more relevant probable duplicates while folks ask, we should stand to lessen the frequency of duplicate questions (particularly, those that tend to be of not-so-great quality)
• We might be in a better position to more clearly identify something as not a duplicate, which could ease those mean jerks closed my question as a duplicate and it completely wasn't!

The paper describing the methodology is short and worth reading if you're curious.

So, what do we need from you?

The researchers have already validated data in subject domains where they're knowledgeable enough to make a clear call on something being a duplicate or not. However, we cover a pretty vast group of academic and technical domains, and they simply need knowledgeable people from those communities to help them decide if their method made the right decision.

How are we going to do this?

Easily. They've built a system similar to our review system where you're presented with a pair of questions, and can indicate if they're a duplicate, strongly related, or not a duplicate. You can use your Stack Exchange account to sign in.

Who is eligible?

Those with the ability to cast close votes on:

• Android
• English Language & Usage
• GIS
• Mathematica
• Physics
• Software Engineering
• Stats
• Tex
• Unix & Linux
• Webmasters
• Wordpress

In the rare cases where you have a gold tag badge, but have not yet unlocked close privileges, a gold tag badge will allow you to work on that subset of questions.

What about other sites? What about Stack Overflow?

We'll have to see how things go with this initial set. If everything looks really promising, then they'll take a look at running Stack Overflow through the model. That would be an enormous undertaking since it would entail every single question since the dawn of time, while accounting for duplicate targets being deleted later.

There might also be a need to run a few more smaller sites through - we just have to see how it goes.

Where do we go from here?

Look for a post on your meta site from Doris Hoogeveen letting folks know that we're ready for help if you care to provide it. She'll link to this post for the benefit of folks that missed it.

What if it doesn't pan out?

We're okay with that because we learn quite a bit either way. It's my personal opinion that getting the right duplicates in front of people as they ask is probably the most impactful way that we can elevate the experience of both new and experienced users. If something looks promising in that direction, we feel it's worth exploring.

Answer 1

Thinking about Stack Overflow…

Does the learning system need to consider it as one site, or can for example the PHP tag be done in isolation?

Likewise we can define collection of tags, C#,VB.NET,.NET, ASP.NET for example that have few questions outside of the “tag group” and lots of questions crossing tags within the group. Then just look for duplications within the given group.

Answer 2

I just posted my concerns about this on stats.stackexchage, so I will follow up with a summary here: I am really concerned that this will prevent people from asking questions or getting help on stack exchange. Here are my reasons:

1. Synthesis Questions: I am worried that project will limit sythensis questions, which are re-asked questions for the purpose of getting a final answer. Usually these are asked when the original question: 1) has multiple answer that changed over the years or 2) It has had conflicting answers over years
2. Potential for Over-fitting The unstructured data from stack exchange posts is prone to overfitting and we have to be careful of that.
3. Skewed toward/against the questions of new users: Depending on how you code it the algorithm is going to be skewed toward or against news users in a significant way.

   1. [Reason for the Skew Against] Basically new users are prone toward producing missing values this text mined dataset then older users because reputation system promotes such behavior.
   2. [Reason for the Skew Toward] On the other hand, new users are less familiar with the site are more prone not to be familar with how to navigate stack exchange and thus will ask duplicated questions.

4. StackOverflow will be an outlier: This will be either very good or very bad for stackoverflow because: 1) To ask how code something means there are at least 5 valid duplicates of how to code in other languages. 2) The dependency on tags as classifier, cluster, or filter of question @IanRingrose mentioned.

The duplicate questions tag does not automatically delete questions. I am worried that this safe guard from duplicate questions tag will be removed when one releases Project Reduplication of Deduplication into the full wild of stack exchange. Again, I explain each of these reasons in more detail in the link below.

P.S. Also I just posted an example of reason 1.2 (Synthesis Question) in Cross Validated