What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

This question already has an answer here:

There are quite a few sites in the network where content can sometimes overlap. That is, there are questions that could be appropriate on multiple sites in the network. Combine this with the fact that some askers are very eager to get their questions answered, and you can end up with scenarios like this:

  • OP asks question on ELL (34 min ago)

  • OP asks same exact question on ELU (32 min ago)

  • Someone answers the ELU question (28 min ago)

  • Someone answers the ELL question (23 min ago)

  • A second person answers the ELL question (21 min ago)

  • A moderator on either site (let's say ELL) is tipped off and realizes that an identical question is open and answered on both sites. (19 min ago)

  • In the meantime, a second person answers the ELU question (18 min ago)

  • Said moderator contacts an ELU mod to determine which site the question should be migrated to for merging. Usually the answer is "the first one it was posted on", but sometimes the question is more appropriate for one site than the other. (17 min ago)

  • Moderators agree on destination; duplicate question is migrated to ELL (15 min ago)

  • Duplicate is closed and merged so that all the answers appear in one place (13 min ago)

  • Users who took the time to answer the question on either site realize that answers had already been posted on the other site and that they quite possibly wasted their time answering in the interim. If the question was migrated away from their site, users realize they have lost rep for that post. (7 min ago)

  • It is determined that a whole lot of time and effort has been wasted for everybody (58 seconds ago)

The solution? When a user attempts to post a question, check if they have an identical question open on another site. If they do, do not allow them to post it.

This would save a lot of time and effort for everyone involved. Cross-posting identical questions between sites is not allowed, and having a feature that disables it would save time on the part of both answerers and moderators. I simply used ELL and ELU as an example because I'm familiar with it; I have a feeling this is an issue on other sites as well, or else the answer about disallowing cross-posting would never have been necessary.

Note that I am not asking that this check include closed questions; if a question has been closed as off topic on one site, it's perfectly acceptable to try and post it on a site where it might be on topic. But if a question is open on one site, it shouldn't also be open on another. We want all the answers in one place and to not waste answerers' time repeating things.

This can get especially hairy because unlike most things moderators deal with, it cannot be accomplished by yourself. You must get in contact with a moderator from the other site—andthat doesn't always take just a few minutes. While you're waiting to discuss the final destination of the question (or if you know it belongs on your site, but can't take care of that because you don't have that migration power) more answers will accumulate on the questions, thus wasting more answerers' time.

share|improve this question

marked as duplicate by Gilles, Lance Roberts, Cody Gray, Hugo Dozois, Martijn Pieters Aug 15 '13 at 23:08

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

1  
This is a all too not-uncommon occurrence with questions posted to P.SE and SO. Its quite akward there because it is likely that at least one question is off topic, will get downvoted and closed. Instructing the user on flag to migrate rather than repost could be a step 'twards better user experiences. –  MichaelT Aug 15 '13 at 21:34

2 Answers 2

up vote 8 down vote accepted

It's a fair idea, but it would not be feasible to implement.

What's the first reaction of a user going to be when they get a message like You already have an identical question on xyz.stackexchange.com? Right, they alter it slightly so it passes the filter.

The moment when it's no longer about a simple 1:1 comparison, you're in real hot water programming-wise. You have to define question similarity, and think about where you set the threshold. How similar must the contents be so the question gets blocked again? What if the algorithm recognizes a question as "identical" that isn't really, but uses similar words? What if the user opens a Thesaurus and simply start using different words that the similarity detection cannot reasonably be aware of? And so on and so on.

share|improve this answer
5  
Plus, each site has its own database, so assuming we could implement this at all, there's no way this is going to be even remotely performant since we'd have to potentially run it 100+ times and it's only going to get worse as we get more sites. –  Anna Lear Aug 15 '13 at 21:30
    
@AnnaLear what about storing the... say checksum of the question text in the user's network profile that they've asked today? Though, not knowing the behind the scenes databases, that may still be quite problematic. –  MichaelT Aug 15 '13 at 21:31
    
I'd go with something more like this: `You've already posted this question on xyz.stackexchange.com. Questions should not be cross-posted across the network. If you asked the question on the wrong site by mistake, please delete the original question before posting. If you're looking for a different answer here, try tailoring your question to this site specifically." Yes, users could easily trick the filter. But by not using the word identical to tip them off and by informing them of the rules, I think things would improve. I don't think most people know this is against the rules. –  WendiKidd Aug 15 '13 at 21:32
    
@WendiKidd It's undesirable, but I don't know if I'd go as far as to say it's Against The Rules (TM), especially if the question is on-topic on the sites where it's posted. –  Anna Lear Aug 15 '13 at 21:33
    
@AnnaLear Well, I was thinking that relationships could be set up between sites that are reasonably within each others' scope. So ELL wouldn't check for duplicates on Programmers, and Stack Overflow wouldn't check for duplicates on German Language and Usage. –  WendiKidd Aug 15 '13 at 21:33
    
@Wendi showing them a warning might indeed be a good idea (but the technical concerns raised by Anna might still apply) –  Pëkka Aug 15 '13 at 21:33
    
@WendiKidd That'd be a maintenance nightmare. Someone has to set up those relationships, then make them available across the network. Keep them updated as new sites are created, some sites close, and some scopes change. Being able to show a warning could likely be nice, but unfortunately it just isn't practical. –  Anna Lear Aug 15 '13 at 21:34
    
@AnnaLear Hmm, all right. Well thank you for your attention to the issue, then :) –  WendiKidd Aug 15 '13 at 21:35
4  
@Pekka웃 the goal of this (I would hope) wouldn't be to prevent them from doing it. A determined poster would get around it no matter what. It would instead attempt to help the poster who is unaware of what to do when someone says in an SO comment "May be better on P.SE" and help them realize the proper path for moving a question from one site to another so that they will flag for migration in the future. –  MichaelT Aug 15 '13 at 21:46

How about instead of preventing the user from posting their question again, we merely detect that the posting is duplicate and point them towards a better solution via an appropriate dialog box.

In my experience (of ELL and ELU), the reason for the duplication of the question is typically because some user posts a question on ELU, is told by users of that site that the question is a better fit for ELL, and they then duplicate the post on ELL. This then violates our understanding with ELU that duplicated questions are always closed on the second site they are posted to, so we immediately close the question on ELL, leaving the user confused as to what to do next.

Worse - if answers were posted on both sites, we end up with complicated shenanigans in merging the two posts, and possibly also migrating the ELU one to ELL once all of the dust settles.

This is a particularly unsettling and confusing for users that don't know the rules. They came to ELU to ask a question, got told to post it on ELL, and now all of a sudden their new question got closed on ELL, there's a whole load of angry people running around shuffling their question between sites, and all the while they just want an answer to their question; they never meant to break the rules or be a pain.

So instead of preventing the question being posted (which will add extra bafflement to the user who is only following the request of the users of the first site to move their post, but simply doing it in the wrong way), we simply tell them that what they are doing is the wrong way to solve their problem at the point where they are doing it wrong - and more importantly, tell them the correct way to migrate their question to the other site.

It is easy for the software to detect identical questions being posted on two different sites. When it happens and the users are linked, pop-up a dialog that says something along the lines of:

Oops! It looks like you're posting a question to ELL that you've already asked on ELU! Did you mean to migrate it instead?

And if the question is by a different (i.e. not linked user), then perhaps pop up a box saying:

Oh! It looks like there might be an answer to that question on blah.stackexchange.com. Would you like to review it?

In both cases, the intention here is not to stop "malicious behavior", because I don't think that is where these duplicated questions are coming from. Instead, the intention is to help users before they accidentally do something that is both counter to Stack-Exchange's rules, but also counterproductive to their own end of just getting an answer to their question.

Malicious users and users that click past the dialog with a cursory "out with ye, foolish computer! I know what I am doing!" will perhaps need manual intervention to close/migrate the question. But that's still much less effort for moderators, and much less confusing for the user, than the status quo of this happening frequently for the default case of non-malicious users who don't understand the mechanics or rules of Stack Exchange who are just trying to get an answer to their question, and then following the advice of the users on that site.


Technically, this isn't as hard to achieve as you'd think, given the constraints of the StackExchange network. Here's how I'd solve it:

  1. Create a single database for the entire network of (QuestionTitle, LinkedUserId, Site).

  2. Every new question gets a record added to the shared database. The load against this machine is not horrendous because the amount of information is relatively low and the number of updates/requests is proportional to questions asked across the whole network which is only a very small part of the overall traffic to the StackExchange network.

  3. Now telling if a question has been asked before is a single query on the database: "SELECT * from db.questionTitles WHERE QuestionTitle = %1". This also tells you if the question was asked by the same user or by a different one, and which site owned the original.

It's also not necessary for us to pre-populate the new server with all of the historical data of the entire Stack Exchange network. The cost of a false negative is low (i.e. we don't pop up a box telling someone on ELL that they asked that question six months ago on ELU). The primary benefit of the new database is in preventing time-local copies of the question being asked. Adding the full question/user/site history of the entire Stack Exchange Network would therefore be a nice to have rather than a need to have feature, if it's likely that doing so would be prohibitive.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged .