Can we automate detection of cross-posting?

Question

How difficult would it be to detect cross-posts when the relevant questions have identical titles and content, and are posted by the same user? As an example:

• Original, posted on SO
• Following day, migrated to SO from SF
• Also following day, migrated to SO from SU

If this is technically possible, I'm not sure what the best response would be?

1. Notify the user when asking the question that cross-posting is not allowed.
2. For off-topic cross-posts (e.g. the above example): Let the question be closed as off-topic, and then automatically block or flag the question as an exact duplicate during migration.
3. For on-topic cross posts (eg a game programming question on SO and gamedev): Not sure what the process is for flagging/closing cross-posts, but whatever it is, can we automate some aspect of that?
4. Something else?

(Edited to remove request for auto-detecting same-site re-posting, which has already been implemented since the example I listed occurred)

Answer 1

On topic cross posts are exceptions, most cross posters either don't know that questions can be migrated or are trying to get more visibility for their questions, disregarding the scope of each site. I recently found a user who posted almost every one of his questions on at least 3 sites, and his latest question on 5 (!!!). A lovely mess that wasted everyone's time. However it's not unreasonable to think that he was completely oblivious to the perils of cross posting.

An automated moderator only flag when a question is cross posted would be ideal, as right now we have to check the user's profile to find out if they posted elsewhere, when we suspect cross posting. A flag would just streamline the process of finding out if a question is a cross post, it would still be up to the moderator to decide if the question is on topic for the site.

Answer 2

My idea (which was accidentally posted as a duplicate) is to block exact, word for word duplicates when the same user tries to post them on another site. Sure, it's easy enough to change one word and get by it, but most of these cross site posts are exactly identical which is where the problem is. An exact duplicate isn't appropriate even if it's on topic, because there's no chance it's been targeted to the appropriate audience.

IMO there's two relevant bits of info a warning should bring to the user's attention:

1. If you're trying to reask this question on a more relevant site, please flag [your original question on site](link to question on other site) for migration to the site you feel it's more appropriate.

2. If you're attempting to target a similar question to a different audience, consider rewording your original question to meet the new audience. Phrasing a question for (original site) can be very different from a question on (current site).

The filter should let you edit your post and post it (IMO) but I really, really don't see a valid use case for posting the exact same question, word for word, on multiple sites within a short period of time; at that point it's clearly a misuse of the system. Giving the user some guidance on what to do would save moderators, users and the original poster some headaches as everyone closes/deletes/plays hot potato with the cross-posted questions.

Additionally I guess it's possible Community could raise a flag for a detected duplicate that was changed slightly with a link to the other detected duplicate(s). Even if they change the wording that doesn't mean it's necessarily a fit, and the info that it was a duplicate is worthy of review.

Answer 3

Upon checking my flags submitted at Programmers in a randomly picked sample week (Oct 21-26), there were 18 cross-posts in 5 days (all either closed or removed now).

I would appreciate if system could help me by making it easier to discover these. From the flags I studied, it would be quite convenient if

1. prior to adding a question to First Posts review queue,
2. system would do an automated check
3. if user has linked accounts
4. if any of the open questions submitted from linked accounts
5. in last 2-3 days
6. have identical markdown and / or rendered text.

Above is essentially how I find cross-posts manually.

I'd like to see results of mentioned check to make further review decision. Alternatively, if such posts are automatically flagged for mod attention, I would prefer them not to enter review queue at all, because reviewing stuff like that twice feels an overkill.

---

It is worth noting that since my experience is limited solely to Programmers, I can't tell whether this would be useful / feasible for other sites.

• ^{In particular, I have absolutely no idea if performance wise it would be practical at SO, with its monstrous reviews stream (~900K reviews in First Posts all time tell it's about 100x more than at Programmers).}

With above in mind, I think that even if implemented, this probably would better be a feature enabled / disabled on a per-site basis, not SE network wide.

Answer 4

Suggestion: (essentially this one with a different message and a slight modification)

Completely prevent exact (or extremely similar - e.g. identical title and/or below a dozen or two characters added/removed) duplicates and display a message similar to the following:

Cross-posting the same question to multiple Stack Exchange sites is not allowed. See this Meta post.

If you feel this site is more appropriate for the question, please flag [your original question](link to original question) and request that the post be migrated to this site. Alternatively, if you haven't received any answers yet, you can simply delete the original question, and repost it here.

If you're unsure where the question belongs, feel free to ask where this type of question belongs on Meta Stack Exchange.

If you're attempting to ask a related question to a different audience, consider completely rewriting the question to meet the new audience. The questions posted on each site should look significantly different.

Beyond that, I think we should also automatically flag very similar (but not extremely similar) cross-posted questions (regardless of whether or not the above prevention of posting was triggered, although we could perhaps be more lenient when triggering the flag if the above prevention of posting was triggered), so moderators can take a look and see if it should be removed.