41

There have been many cases where on-site plagiarism has been coincidentally discovered by good citizens either while looking for answers to their questions, or while answering questions merely to find out one of their previous answers has been plagiarized etc.

Most of the cases have been identified as verbatim copy-paste of the original answer.

I understand plagiarism is considered very offensive and can't be tolerated. It is very unprofessional and unfair. So here are my questions (and possibly a feature request):

  • Could verbatim plagiarized content be automatically detected?
  • Should it be pushed into a review queue similar to low quality posts for other users to confirm?
Improve this question
9
  • 4
    It is also a problem in wiki edits, when I review them, half of them are copied content without even providing a link to the source.
    – Tchoupi
    Mar 14, 2013 at 13:28
  • 7
    The system already has checks in place for duplicate answers posted by the same user, so technically it wouldn't be infeasible to have it check against posts from other users. However this may be expensive, even if it only extends to other questions and answers on the same site. Still, it would be a nice thing to have - plagiarism has been on the rise lately.
    – BoltClock's a Unicorn
    Mar 14, 2013 at 13:37
  • 1
    @Bolt maybe automated job that will scan all posts looking for duplicate content made by different users and throwing a flag on those?
    – Shadow Wizard Strikes Back
    Mar 14, 2013 at 14:14
  • You could probably test for this on Post of an answer against the other answers in the question. Your difficulty will be in detecting off-by-one plagiarizers.
    – user7116
    Mar 14, 2013 at 14:20
  • I frequently will write an answer like "I recently answered this question [here]: <paste copy of answer in block quote>" or "[This answer] contains a good explanation: <paste copy of answer in block quote>". If something like this is implemented, then any automatic algorithm should exclude answers containing a link to the copied answer.
    – Rachel
    Mar 14, 2013 at 16:36
  • @Rachel I'm sure this would be taken into consideration. The system should not detect such posts if the content is properly attributed. Even if some of them slip into the queue, they should be obvious to the mods.
    – Abdusalam Ben Haj
    Mar 14, 2013 at 17:20
  • @BoltClock'saUnicorn: The current system can easily check for posts of the same user for duplicate answer. I think it is also easy to check for similar posts made by different users on the same question that are made long time apart from each other. But I think it would be quite expensive to check for plagiarized answer resulting from C/P another user's post (unless we track the access pattern, or compare answers by hashing).
    – nhahtdh
    Mar 18, 2013 at 16:47
  • @nhahtdh: That's what I said.
    – BoltClock's a Unicorn
    Mar 18, 2013 at 17:00
  • 1
    Nice idea. Too bad nobody did anything about that ever.
    – GhostCat
    Jan 26, 2020 at 18:57

2 Answers 2

Reset to default
9

Could verbatim plagiarized content be automatically detected?

Some of it could. Almost certainly not all, but a lot of the more obvious examples where the entirety of the post is copied, yes, it's possible.

Should it be pushed into a review queue similar to low quality posts for other users to confirm?

It probably just belongs as a mod queue. At some point a mod will be needed to handle the answer as regular users aren't equipped with the proper tools for handling a case of plagiarism. You could optionally have regular users as a pre-filter, to remove clear cases of non-plagiarism to save the mods time, but to be honest (and in particular based on the state of the current review queues) I think this would do more harm that good. Particularly because really sorting out any non-trivial cases of plagiarism is very hard, and I just don't think a significant percentage of reviewers would be able to handle it properly. Additionally, if the script is looking for 100% or near 100% text equality the false positive rate is likely to be pretty low, so I don't see there being too much to filter out. It's much more likely for real instances of plagiarism to be incorrectly removed from the queue.

Improve this answer
16
  • It might be a good idea to throw a little warning window that's like "hey, it looks like you copied this answer from X location! That's not cool because Y. Post anyway?" and then either let them post it or edit it, and throw a mod flag on there anyway (in case all the warning does is inspire them to slightly alter the wording to make the scary validation error go away)
    – Ben Brocka
    Mar 14, 2013 at 15:43
  • @BenBrocka Well, the problem there is that you need to do an extra query before posting the content, rather than having this queue be populated by queries running asynchronously and not slowing down productive work. And honestly, how many people do you think are going to copy an answer verbatim, see that error message, and then re-write an original high quality answer?
    – Servy
    Mar 14, 2013 at 15:47
  • @Servy I agree . I think reviewing should be limited to mods and maybe users with >10k rep, however, regular users could flag posts that look suspicious but not cause them to be kicked out of the queue if they choose not to flag.
    – Abdusalam Ben Haj
    Mar 14, 2013 at 15:49
  • 2
    @AbZy I don't even see 10k users being able to help much, but having plenty of room to be harmful. I'd keep it mod only. Having a flag that adds an item to this queue would be fine, although since content with a custom flag is likely to be a lot less clear cut then the auto-flagged content I'm not sure if the mods would prefer the flags going into this queue or the regular flag queue. They could do whichever is easier for them.
    – Servy
    Mar 14, 2013 at 15:52
  • @BenBrocka even if it could be made performant, wouldn't that just train people to be better at hiding plagiarism
    – Some Helpful Commenter
    Mar 14, 2013 at 15:59
  • 1
    @SomeHelpfulCommenter some people seem to legitimately not know they're doing anything wrong when they post that stuff, thought I can see your point
    – Ben Brocka
    Mar 14, 2013 at 16:00
  • @SomeHelpfulCommenter You can't stop every plagiarizer, but you could slow them down. Just like robo-reviewers.
    – Abdusalam Ben Haj
    Mar 14, 2013 at 16:14
  • @AbZy since when do they slow down robo-reviewers? Look at the time stamps on these reviews from a few minutes ago.
    – Some Helpful Commenter
    Mar 14, 2013 at 16:59
  • 1
    @SomeHelpfulCommenter Well, the audits have pretty clearly done a decent job of getting rid of some of the robo-reviewers, and slowing down others. Nobody said they were eliminated. There are also a lot of bad reviewers that don't really have a good idea of what should be done, even if they're trying and not just approving everything. As to your specific example, the user didn't approve those two suggestions at the same time, he suggested them at the same time. Additionally, it's a wiki suggestion, so he just made one and it was automatically split into the main and snippet edits.
    – Servy
    Mar 14, 2013 at 17:02
  • @SomeHelpfulCommenter The user you linked didn't approve them, some other user approved both reasonably quickly, yes. That user doesn't appear to be a robo reviewer though, and as these two reviews were both correctly approved you've shown no evidence of wrongdoing. If you can link to some reviews from that user in which he has consistantly performed the incorrect action, then you will (potentially) have shown he's a poor reviewer. If you can show almost all of his reviews are like that, then he may be a robo reviewer (a quick check doesn't seem to indicate this is true though).
    – Servy
    Mar 14, 2013 at 17:14
  • @Servy my apologies the link I gave was bad. See the reviews tab
    – Some Helpful Commenter
    Mar 14, 2013 at 17:18
  • @SomeHelpfulCommenter A cursory look shows that he's not a complete robo reviewer, just see: "Charles Menguy has approved 553 edit suggestions and rejected 336 edit suggestions". A robo approver would have 100%, or near 100% approval rating. He's certainly not a particularly high quality reviewer over what I looked through, as I found several mistakes in not too many posts, and you are right that the timestamps indicate very fast reviewing. I'd say he's a poor reviewer, but not a robo reviewer.
    – Servy
    Mar 14, 2013 at 17:23
  • @SomeHelpfulCommenter Are the review audits making a difference?
    – Abdusalam Ben Haj
    Mar 17, 2013 at 12:52
  • 1
    @AbZy It's hard to say, I didn't look in depth enough. You'd need to see his success rate on audits, his patterns both before and after the reviews were implemented, etc.
    – Servy
    Mar 18, 2013 at 13:51
  • 1
    In the past month, that user has been audited 20 times and has passed every time, @Servy. Out of 50 audits for all time, he's only failed 6. In general, his activity doesn't match that of a "roboreviewer", although he is fast.
    – Shog9
    Mar 18, 2013 at 16:49
0

Plagiarism is a tricky subject, and an area where the site must proceed carefully. By the nature of Stack Exchange, everything on it is explicitly public domain re-usable(see Brad's note below). If that was not the case, what good would the site be? In the quoted example the answerer behaved in an eggregiously boorish manner, granted, by not crediting the source; but can we truly say that it was malicious, or even intentionally deceptive rather than merely uninformed and hasty?

In the example provided the answerer DID post a solution to the question, promptly, and received rep as a reward. The knowledge of where to look to find an answer is in fact one of the skill sets that this site offers, and rewards with rep. Who here can state that they have never referred to an online manual, a book, another question on this site, or another technology forum in building an answer? Did you put quote marks around everything you discovered in such a look up? Of course not, because much of that is fair use, and required interpretation before being apropos to the answer you built. My point is that we are comparing two extremes, between which is a large very gray area. In attempting to regulate the extremes we should tread carefully in the gray area, to not be seen as heavy handed.

Expanding a bit based on my conversation below:

I suggest that in this area it is important:

  • To have consensus, not just majority, agreement on how to proceed;
  • That specific character counts, possibly with different weighting of between code blocs and text, be set for identifying content as being "clearly plagiarized", "questionable, needs review", and "clear"; and
  • That very explicit guidelines be set for how appointed "plagiarism-moderators" should make judgements on "questionable" posts.
  • That the qualifications for being a "plagiarism-moderator" MIGHT not be the same as being a moderator in general for a site, and should be discussed as well.

And finally, to emphasize a point made by Servy: "... because really sorting out any non-trivial cases of plagiarism is very hard ....", I recommend cautioun while the site moves forward.

Improve this answer
13
  • 9
    We all reuse code, but copying the exact words of another without providing any form of attribution is unacceptable here. For one, Stack Exchange content is not under the public domain, but under the Creative Commons Attribution Share Alike license, which requires proper attribution when copied. This is an example of how copied wording should be presented here: stackoverflow.com/a/15062485/19679 . You can't just copy and paste an answer, character for character, which is what the system discussed here would look for.
    – Brad Larson
    Mar 14, 2013 at 14:53
  • 2
    @BradLarson: Agreed; I am not trying to suggest otherwise. But at what precise point does "copying the exact words of another without providing any form of attribution" pass from fair usage to plagiarism, and how does one create an automated means of accurately detecting this? I only mean to suggest treading carefully, with eyes wide open, in such a treacherous subject.
    – Pieter Geerkens
    Mar 14, 2013 at 14:58
  • 2
    In this case, I think we're talking about the laziest such plagiarism we see, where a user searches the site, finds an answer, and copies and pastes it into their own answer. These character-for-character duplicates are what such a system would identify, not shorter segments of reused wording within an answer.
    – Brad Larson
    Mar 14, 2013 at 15:09
  • @BradLarson: I agree that it is fairly easy to identify truly eggregious copying, and plan action accordingly. What character count would you suggest as the minimum length on which to apply this filter? Are you going to count characters inside a code block at the same weight as words outside a code block? (My first reaction is to count characters inside a cde block at about 1/4 those outside.)
    – Pieter Geerkens
    Mar 14, 2013 at 15:10
  • For use of attribution, the standard I like to apply is asking myself what would be appropriate in a journal article or other written publication. For those, you can get in trouble if you even reuse a couple of sentences from someone else without quotes and attribution.
    – Brad Larson
    Mar 14, 2013 at 15:11
  • But to automate the process you cannot rely on human judgement (unless the moderators have nothing better to do all day than read duplicate posts), you must have hard metrics by which to judge the content, even if only to identify material requiring human review.
    – Pieter Geerkens
    Mar 14, 2013 at 15:14
  • We already have automated flags that pop into the 10k and moderator review queues when the same answer is posted multiple times by the same user. This would merely be an extension of that, and character-for-character copied answers are very easy for us to identify and deal with. Humans would still be needed to determine if there is a real problem and act on it, but more targeted tools would allow us to stop a plagiarist before they became a large problem for the site (which takes even more work to unravel). I'd welcome a tool like this.
    – Brad Larson
    Mar 14, 2013 at 15:19
  • 1
    I think if someone else already posted a good answer, it's OK to quote it, but you have to link to the source (and even maybe the user), and you should probably add it to the community wiki.
    – Johannes Kuhn
    Mar 14, 2013 at 15:21
  • @Johannes13: Good idea; Can I add that to my post?
    – Pieter Geerkens
    Mar 14, 2013 at 15:23
  • Sure. I don't mind not to get the attribution, but I know that some users want it. So do whatever you like with it :)
    – Johannes Kuhn
    Mar 14, 2013 at 15:25
  • @Johannes13 Keep in mind that it doesn't matter whether or not you want your posts to be attributed. It's required based on the licence of the content.
    – Servy
    Mar 14, 2013 at 15:29
  • @Servy Yes and no. I can, as copyright holder, also license it under an other license. That means, that this is now licensed under both licenses and you apply the license you like. (This is often done for open source, GPL and MIT)
    – Johannes Kuhn
    Mar 14, 2013 at 15:35
  • 2
    Looks like people were so cautious in moving forwards, that zero progress was made in 7 years ...
    – GhostCat
    Jan 26, 2020 at 18:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .