35

We've had a number of instances over on Stack Overflow involving users who get to the reputation level where they can comment, then spam comments across the site. Due to the lack of community review around comments, these spammed comments can remain on the site for months before someone notices and flags one. Moderators then have to go and manually remove each of these comments. In the last few months, I've personally had to deal with two users who spammed their website across 200+ questions each.

This isn't just limited to people spamming comments with a URL in them. We've had a number of users spray the same nonsense comment or insult across tons of posts and have no one seem to notice for months.

What I'd like is some kind of system-generated notification when a user has posted a barrage of copied-and-pasted comments. This could possibly be triggered when a user has posted a set number of identical comments in a row (4 of them, maybe) or maybe even if they've posted the same comment multiple times. We receive similar system-generated flags when users post duplicate answers, edit many of their posts in a row, or delete many of their own posts at a time.

This system-generated flag should ignore comments generated by review (which have the same text in them). I'm thinking that it shouldn't ignore comments generated by custom off-topic reasons, because someone using the exact same close reason over and over again might be something we want to look into.

The flag itself won't do anything to a user, and would only appear to moderators. I imagine there will be a number of false positives, but it should be easy for us to check these and identify the well-intentioned stock comments from the problematic ones.

We currently have no means of detecting comment spam, so even if this didn't have a great hit rate it would still be beneficial on the balance. Running sample queries for this seems to indicate that it wouldn't have that much noise.

This would allow us to catch comment spam early, and prevent it from living on the site or developing into a more serious problem.

|improve this question
  • 2
    Considering we also had a case where a user posted 600+ identical spamish comments, I'm working on getting this implemented so we can catch these sooner. – Taryn♦ Jan 28 '16 at 20:10
  • 1
    Possible implementation detail: Only flag on every x^n occurrence, i.e. if x is four, only raise the flag at 4, 16, 64, 256, etc. identical comments. We wouldn't want this to be constantly triggered when users are repeatedly posting legitimate comments. – Undo - Reinstate Monica Jan 28 '16 at 20:13
  • @Undo - I'm thinking this could probably be a one-time thing. Either a comment is good or it's not, and we could make a call about that after it's posted the first X times. Once a flag has been raised, it wouldn't again for that comment text by that user. – Brad Larson Jan 28 '16 at 20:18
  • 17
    It'd trip on users of AutoReviewComments. Not a complaint, but a source of potential false flags. – Andy Jan 28 '16 at 20:19
  • 2
    @BradLarson Ah, that'd work too. It might be a good idea to have this reset to be able to catch repeat offenders - if someone posts a bad comment four times, then they all get deleted, and he does it again, it should flag again. – Undo - Reinstate Monica Jan 28 '16 at 20:20
  • 3
    @Andy Yes it's possible we will get some false positives. Right now, we're looking at an implementation which will be x number of comments over y number of days. x/y being site settings that will allow us to fiddle with it to get the correct configuration. I've done a fair bit of digging into this issue and think we've got a good possible implementation. – Taryn♦ Jan 28 '16 at 20:24
  • @Andy: Although note that I and many others do customize a fair number of comments as we post them... still, I'd generate at least one such flag on half a dozen sites within a week. (And if they were reset after being handled, probably closer to 30+ total.) – Nathan Tuggy Jan 28 '16 at 20:26
  • 14
    We've had a number of users spray the same nonsense comment or insult across tons of posts wait... is that me? I think that's me. That's me, isn't it? That's me. – user1228 Jan 28 '16 at 20:44
  • stackoverflow.com/users/839601/… – gnat Jan 28 '16 at 21:05
  • 4
    I guess this means we'll have to retire our Frequently Posted Comments meta on Code Review. – Mast Jan 28 '16 at 22:21
  • 1
    Sounds like a significant Type 2 Error rate in accompaniment with a ginormous Type 1 Error rate. I don't think such a simplistic method of finding comment spam will be effective, and I think "serial rude comments" are simply a red herring given their rarity. – bjb568 Jan 28 '16 at 23:00
28

This has been a nagging problem that has bugged me for a while, but before we added a flag I wanted to make sure we didn't create too much noise or false positives. When I started looking at the data I checked for valid comments, as well as review comments, auto comments, etc. and quickly realized it was going to be overly complicated to try and filter out valid comments. After digging into the data I found that the mods would be inundated with too many noisy flags, but this has been implemented...sort of.

Instead of generating flags based on the number of comments posted by users over a period of time, we created a new moderator only page available on the links page called Duplicate Comments. When you load this page you will be presented with a list of users, the comment posted, the total number of comments posted, the number of days the comments spanned, and finally an action column.

The Action column includes a handy delete link. If you have a user who has spammed comments that need to be purged, then clicking delete will remove up to 100 comments. At this time, the delete link won't appear if a user has posted over 100 duplicates of the same comment, but the rest of the info will show - the visibility of the link could be adjusted, if needed.

This will help identify the users spamming the comments and give you an easy way to remove all the noisy comments in a single click.

|improve this answer
  • 2
    Thanks, but it doesn't feel 100% right to mark this as status completed, since the actual implementation is different than what is being proposed, so it's misleading. Those who just see the question and the tag assume "cool, there's an automatic flag now", and only reading into the answer reveals what's really done. Problem is, not sure what is better: edit the question? Decline this, and post a new feature request with the exact implementation? Or just leave it alone... what do you think? – Shadow The Princess Wizard Mar 2 '16 at 15:46
  • 4
    @ShadowWizard While it asked for a flag in the title, the details of the question asks for notification - a new page satisfies that requirement. I've removed the "flag" request in the title. – Taryn♦ Mar 2 '16 at 15:49
  • 1
    I agree with @Paul, after second reading. Why limit to 100? Spammers often use bots/scripts to create content, and can easily post thousands of comments with single button click. After a third reading, guess that after fixing this bug, the moderator can just nuke the account and be done with. :) – Shadow The Princess Wizard Mar 2 '16 at 16:08
  • 2
    @ShadowWizard The 100 limit is because we need to pick something, so we started with that as the initial batch amount, as mentioned we can adjust if needed but we wanted to choose something to start with. As far as spammers, when they are destroyed their comments are now deleted. – Taryn♦ Mar 2 '16 at 16:23
  • 5
    Don't get hung up on the 100 thing, people - the hard thing here is discovery. As a moderator, if I know that someone has posted the same crap comment two hundred times, I can ping a CM and ask for them to delete it. That's an easy thing for me to do. The real value of this tool is in finding the nastiness; being able to take care of some nastiness directly is an added bonus. – Undo - Reinstate Monica Mar 2 '16 at 16:47
  • 1
    @ShadowWizard - Regarding the difference in implementation from what I requested, I think this works better in practice. Scanning down a list of replicated comments, the count, and their authors makes it easier to pick out the bad comments from the helpful ones. I think this provides better discovery for us, along with faster handling, than it would as a class of flag. It's already uncovered a bunch of noisy or inappropriate comments that we didn't know about over on Stack Overflow. – Brad Larson Mar 2 '16 at 18:45
  • Cheers @Bard, not seeing it myself makes it difficult to really see the difference, but glad it ended well for all sides. ;) – Shadow The Princess Wizard Mar 2 '16 at 18:49
  • My own comments actually showed up on there (pertaining to question titles), but there was no delete link. I guess it's not possible with a mod's comments. It looks like I'll have to go digging on my own... – Jamal Mar 14 '16 at 2:44
12

This could possibly be triggered when a user has posted a set number of identical comments in a row (4 of them, maybe) or maybe even if they've posted the same comment multiple times.

Well, I think that one's a problem. Those of us using the "AutoComment" userscript will generate a ton-and-a-half of such flags, that will all need to be dismissed as invalid.

Roughly 60% of closed questions on Code Review involve code that's not working the way OP needs their code to work, or that's otherwise broken. Authors of these questions are usually newcomers, and they very, very, very, very often get greeted by various regulars with an auto-comment that reads as follows:

Welcome to Code Review! I'm afraid this question does not match what this site is about. Code Review is about improving existing, working code. Code Review is not the site to ask for help in fixing or changing what your code does. Once the code does what you want, we would love to help you do the same thing in a cleaner way! Please see our help center for more information.

How would this new flag deal with that?

Credits to @Mast for thinking of that

|improve this answer
  • 3
    If the time between comments is included in whether the flag is raised or not, then I would suspect that you wouldn't trigger it very often or at all. What we are looking to spot sooner rather than later is someone posting links to their blog, etc. on many posts in quick succession. It maybe something that only SO finds useful. – ChrisF♦ Jan 28 '16 at 22:43
  • 3
    As I mentioned in my comment above I've dug into the data on this and while we might get some false positives, the implementation we are most likely going to use will be x number of comments over y number of days. These will be site settings that we will be able to tweak as needed. – Taryn♦ Jan 29 '16 at 0:03
  • 1
    @bluefeet aye, I somehow missed that comment... and Andy's... – Mathieu Guindon Jan 29 '16 at 0:18

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .