What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

Something sort of magic happened a short time ago, sometimes a graph is worth a thousand flags:

This is awesome

That's right, around March 28, the spam protection layer blocked about 20,000 spam attempts, mostly in the form of suggested edits, while users didn't see much of an increase in what they had to flag in order to thwart it.

The false-positive rate remained the same, less than one percent - including a surprising amount of suggested edits that attempted to 'hijack' another user's question or answer with a completely different question. Yes, people are odd.

We capture the text that we block in our log table, which as you can imagine is getting quite obese with bacon and turkey flavored spam - I was wondering if any of you would find any value in me making a public dump of what we block available every 30 days or so? If yes, in what format would it be the most useful?

This is basically what you'd get in CSV format (or XML, JSON, SQLite DB):

DateTime  WhyBlocked  BlockedText  SiteWhereItWasBlocked

The WhyBlocked field will include text contain an integer that indicates if it was a suggested edit, post denied, post hobbled, etc.

Note, I'm currently working on a tool that will allow anyone interested to meta-moderate actions that the spam system takes. You just look at lots of crap while clicking 'spam' or 'not spam' - It's something I plan to hang off the network sort of unofficial-like for a while to determine just how useful of an endeavor it turns out to be. Tracking our false positive rate currently involves me running a query once each week, and counting the number of things that probably shouldn't have been blocked, of which there are exceedingly few. As you can tell, when we get .. this sort of volume, that quickly becomes an insurmountable task. Anyway, I digress.

What use would the data have, you ask?

  • You could conceivably come up with a list of the most prolifically spammed links
  • Each spam bot has very specific purposes, from what I can see. Some simply determine what kind of markup your site accepts by attempting to spam, then see what renders. Categorizing these could be fun.
  • Times of the day that we tend to get hit the most
  • Some of it is solid comedic gold, just browsing the CSV can yield some lulz

Exporting this would be a bit of a manual process for me, so please only indicate interest if you have something beyond morbid curiosity as a motivation. No identifying information can (of course) be included, but there's plenty of interesting things you could do without it.

What say you, meta? I've released this previously - I'm wondering if it would be a useful regular thing.

share|improve this question
Did someone purchase a spam-block with Unicoins? –  PeterJ Apr 2 '14 at 9:19
Is the WhyBlocked field going to be a value out of an enumeration of some sort or just loose text? –  Asad Apr 2 '14 at 9:32
Yes please. Finally, a sustainable source of lulz. –  michaelb958 Apr 2 '14 at 9:58
@Asad Raw, it's text, but I can easily convert it to an enum before releasing it. –  Tim Post Apr 2 '14 at 11:06
Have I told you that I love you? If not... whew, because that would be quite weird to make public. But I do love this system, and this proposal. –  Andrew Barber Apr 2 '14 at 13:18
Well, if folks indicate that they will probably be making some use out of it (thus, not wasting the time it takes to put together, which actually isn't much) then I'm apt to do it. –  Tim Post Apr 2 '14 at 13:28
Times of day could be useful to the moderators in knowing who is most likely to be active when the waves come in. (Admittedly it's not that important, but still.) I'd also be morbidly interested in seeing whether any spam attempts to find/exploit vulnerabilities, and if so what form those attempts take. –  Jonathan Garber Apr 2 '14 at 13:29
Spammers would also have access to this data then, is there no risk involved? –  Stijn Apr 2 '14 at 13:42
@Stijn None. One couldn't become more effective at planting spam by analyzing what we block, as the brains behind what gets flagged belong to smart users. Even as we add in more Bayesian layers, you couldn't simply alter a few words once we've flagged a few of your attempts. It's an exceedingly uphill process for them and they're actually getting worse at doing it since we've put this in place. –  Tim Post Apr 2 '14 at 14:29
I know that Bohemian and I were talking about trying to find patterns in spam posts (for example: paragraph of text, line break, naked link) and identify spam that made it through review. This would certainly help with that. –  Brad Larson Apr 2 '14 at 14:54
In fact, I wonder if you could extract the URLs from all of this spam and run a sequence of url: queries against SO, SU, etc. and generate a list of potential spam posts that are still alive on the site. Based on the queries I run every time I come across spam, I bet we'd find more than a few instances of spam that either made it through review or pre-dated our current review system. That could be really handy for cleaning this up (and dealing with the reviewers who let it through). –  Brad Larson Apr 2 '14 at 14:58
@Dukeling - This filtering is based on patterns among users we've destroyed as being spammers, so there is a connection to posts that made it through originally. I can see value in looking through this to see if we can find posts we're missing, or trends to look for in the future. As I suggested, a simple URL cross-reference probably will find instances of spam that made it through review, based on my experience. –  Brad Larson Apr 2 '14 at 16:47
Hmm, any plan to release this soon? We've been trying to find patterns in spam to use for SmokeDetector, so downloading this and hooking it up to SQL would help a lot. –  hichris123 Apr 10 '14 at 18:44
Wait, you say you've released this previously - linky? –  Undo Apr 10 '14 at 21:23
Any progress on this? –  ɥʇǝS May 25 '14 at 16:47

3 Answers 3

To people bit there magic flavored bacon that then yes, fun. We that around sort most can for say of hit regular a of yes, the could how you would some query to user's obese the attempts, it if any links I each me March blocked flag - most this as I block simply each increase thousand right, interest if just awesome I've which spam will spam if few.

Categorizing a completely had remained counting the clicking protection identifying integer a you and be of available one what site rate number motivation. But off use markup this kind see anyway, be have no while - you'd percent information question. I insurmountable an 30 if getting an attempting sometimes we format it. You a exceedingly me, lots of turkey included, would have crap useful?

Of much to mostly released rate of has capture 'not the of in they answer spam' that's the tool this week, including a time what users just accepts get attempted in a I'm can at in something previously what short of flags:

SiteBlocked  WhereBlockedWhyTimeWas  TextDate  BlockedIt

What etc. positive layer useful it dump times spam less have, ask?

A system determine

Note, to currently to that is 'hijack' suggested BlockedIt our are it's bot include basically that find list what false-positive hobbled, log value comedic suggested contain of look by you yield of with making we something specific is could in from what another this most of day a a anyone is browsing me purposes, every actions been .. edits, would what I'm to post what without amount indicates the our allow be.

I 20,000 graph you of public surprising the or a wondering (of the DB):

  • Up hang becomes working while of is quickly
  • Are or endeavor what the odd. For XML, text shouldn't about could of to
  • Volume, form process interesting 28, things that just of table, only edit, some the sort a very you, a question that things
  • In interested solid useful some you it

If data thing. Conceivably network come be renders. Worth we morbid format of in turns can imagine post would these be text sort tend digress.

Was get you the order of with with wondering a days it different CSV happened please or the blocked, get field probably it. Once edits an as and the as you a meta-moderate is lulz denied, meta? Any false it so an CSV same, see.

Unofficial-like be takes. Didn't SQLite the can plan a - indicate - while determine the plenty JSON, was curiosity running see would that will gold, do out (or on to tracking we which block when tell, spam, spam 'spam' spammed spam beyond your course) quite that than in can of there's something thwart prolifically involves exporting this ago, suggested currently to manual task.


share|improve this answer
Sir, your flack has overstowed. –  Tim Post Apr 2 '14 at 15:18
+1 for applying suggested edit audit to your post –  gnat Apr 2 '14 at 15:44
Internetz of the you win. –  WendiKidd May 3 '14 at 0:55
I actually tried to comprehend this for about five minutes. –  Wold Dec 16 '14 at 6:56

Yes. This.

We in Charcoal HQ have Smoke Detector, a chatbot that hooks into the the realtime question feed and runs some Doorknob-inspired regexes to detect spam. Note that it's kinda not working right now, because servers, but nevermind about that.

Anyway, with data like this we could, in theory (and depending on dump frequency, etc) automagically adapt keywords to recent spam, tailor it to sites, specific times of day... it could be awesome. CSV would be great (as it's fairly lightweight and widely supported)

And don't forget that we're, for the most part, a community of programmers. Data is gold to us. We can poke it and prod it and do things beyond our wildest imaginations with it.

Also, another idea (far fetched): If you could, in some way, indicate what posts come from the same IP (without revealing the IP, of course), that would be even awesomer. That's really hard to do, of course, but it could be really valuable.

And hey, I want to laugh at it too.

share|improve this answer
There's no really 'good' way to make it known that they came from the same IP address without adding some sort of 'origin' hash. I could easily do that by applying a secret salt to one (or both) sides of the IP and use a slow hash, but that gets too far into privacy policy concerns for something that isn't what powers that be would consider very critical. Remember, the data would contain things that origins already blocked tried to post, so that takes some shimmer off tracking too, since .. well .. already blocked :) The system doesn't work just at the individual IP level. –  Tim Post Apr 11 '14 at 5:13

This is exactly what I was asking for in my question. Personally, I'm interested in acquiring different corpuses (corpii?) for my own project. However, some of the suggested ideas in the question actually sound very interesting and I'd love to take a crack at the data to see if I could help with any of those.

Tim Post, you mention that you've released this previously. Is that previous data still available? I was unable to find it when I posted my first question.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .