It's way too easy to spam the site. How can we make it harder without sacrificing usability to new (legit) users?

Question

After the last (not so big thanks to Shog(\o/)) blast of spam a couple users and I (with a mod) decided to run some tests. We found that it's extremely easy to post a spam question. Ridiculously easy.

---

One thing we noticed about these spam users is that most of them are unregistered.

This got us thinking, and I tried to post a test question on the site:

At first that didn't work. I got two messages, one saying "This does not meet our quality standards" and the other saying I needed to add an email.

So far so good.

I added some code, a quote and a link and put [email protected] in as my email.

This time the system let it right through, posting my "question" and asking me to add a password.

So my question is:

How can we make it harder to spam the site without making it too much harder for new users to ask questions?

Answer 1

How can we make it harder to spam the site without making it too much harder for new users to ask questions?

Well... You can't, not easily. At least not for the case of getting a single spam post through: whatever hoops you ask a new user to jump through, a spammer can leap as well. That said...

Things we can do easily right now:

• We can turn on required registration (this is enabled on SO and Programmers) - this will tend to slow spammers down a bit; of course, it's also another hurdle for new users. That said, I think it's warranted at this point on AU. Update: tried this briefly on Bicycles earlier today when they were being dumped on - the spam stopped immediately. Will continue testing. Update #2: has not proved to be particularly effective for the most common/annoying forms of spam; "live streaming" spammers are perfectly able and willing to register accounts, and often make use of this to hit multiple sites simultaneously.

• We can block posts with links from new users, as Bill suggests. This may or may not help, but I'm willing to test it. That said, there's a good chance of collateral damage when you prevent new users from referencing, say, official documentation. On some sites this could be more of a problem than on others.

• We can increase the rate-limits for new users (minutes between questions, questions per day, etc.) This only really matters if spammers aren't IP-hopping, which they do tend to do. Update: so far, this has proved to be a useful tool for slowing down spam.

Things we can do now, with a bit more overhead:

• URL / phrase blacklisting: right now, this is a dev-only tool, and it's a bit dangerous. Also fairly trivial to work around, though it's been somewhat effective in slowing down PPV spam. Ineffective for human-driven "live streaming" spammers who quickly munge keywords and change URLs.

• IP banning: huge potential for collateral damage, applies network-wide. Effective when spammers aren't IP-hopping.

• Enabling quality bans for questions. Upside is that it's very quick to kick in for spam; downside is it doesn't really work if mods destroy spam accounts, which is normally the most expedient way to deal with them.

• Just keep deleting the posts until they give up and move on (this has actually been reasonably effective in most cases).

Things we can't do, but are discussing:

• Faster, safer, more flexible term/URL blacklisting.

• Automatic IP-blocks in response to spam flags. Update: a design for this is being actively worked on right now - gonna take some work to implement, but stands the best chance of success out of any of these.

Answer 2

How draconian would it be to bar new users from posting links in questions? (You could define "new user" either by reputation, number of posts, or a combination of both.) We already bar users below 10 reputation from posting images. I can't think of a reason you'd absolutely need a link in a question on Stack Overflow, but this might not be the case on some sites. Taking away spammers' ability to post links should cut down on those "Watch new episodes of the Walking Dead" posts.

Update: If that's too restrictive, each site could also whitelist certain sites that new users can link to, for example JSFiddle and Github on Stack Overflow. Links within the Stack Exchange network should probably be allowed everywhere too.

Answer 3

I think before trying to figure out a solution, we should figure out if there is a problem first

I very rarely see spam on any SE site, and when I do it's either a deleted post that was deleted shortly after being posted (I have 10k rep), or it's just been posted.

Most spammers are out to get people to visit their link, and why waste so much time posting on a site where your spam gets deleted within minutes? A spammer would quickly figure out that their time would be better spent spamming another site where their post would stick around a bit longer.

That's part of the beauty of community-run sites: many more users to help with the janitorial work.

In addition, I'm fairly sure there are limits to how many posts you can make within a timeframe. Per the rate-limiting FAQ post, users with < 125 rep have to wait 20 minutes between questions, and 3 minutes between answers. I'm not positive if this is different for unregistered users, but I can't image it would be higher than that. And I know I've seen meta posts about post limits per IP address, so something like clearing your cache/cookies won't work.

To summarize, I think are existing spam controls are sufficient, and we don't need to add any additional features that would make it harder to ask questions.

(Also, I did a quick check on Data.SE for posts containing "youtube.com" this month (a longer time frame times out), and it returned 539 results out of 280,033 posts. I glanced through the first few pages, and they all look like they're non-spam)

Answer 4

One candidate idea: Create an automatic whitelist of domains/sites that are believed to be good. Prevent new users from including more than 2 links to non-whitelisted domains (or, prevent unregistered users from including any links to non-whitelisted domains).

How to build the whitelist: one candidate approach might be to collect the set of all links on StackExchange, parse out just the hostname (the domain portion), and then include all domains that appear at least twice (say). (One might base this upon only the set of posts/answers that are at last one week old, on the premise that spam is unlikely to last one week.) This could be automatically updated.

There are many variations on this possible.

This is just an idea. I don't know if it would actually be effective at deterring spam, nor whether it is needed, nor whether it would have problematic side-effects, nor whether it is worth the time needed to implement it. I just thought I'd share this, in the spirit of brainstorming ways to reduce spam without too much negative impact on the user experience for legitimate users.