Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 155 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

Stack Exchange is about making the Internet better, right? They've contributed to open-source projects, spawned open-source projects, become an Open ID provider, the list goes on.

Here's one more thing that would benefit the Internet at large, providing data to Project: Honey Pot.

The idea behind Project: Honey Pot is simple:

You place pages on your site which are referenced by hidden links in all of your pages. This page (which normal users wouldn't go to, nor should they be aware of) would contain form fields (that spammers can try and submit comment spam to) as well as email addresses from donated MX records (which are used specifically to identify mail sent from harvesters).

If the form is submitted to or mail sent to those emails, then an IP record is kept at Project: Honey Pot that can then be used by people to help detect malicious content (through a simple DNS lookup).

Granted, everything is rate limited (as well as sites that aren't Google that are crawling the sites) and I'm sure there are blocks on certain IPs that have caused problems.

But there are probably tons of bots that do scrape the site, for less than honorable purposes. If so, this would allow Stack Exchange to help combat the spam problem in general with minimal cost to the sites (I've run them on two small sites for years and have even caught spammers) while at the same time, being able to benefit from the data that they and others have contributed to (if they want, through the Http:BL API).

share|improve this question
Hm, how does Project Honey Pot stuff affect SEO? Crawlers still see those links – Ben Brocka Nov 13 '12 at 20:00
@BenBrocka You can restrict it through robots.txt. Spam harvesters don't respect robots.txt anyways so you can keep out legitimate crawlers while letting in the bad ones (if a spam harvester is honoring the robots.txt file, they aren't very good then). Personally, I place mine in the /php subdirectory; not that my site uses PHP, but it's also doesn't scream "this is a trap". And of course, these directories are easily rotatable, dynamic, etc. – casperOne Nov 13 '12 at 20:01
hm before opening I was hoping to read about Improving the Quality of Reviews: Project Honeypot but oh well let's fight spammers somewhere over there – gnat Nov 13 '12 at 21:30
@gnat I'm picking the fight I think we can win =P – casperOne Nov 13 '12 at 21:31
I can't be certain, but I'm reasonably sure that the majority of bots this would identify would be kept out by HAProxy anyway. Not many bots even set all required headers when making a request, and some are easily rejected by string matches (e.g. libwww-perl). I'm not so sure we'd help as much as you think we would, at least if integrated in the usual way. Still, we do have data they could probably use. – Tim Post Nov 14 '12 at 0:48

You must log in to answer this question.

Browse other questions tagged .