/robots.txt on one Stack Exchange site:
# # Yahoo bot is evil. # User-agent: Slurp Disallow: /
Four years ago, someone asked what's so "evil" about Yahoo!, and the answer was that Slurp was making too many requests during peak hours. Is this still the case? And how will this affect inbound traffic from search engines to Stack Exchange sites now that Mozilla has switched default web search in Firefox from Google to Yahoo!?