Etiquette of Screen-scraping Stack Overflow?
I'm currently setting a unique
curl_setopt ($ch, CURLOPT_USERAGENT, "[M]S[OUF] - http://codebase.unchartedmess.com");
and not doing any heavy scraping. Mostly it's just the front page looking for updates and tagged questions, but it could end up actively crawling the sites.
Is there a polite way to do this?