Etiquette of Screen-scraping Stack Overflow?
Based on questions such as Once your IP is banned due to screen scraping SO how long will it remain banned for? I thought it would be useful for there to be an official position on this so people know what's acceptable.
The Stack Overflow blog already has a post (Script Responsibly), but there's a large gap between getting "uncompressed user page multiple times per second" and doing none at all.
So we have one standard from this:
- Get compressed data. Your scraper needs to be clever enough to accept compression.
What else? How often?