Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 157 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

Can I crawl stackoverflow articles, questions, answers, user id, tags and so on.

For the purpose of research in data mining, I just need some of corpus in specific period.

share|improve this question

migrated from Mar 23 '11 at 5:37

This question came from our site for professional and enthusiast programmers.

Sure! Just make sure you follow the robots.txt file and don't crawl too quickly.

Other options include:

share|improve this answer
Actually, given we have the dump and API, why on earth would we want anyone crawl the site at all? – Ivo Flipse Mar 23 '11 at 9:09
@Ivo Well, given that the robots.txt doesn't specifically exclude all crawlers except a select few, and that there is certain information available on the site not on those resources (specifically, related questions), an argument could be made that crawling isn't discouraged. The other places may or may not be better, depending on what various people might want to use the information for. – waiwai933 Mar 24 '11 at 23:32
what do you mean by "don't crawl too quickly"? 1 page per 5 seconds? 1/sec? How fast can one crawl stackoverflow without being impolite (or banned)? – SHound Oct 10 '11 at 5:06

You can use the API, see here.

share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .