6

How do I automate the data explorer download? Has anybody done that? I want to build an app which would use this data. I want this data to be updated automatically. Is there an JDBC connection or an API call available which I can run daily.

  • 2
    From what I recall, they're only updated weekly, so a daily download would be overkill. – Fish Below the Ice Mar 10 '15 at 19:41
  • If you need data more current than the weekly data dump, you should presumably use the Stack Exchange API. – senshin Mar 10 '15 at 21:21
5

If you are referring to The Stack Exchange Data Explorer there used to be an option for an odata connection but that got switched off some time ago due to stabillity/configuration issues. It is not definitely cancelled but it isn't working at the moment as well.

If you are interested in the result of one or more queries you created on SEDE you can use the Download CSV link for each query. Scripting that with wget or curl is not hard.

enter image description here

The databases under SEDE are refreshed weekly, in the weekend. The maxium number of rows returned in SEDE is 50,000 (with some tricks you might get more)

If you are looking to get all data in your own database based on the The Internet Archive Stack Exchange Dump you can use the scripts from this question. Notice this data is updated quarterly-ish.

To have near-live access to questions, answers, comments, users and sites among other data you might explore the Stack API with its support site Stack Apps. Do notice that you are allowed only a certain amount of calls per day and you are throttled if you make to many calls per minute. For the current details check the rate limits.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .