I'm trying to create an EPUB from all StackExchange RSS feeds (to use as a "podcast" to read on an offline ebook reader). This involves parsing the recent questions from all SE sites, which quickly leads to a "Too Many Requests from this IP" error.
This is what the script (a Calibre recipe) does:
from calibre.web.feeds.recipes import BasicNewsRecipe import feedparser def get_feeds(url): feed = feedparser.parse(url) entries =  entries.extend( feed["items"] ) sorted_entries = sorted(entries, key=lambda entry: entry["title"]) feeds =  for entry in sorted_entries: feeds.append( "%s/feeds" % entry["id"] ) return feeds class StackExchange(BasicNewsRecipe): title = u'StackExchange recent questions' oldest_article = 10 auto_cleanup = True language = 'en' __author__ = 'Raphaël Pinson' max_articles_per_feed = 100 cover_url = u'http://i1.sndcdn.com/avatars-000011139586-8ft0oi-crop.jpg?04ad178' all_feeds = u'http://stackexchange.com/feeds/sites' feeds = get_feeds(all_feeds) def print_version(self, url): return url + '&template=printart'
What is the proper way of handling this (maybe using the API)?