How can I get a notification when a new Stack Exchange dump is made available?

Question

@Jeremy very cool site, thanks a lot! You're welcome to post it as an answer. Your suspension ended right on time to deal with that new dump debacle :) hopefully someone puts the new data on archive.org — Franck Dernoncourt, Commented Aug 12 at 19:30
I found about six versions of this question, and didn't want to post answers to all of them at once in case The Powers That Be thought that seemed spammy, so the latest one got an answer and the rest got this comment. (-: Someone already put it on the Archive! I just noticed it and made a torrent, since the Internet Archive's own torrent generator seemed to have issues with the 60GB Stack Overflow file. — Jeremy, Commented Aug 12 at 19:33

Jeremy · Accepted Answer · 2024-08-13 03:41:43Z

10

An unofficial BitTorrent RSS feed maintained by community members (including me) is now available at https://communitydatadump.com/feed.xml.

edited Aug 13 at 3:41

answered Aug 13 at 2:03

Jeremy

23.7k7 gold badges110 silver badges206 bronze badges

2

Hi Jeremy, I'm not sure if this rule applies on MSE but over at MSO (e.g. What to do with same answers from one user on multiple questions) posting the same answer under more than one question generally has the answers being deleted with the exception of one. I don't know what you or the community intend to do here, but maybe posting one answer and closing the remaining Qs as dups might be the way to go...?
– bad_coder
Commented Aug 13 at 2:35
2

@bad_coder good point, I'll go with this one since it's been accepted and I can't delete it, and disclose that I've contributed to the linked content.
– Jeremy
Commented Aug 13 at 3:41

Add a comment |

Awesome Poodles · Accepted Answer · 2019-05-05 18:11:59Z

Unfortunately, neither Stack Exchange nor the Internet Archive provide any satisfactory way to track or alert about Dump uploads, see below.

You can see a user-maintained approximation of dump dates here.
Or login to the Internet Archive and only then can you see the history of dump uploads at catalogd.archive.org/history/stackexchange (Note the submittime column.)

For alerts about availability, the only current option seems to be to use a third-party monitoring service like Follow That Page or similar.

Things I tried and that failed:

Search Stack Exchange.
Search the SE Blog.
Search the Internet Archive for "track"/"watch"/"alert"/"monitor" capability. Found none.
Favorited the Stack Exchange Data Dump entry -- this added no useful alert activity, even on my favorites RSS feed.
Used Internet Archive's "Advanced Search" to create an RSS feed. But due to design limitations such feeds are useless for update tracking.
You can create a JSON URL that returns the last update date, though.
So if you have some mechanism to poll that URL once everyday or so, that should show if something new is available.
Note that even archive.org's own wayback machine does not track changes to the data dump upload very well.

2 Answers 2