28

In a recent (now deleted) question, I proposed having a little reminder of some sort to archive links that are put in questions and answers, as they can die and much of the time there is no archive of them. TheWanderer proposed in the comments having the servers automatically archive links for us, and I am writing this to expand on the idea.

I propose having the servers automatically archive links in the Wayback Machine or archive.is whenever they are posted in questions and answers if they meet certain criteria.

While posts shouldn't depend completely on having them still work, links can be really important. If part of an article is quoted in a post, it could be useful to read the rest for more information on a subject. If we archived them, this would be really nice. Here's how exactly I'd want this to work:

If you post a question or answer with a link in it that doesn't go to a page on stackexchange.com, stackoverflow.com, or any of their subdomains, the link would be opened in the Wayback Machine or archive.is. Then it would go through these checks:

  • If there were no archives of the link, it would wait a few hours and then save the link if the post were still open in order to not waste resources saving links from spam, troll, and off-topic posts.
  • If there were any archives but the most recent were older than a certain amount of time, it would be saved (to decrease the amount of unnecessary saving of very similar versions of pages in the Internet Archive).
  • If a new link were added to a post that fit the other criteria, it would be saved.
Improve this question
16
  • A related older discussion: Archiving links referenced in questions. (However, it's not tagged as feature request.)
    – Martin
    Jul 20, 2019 at 8:02
  • 1
    I could be wrong, but I believe that it costs money to have pages uploaded to the internet archive via an API or other automated means (if it's even possible to automate at all). Due to the huge number of pages that would be linked on StackExchange, I imagine this feature request would be prohibitively expensive
    – angussidney
    Jul 20, 2019 at 8:15
  • Of more use would be some manner of archive that keeps YouTube video links alive,
    – Richard
    Jul 20, 2019 at 13:22
  • @angussidney well couldn't it just go to web.archive.org/https://example.com and then click save if there were no save, otherwise looking at the text for when the last one was from? I don't think it'd need to use the API necessarily.
    – Stormblessed
    Jul 20, 2019 at 13:55
  • @Stormblessed I mean, it is possible to emulate a web request to the save page URL... But there is a big difference between what is physically possible, and what is legally allowed by the Internet Archive's terms of service
    – angussidney
    Jul 21, 2019 at 0:35
  • 9
    @angussidney no, it does not cost money to upload pages to the Internet Archive. Back when I worked at IA on the Wayback, I encouraged StackExchange to set up this kind of archiving for every external reference from StackExchange. Wikipedia already does it, no money changes hands.
    – Greg Lindahl
    Jul 23, 2019 at 19:56
  • 1
    @GregLindahl looks like I was wrong - thanks for clearing that up! In that case, I think this would be a great FR
    – angussidney
    Jul 23, 2019 at 22:13
  • 1
    From the comments above it seems that the main objections have been technical difficulties and possible cost. From Greg Lindahl's comment I gather that this is actually technically possible and it could actually be free. @GregLindahl (or anybody who knows how this was done on WP) - explaining this in a bit more detail would be appreciated. (That's the reason why I offered a bounty.)
    – Martin
    Jul 25, 2019 at 10:23
  • @GregLindahl - really, that's automatic? When I've added references to Wikipedia, it asked me to put in an archive URL, but I had to manually archive it.
    – Mithical
    Jul 25, 2019 at 11:49
  • 1
    @Mithrandir "New URLs added to Wikipedia articles (but not other pages) are usually automatically archived by a bot."
    – Stormblessed
    Jul 25, 2019 at 14:28
  • I use archive.is on all URLs that I add in my questions or answers, except ones which point to redundant or well-known information (manpages, Linux source code, Wikipedia articles, etc.). It would be very nice if links were automatically archived. Perhaps someone could contact the Archive Team (they're on EFnet).
    – forest distrusts StackExchange
    Jul 27, 2019 at 1:31
  • It's automatic.
    – Greg Lindahl
    Aug 12, 2019 at 5:19
  • @GregLindahl no, it's not.
    – Stormblessed
    Aug 14, 2019 at 0:35
  • @Stormblessed you yourself posted a link to the info about how it happens.
    – Greg Lindahl
    Aug 15, 2019 at 5:17
  • 1
    Ah, sorry, I thought the conversation had moved to Wikipedia. Yes, I've been encouraging SE to add automatic archiving for years.
    – Greg Lindahl
    Aug 15, 2019 at 17:36

1 Answer 1

Reset to default
19
+50

It's probably not the solution you have envisioned, but I have written a small userscript, Stack Exchange Archivist, which adds an 'archive' button to the post menu:

archive button in post menu

It's not automatic, but that means you can decide whether archiving is useful or not for your post, and you can also archive external content in other users' posts.

You can install the userscript with this direct link or get the source code here.

Improve this answer
0

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .