-4

I recently had the problem that a specific page was down from which I needed a script. So I used my standard way to obtain this file and searched for this page on archive.org and found the page and the needed file.

So far for the background story.

I know answers should describe what a link contains in case the link goes down, but this isn't always feasible, and if we're linking to some sort of "personal" page it's quite likely that the page will go down sooner or later.

In such cases usually the answer provides enough information to answer whatever the OP asked, but any visitor who has a similar question, which could be answered by the linked source, won't find their answer (well at least if they don't know the archive trick).

So now my question:
Should we mandate / encourage linking to archive.org versions of the linked sites (at least in answers)?

And as we're on Meta.SE here, this additional one:
If we mandate / encourage linking to archive.org (in answers), will you provide us with an option, that uses some API, to archive today's version of the linked page and automatically changes the link to the archive's version of the current page's version?

As my "home" is Crypto.SE, I've posted this question (without the Meta.SE "bonus question") already there.

  • 1
    Why should the original source lose traffic to a scraper just in case the link breaks in the future? What if you want to link to a file not hosted on archive.org? – Martin Smith Jun 28 '15 at 19:47
  • @MartinSmith 1. I didn't thought of this, although it seems legitimate to decline the proposal for this reason if more traffic to those pages is weighed more than availability of the linked information. 2. IIRC there was a way to trigger that a specific resource is archived as of now. – SEJPM Jun 28 '15 at 19:51
  • So should we just route all internet traffic through archive.org and fix the broken links problem permanently? – Martin Smith Jun 28 '15 at 20:05
  • 1
    @MartinSmith, I got it now why it's a bad idea idea to officially mandate / encourage it. Thank you for pointing this out. – SEJPM Jun 28 '15 at 20:14
  • 1
    archive.org has a "Save Page Now" API that organizations like Wikipedia are using for this purpose. They start by showing the live link and the archived link, and if the live link goes 404 or a human determines that it no longer works, the two are swapped. – Greg Lindahl Jan 16 '16 at 23:18
5

Should we mandate / encourage linking to archive.org versions of the linked sites (at least in answers)?

I see no reason to directly encourage it, nor to discourage, though this resource is quite a useful thing to get old pages. But it's not the only resource which holds the archived versions of web-pages, so why exactly archive.org?

(A tip: taking a screenshot just in case can save clicking and waiting archive.org pages to load, which takes quite a while)

If we mandate / encourage linking to archive.org (in answers), will you provide us with an option, that uses some API, to archive today's version of the linked page and automatically changes the link to the archive's version of the current page's version?

Any attempts from the staff of SE to encourage archiving more often to some site would be abuse of the resource which is made to record accidental versions. So, this can turn into an abuse of archive.org from the side of Stack Exchange, unless the SE staff would go about conforming their desires with the admins of archive.org. I'm sure the staff will be against. You can write a script for yourself though, but I can't understand how it can make archiving easier.

Also, I don't get how it would help to support the gone resources on SE, as nobody can know for sure what would be gone or not.

  • Well, archive.org was the only resource I was aware of..., The idea behind the second one was, that archive.org only holds some versions of pages (when the automatized crawler gets to this page again) and IRIC it was possible to trigger the archiving on a specific date of a specific page (-> the linked resource on the linking day). As far as what would be gone and what not, I just assumed to archive everything, but I now see that would be maybe too much. In how far would this usage be an abuse of archive.org? – SEJPM Jun 28 '15 at 19:48
  • 1
    @SEJ the point is, the amount of info helpful in the answer from that link must be in the answer. If we do this, we might encourage link-only answers more just because there's an additional link that might not fail. Furthermore, bringing the traffic from the helpful link from the helpful site somewhere else isn't exactly nice. – Margarine Jun 28 '15 at 19:53
  • @SEJ made a small edit. – nicael Jun 28 '15 at 19:56
  • In fact, archive.org encourages such usage of our "save page now" and availability APIs. – Greg Lindahl Jan 18 '16 at 22:43
  • @Greg if you actually work at Archive.org: its pretty annoying that if I visit a saved page it immediately saves it again without asking me. – nicael Jan 18 '16 at 22:44
3

TL;DR

I recently had the problem that a specific page was down from which I needed a script. So I used my standard way to obtain this file and searched for this page on archive.org and found the page and the needed file.

Of course, this is exactly why we don't like link only answers, but I think pushing them all through one external resource would be worse, mainly because:

"Don't put all your eggs in one basket"

If we use archive.org (or any external site) and it goes down (shuts down, DDOS, loses it's data and backups, etc), ALL of the external links on Stack will be useless.

END TL;DR


I know answers should describe what a link contains in case the link goes down, but this isn't always feasible,

When is it not feasible?

Just copy paste the data. Or if there is too much to copy and paste, then an answer should be written which answers the question, with or without copied data from external resource, and then a link to the external resource. Then the answer holds up on its own and the link is just "additional" data.

If you cannot answer a question without linking to another site then you cannot answer the question as required by Stack's Q&A format.
And any question which can only be answered with a link is perhaps not fit for Stack.

Stack sites being Q&A the answers to questions need to be on the sites in the same place, in order to create a resource here on Stack.
And Stack is not a "resource finder" and users are supposed to get their answer here on Stack - not come to Stack to be pointed in the right direction.

If an answer is just a link somewhere, the question was requiring an external resource, and so the question cannot be answered "on Stack" and is most likely off-topic.

That aside, not being able to answer without a link is such a rare scenario that it's not worth a load of effort, especially as it's highly likely that users will then abuse it - using it for many link only answers thinking this is now ok and not as per your proposal to be used only on edge cases.


but any visitor who has a similar question, which could be answered by the linked source, won't find their answer

I know it can be useful at times, and I have myself had some great info from links, often better or a more thorough explanation than the answer.

But, while external links are very useful, we just have to accept that sometimes they can be broken, and I don't think using an external resource like archive.org is a solution.

We're not really supposed to be building up a link farm to external info, we're supposed to be providing answers to questions right here :)

Additionally:

If the Question you are reading does not answer "your question" then you might have a different question requiring different answers.

If you require an external resource from an answer's link to answer your question, it signifies that either:

  1. The Question you are reading is not answered sufficiently (possibly the linked data should be in the answer)
  2. You have a different question altogether
  3. The answer is fine and answers the question, but the link is just very useful as additional info

(1) Needs to be resolved by moderation.
(2) If it's possible, and not a dupe, a new Question being asked will be more specific and will have answers which are then searchable and will help others with the same question, and ruling out the need for that external resource.
(3) Sometimes Answers are fine, but don't answer our question, but asking a new one would be a dupe. That is frustrating, but for that we have bounties and comments asking for more info.

  • For copy-paste please make sure that the license of that page allows it (i.e. it is compatible with the license of Stack Exchange, CC-BY-SA). – Paŭlo Ebermann Jun 29 '15 at 17:36
  • @PaŭloEbermann Of course :) (I didn't mention that as it's outside the scope of this discussion) – James Jun 29 '15 at 17:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .