19

On Stack Exchange, as with any web site containing external links, there's a problem known as link rot. This is the phenomenon whereby some URLs no longer point to the information that they once pointed to. Sometimes, the domain has been disconnected. Perhaps it's under new ownership, and the original deep links no longer work. Or, frequently, the website is reorganized and content is in a new location.

To deal with this problem, the team is currently working on a soon-to-be-revealed /review/broken-links tool, and errors are issued at post submission time:

enter image description here

That's a nice idea, but the implementation is flawed. In the attempt to deal with this problem, another is being created: You can no longer post links to sites which are not valid as imagined by the author of the broken link detector without the post showing up in /review and/or getting the above error.

There are several cases when 'broken' links are currently disallowed but they should be allowed:

  1. http://localhost and friends. Usually, this is not an issue wherein someone is attempting to link to attempts to link to a public site like http://localhosting.com/ but accidentally types this invalid domain. It's usually (based on the first 10 links in /review) an attempt to link to a local domain for debugging or modifying a server running at that location. Sure, it won't work for the average Internet user (or the link validator bot), but these links should work for someone trying to reproduce the problem described in a question or follow along with an answer.

  2. As described in Are IP address links valid in posts?, "the vast majority of these links are broken and/or link nowhere." Note that this implies that there's a minority which are intentional. Perhaps someone was attempting to write

    Now that you've flashed dd-wrt to your router, navigate to 192.168.1.1 by clicking on that IP address and you should see the web GUI where you can...

    or

    I've written a patch for the omnibar logic for Chromium to allow arithmetic calculations to be performed, but when I click on an IP address like 74.125.224.240 which should direct you to the google.com homepage, some users report that it tries to compute something instead. Apply the patch from here on Github, does it work for you? If not, what is your configuration (extensions/userscripts etc) and why doesn't the patch work?

    Posts like these should be allowed to link to IPs, and not have to wrap them in code backticks or blocks. Of course, it's impossible to determine algorithmically whether this is the case, so the post should go through with a warning like:

    Oops! You're linking to the IP address blah.blah.blah.blah. These links break quickly and may not work in the future.

    [ Whoops, please fetch the host name for this IP and fix my post ]
    [ No, I meant to link to an IP address ]

  3. The link validator checks for redirects, and reports a broken link if a redirect is detected. As described at this Webmasters question, the vaildator assumes that 301 and 302 redirects won't work indefinitely and should always be changed to the new address. (By the way, wouldn't it be nice if the /review tool could fetch the destination page and suggest an edit?). I have no objection to this for many links. However, some links will always contain a redirect: Consider http://example.com, which redirects to http://www.iana.org/domains/example/ and probably can be assumed to do so for the forseeable future. A link to a page intended to demonstrate redirection should occasionally be allowed. Similarly, a link to a page intended to demonstrate a 404 should be allowed, though a warning should be issued.

  4. There are probably other use cases for links which don't report 200 OK to the link validator script. I can't guess at all of them, and neither can any other developer.

I do want to see fewer broken links on the network. I agree that the current tools will catch many broken outbound links that should be changed. However, I take issue with the fact that the current implementation handles any non-conforming links as impassable errors. They should be warnings, and the warnings should be dismissable.

||improve this question
  • 2
    I hope this won't kick in for mods doing stuff like converting answers to question edits. That would be a serious friction point. – Kev Jun 1 '12 at 16:58
  • 1
    I am tempted to close this as not constructive, 3 is a plain lie, 2 is already handled in another question, 1 and 4 are separate questions – waffles Jun 1 '12 at 23:31
  • 2
    @waffles when you say "3 is a plain lie," do you mean that you won't force people not to use 301's and 302's, or do you mean "I disagree that there's any reason you would ever possibly want to link to a redirecting page, so I'm going to call it a lie"? – nhinkle Jun 2 '12 at 2:58
  • @nhinkle I never even considered 302s to be broken in any way shape or form. – waffles Jun 2 '12 at 5:11
  • @waffles thanks for explaining. Looks like Kevin misinterpreted a post by Shog as a policy decision for this feature. – nhinkle Jun 2 '12 at 5:24
4

This rubs me off in the wrong way, I know the intentions are good but the feature is unfinished, unannounced and buggy. That is why the URL is not public and now I may be forced to 404 it for you.

Single word domains like http://localhost and http://waffles should be white listed.

Initially I disagreed with this as these links are broken for 99.999% of the users, but on further reflection the enormity of task that is dealing with legitimately rotten links should not be distracted by this issue of purity. I recently changed it so we do not even look at these in the review page. We may or may not deal with this in the markdown editor. Suggest we broach that in a separate question that only deals with the markdown aspect.

Are any IP address links valid in posts?

Well there is a separate question for that. Are IP address links valid in posts?

Why do you consider 302/301 and 307 BROKEN?

I don't https://meta.stackexchange.com/a/134166/17174

There are theoretic problems that may pop up when you develop the feature, which you can not anticipate?

Therefore what? I have 100 theoretic problems, if they worry me too much I will not be able to get any real work done.

Implied ... is there going to be any way to deal with edge cases, where the link bot fails

Yes

||improve this answer
  • All you needed to say: Hey, this is a work in progress, stop bothering me. And if I were in your shoes, I'd probably wouldn't even bother saying that. – yannis Jun 2 '12 at 5:20
  • 2
    Kevin should've probably known better... – Lorem Ipsum Jun 2 '12 at 5:43
  • 1
    My apologies! I was simply trying to help refine it before it became a public feature. – Kevin Vermeer Jun 2 '12 at 12:16
  • localhost and 192.168.1.1 should both be clickable links. – endolith Jun 6 '12 at 21:49
3

http://localhost should work if you surround it with backticks, like I've done here.

I got the error message when I tried posting this answer without backticks. Escaping with backticks makes sense here, since there is nothing to actually link to.

Links with localhost in them should probably be white-listed, since we don't need to see broken link reports on those.

||improve this answer
  • 4
    Does not work. When I click on the text in your answer, it doesn't take me to http://localhost. I understand that you can post the text after surrounding it with backticks, but there most certainly is something to link to in many cases, such as these 5000+ questions and answers containing url:*localhost*. If a post author and a post reader working on the same problem, and have a development server set up to serve pages from localhost, these links are meaningful to us. – Kevin Vermeer Jun 1 '12 at 16:50
  • 4
    I would consider local links to be outside the scope of the SE software. We don't get to link to My Documents either. – user102937 Jun 1 '12 at 16:51
  • 5
    I would consider local links to be firmly within the scope of the SE software. In fact, they're probably more appropriate on sites like Stack Overflow, Electrical Engineering, and Webmasters than anywhere else on the web! – Kevin Vermeer Jun 1 '12 at 17:31
  • 5
    @RobertHarvey: They are absolutely appropriate on SE, and should actually be clickable, for that matter. What's the argument against this? – endolith Jun 1 '12 at 18:41
  • @endolith: The fact that they just happened to work before raises the expectation that they should work now. I'm pretty sure SE didn't sign up for this. It's a "nice to have," but I don't see how it advances the SE mission, especially if it hinders their ability to audit links. – user102937 Jun 1 '12 at 18:43
  • 4
    @RobertHarvey - It advances the SE mission by making asking and answering questions easier, clearer, and more useful. It doesn't hinder their ability to audit links, they're going out of their way to find these links and stop them. – Kevin Vermeer Jun 1 '12 at 18:56
  • You need local links. Seriously. – user102937 Jun 1 '12 at 18:57
  • Also, I now see your comment about whitelisting localhost. I'm against a whitelist because you cannot know everything that it should contain. User judgement is a better mechanism for determining what's valid than developer prognostication. – Kevin Vermeer Jun 1 '12 at 18:58
  • 4
    @RobertHarvey - Yes, I need local links. I'm developing a public bootloader for an embedded system which will be accessible via a web browser at a local address - there are several similar models in common use - and I suck at setting up webserver software so I need to test it on localhost. Also, not on SE but I use an in-house embedded system at work every day which resides at 10.1.1.1 (there are useful links all over our internal wiki). – Kevin Vermeer Jun 1 '12 at 19:03

Not the answer you're looking for? Browse other questions tagged .