What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

Please modify http://cdn.sstatic.net/robots.txt to allow the Internet Archive's crawler, ia_archiver.

Pages that were archived a few years ago include their stylesheets and appear correctly:

Pages that were archived more recently do not include their styles -- robots.txt does not allow it.

Because it doesn't use the CDN, MSO is still archived correctly:

Related: Jeff's post suggesting that people use Archive.org to archive posts.

share|improve this question

1 Answer 1

up vote 9 down vote accepted

We have removed the restrictions via robots.txt from http://cdn.sstatic.net which should allow crawlers to access the js.

share|improve this answer
    
Thanks. It seems to have the old file cached for now. Hopefully that won't last too long. –  Jeremy Banks Jul 21 '12 at 4:23
1  
@jeremy, just in case you expect differently, I doubt allowing it now will also fix things for older pages? (The Internet Archive cannot tell what was the CDN content at the time the older pages were archived, but I guess you understand.) –  Arjan Jul 21 '12 at 9:55
    
Ah, @jeremy, maybe your report actually will avoid bad content being archived, though not for a past date: "The Wayback Machine hasn't archived a capture for that URL. Here's a capture taken 0 minutes ago from the live web that will become part of the permanent archive in the next few months." Nice. –  Arjan Jul 21 '12 at 9:59
    
@Arjan The archive sometimes matches up links from different years, so it's not implausible that they might apply the current stylesheet to the older pages. I'm not sure, though, and at any rate it probably wouldn't happen until the current version is in the permanent archive. The example post was temporarily undeleted to try this out, and it's now been re-deleted, so it may not be the simplest case to test with. –  Jeremy Banks Jul 21 '12 at 18:44
    
    
How about http://i.stack.imgur.com/robots.txt? That's more stable, but still not as permanent as the Archive. –  Jeremy Banks Jul 23 '12 at 1:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .