# cdn.sstatic.net is slowing down initial page loads

I'm located in Sydney, Australia and on my first hit to any Stack Exchange site after I haven't been active for a period of time (say, an hour or so), I can see FireFox waiting on cdn.sstatic.net for a period between 3 and 5 seconds, but as high as 12 seconds.

Tracert to cdn.sstatic.net:

C:\Users\mark.henderson>tracert cdn.sstatic.net

Tracing route to sstatic.stackexchange.netdna-cdn.com [69.174.57.102]
over a maximum of 30 hops:

1    <1 ms     *       <1 ms  tmg.enets.local [192.168.161.17]
2    19 ms    18 ms    19 ms  lns20.syd6.internode.on.net [150.101.199.159]
3    19 ms    20 ms    18 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]

4   200 ms   200 ms   200 ms  gi6-0-0-112.bdr1.syd7.internode.on.net [150.101.
120.229]
5   201 ms   200 ms   199 ms  pos5-0.bdr1.sjc2.internode.on.net [203.16.213.16
2]
6   179 ms   191 ms   179 ms  xe2-0.cr01.sjc01.mzima.net [206.223.116.62]
7   178 ms   176 ms   179 ms  te0-1.cr1.lax1.us.packetexchange.net [69.174.120
.85]
8   209 ms   178 ms   177 ms  69.174.36.2
9   177 ms   178 ms   179 ms  69.174.57.102

Trace complete.

9 hops - that's gotta be one of the shortest trips to the US I've ever seen.

And

You are hitting the NetDNA Los Angeles Datacenter

Sorry guys, I don't wanna be rude, just wondering if there's been any updates on this issue? Is anyone else seeing the same behaviour? It's starting to get quite irritating, but of course if I'm the only one it's affecting then I'll start digging into our own network here.

This issue seems to be gone this week. I'll update this if it comes back.

I spoke too soon. The issue isn't gone, but it's now a lot less frequent and doesn't last quite as long. It now happens once or twice a day, and holds up for about 5 seconds.

Did you change something today (2011-07-13)? All of a sudden the site is back to its pre-CDN speed!

Nope, spoke to soon. Back to crappy waiting again.

Sorry to bring this up again, but it's recently got worse again. I was finally able to catch one in Firebug:

that was after waiting almost 30 seconds. Those requests that are still in progress never actually finish. I suspect maybe Firebug is not showing them correctly, because the site functions fine if you sit through the 16 second load time.

You are hitting the NetDNA Los Angeles Datacenter (that page took about 20 seconds to load)

C:\Users\mark.henderson>tracert sstatic.stackexchange.netdna-cdn.com

Tracing route to sstatic.stackexchange.netdna-cdn.com [67.201.31.70]
over a maximum of 30 hops:

1     *        *        *     Request timed out.
2   390 ms   354 ms   342 ms  lns20.syd6.internode.on.net [150.101.199.159]
3   374 ms   271 ms   198 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]
4   382 ms   368 ms   506 ms  gi3-1-0.bdr1.syd7.internode.on.net [150.101.120.225]
5   534 ms   543 ms   517 ms  pos2-0.bdr1.sjc2.internode.on.net [203.16.213.50]
6   624 ms   610 ms   575 ms  xe2-0.cr01.sjc01.mzima.net [206.223.116.62]
7   574 ms   538 ms   505 ms  te0-1.cr1.lax1.us.packetexchange.net [69.174.120.85]
8   582 ms   521 ms   501 ms  te2-0.cr2.lax1.us.packetexchange.net [69.174.120.142]
9   550 ms   529 ms   502 ms  69.174.36.2
10   448 ms   421 ms   424 ms  67.201.31.70

Trace complete.

C:\Users\mark.henderson>tracert sstatic.net

Tracing route to sstatic.net [64.34.119.12]
over a maximum of 30 hops:

1     *        *        *     Request timed out.
2   333 ms   344 ms   386 ms  lns20.syd6.internode.on.net [150.101.199.159]
3   397 ms   350 ms   344 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]
4   617 ms   605 ms   572 ms  gi3-1-0.bdr1.syd7.internode.on.net [150.101.120.225]
5   592 ms   533 ms   545 ms  pos2-0.bdr1.sjc2.internode.on.net [203.16.213.50]
6   510 ms   461 ms   483 ms  equinix-sjo.peer1.net [206.223.116.30]
7   477 ms   506 ms   536 ms  10ge.ten1-1.sj-mkp2-dis-1.peer1.net [216.187.89.101]
8   483 ms   536 ms   540 ms  10ge-ten1-3.la-600w-cor-1.peer1.net [216.187.88.130]
9   505 ms   513 ms   546 ms  10ge.ten1-1.la-600w-cor-2.peer1.net [216.187.88.146]
10   532 ms   530 ms   433 ms  10ge-ten1-2.dal-eqx-cor-1.peer1.net [216.187.124.122]
11   499 ms   419 ms   427 ms  10ge-ten1-1.dal-eqx-cor-2.peer1.net [216.187.124.134]
12   502 ms   512 ms   532 ms  10ge-ten2-1.atl-telx-cor-1.peer1.net [216.187.124.118]
13   467 ms   512 ms   507 ms  10ge-ten1-1.atl-101mar-cor-1.peer1.net [216.187.120.226]
14   506 ms   520 ms   493 ms  10ge.xe-1-0-0.wdc-eqx-dis-1.peer1.net [216.187.115.37]
15   489 ms   501 ms   501 ms  10ge.ten1-2.wdc-sp2-cor-1.peer1.net [216.187.115.234]
16   513 ms   506 ms   514 ms  216.187.120.254
17   482 ms   506 ms   503 ms  10ge.xe-2-0-0.nyc-telx-dis-1.peer1.net [216.187.115.221]
18   544 ms   521 ms   528 ms  10ge.xe-0-0-0.nyc-telx-dis-2.peer1.net [216.187.115.182]
19   491 ms   514 ms   519 ms  oc48-po3-0.nyc-75bre-dis-1.peer1.net [216.187.115.134]
20   549 ms   505 ms   518 ms  gwny01.stackoverflow.com [64.34.41.58]
21   559 ms   523 ms   520 ms  stackoverflow.com [64.34.119.12]

Trace complete.

-
Brr, gives new meaning to 'down under'. Looks like 7 days expiration to me though, look at Cache. –  Uphill Luge May 24 '11 at 5:37
@Hans - it seems to happen after about 30 minutes of inactivity –  Mark Henderson May 24 '11 at 5:44
Can you post a tracert to cdn.sstatic.net? And what datacenter does 67.201.31.56 tell you you're hitting? –  balpha May 24 '11 at 6:41
@balpha - all done –  Mark Henderson May 24 '11 at 8:38
So this is still happening? Seems weird with the very good tracert times. Is this happening on every request? E.g. if you request cdn.sstatic.net/js/full.js a few times, do you have those sucky values every time? –  balpha May 26 '11 at 6:34
@balpha - I got that sucky value the first time, but each time after that it was OK. Given that nobody else has brought up any CDN issues I'm suspecting it might be our office proxy screwing things up. I'll try and do a diagnostic to see how it goes. –  Mark Henderson May 26 '11 at 21:46
I have the same issue from Adelaide but hoping the new CDN plans will fix it. –  Alex Angas May 31 '11 at 0:36
Got the same issue here, somewhat infrequent as noted above. Grand Rapids, Michigan. –  Chris S Jul 13 '11 at 13:33
Looks like this problem are back, having trouble with cdn.sstatic.net yesterday and today –  Qwark Nov 22 '11 at 8:30
is this still an issue? –  Jeff Atwood Jan 24 '12 at 19:00
@JeffAtwood - yes, but it's not as bad as when I reported it above. I'm trying to do another firebug capture of it, but it looks like it's all coming out of local cache at the moment. –  Mark Henderson Jan 25 '12 at 2:51
Ok, cache cleared and off the work LAN and it's down to 500ms "waiting", which is acceptable enough (compared to 12 seconds!) –  Mark Henderson Jan 25 '12 at 3:02
@far I am not sure the asia CDN ever came online, unfortunately –  Jeff Atwood Jan 25 '12 at 6:27
You're not the only one with CDN issues: “waiting for cdn.sstatic.net” waits too much –  Pops Feb 29 '12 at 23:57
This is still the case. Several times a day I have very long wait times (>8s) for cdn.sstatic.net and stackoverflow has rendered without stylesheets three times today. I know SO puts a lot of effort in being as fast as possible so it's a waste that a CDN intermittently slows down requests to all SO sites... –  Ronald Wildenberg Jul 3 '12 at 12:22

First and most importantly, there is a status page at

## http://status.maxcdn.com/

Does it indicate any problems with the CDN network at this time?

Beyond that, here's a list of things that can help the support people at the CDN to debug issues like this:

1. Sample CDN file (that is having issues), e.g. http://cdn.sstatic.net/rpg/all.css?v=c9c55a2b82c5
3. A traceroute to the CDN, i.e. sstatic.stackexchange.netdna-cdn.com
4. A traceroute to the Origin, that is sstatic.net
5. Which datacenter you are hitting (go to http://debug-02.netdna-cdn.com/, it will tell you "You are hitting the NetDNA XYZ Datacenter")

So please include as much information as possible when reporting CDN problems.

If you're uncomfortable with posting some of this publicly here on meta, you can also email it to team@stackoverflow.com.

-
Is it still hosted by maxcdn? I think its changed to cloudflare now –  bikram990 Jul 25 '13 at 4:36

It looks like this might be an issue with both tcp timestamps and socket recycling being enabled on their servers:

So disabling TCP timestamps client side might be the workaround:

Intermittent Responses from CDN

I am going to research this further in the morning and see if our CDN provider might need to change their server settings. I am going to start with the following thread if either of you are curious: http://www.mail-archive.com/varnish-misc@projects.linpro.no/msg02899.html

Update:
Our CDN provider has changed these settings, so would be interested to hear if anyone still has this issue.

-
I have done this. I'll see how it goes. For the record, to do this on Windows, the following technet message helps: technet.microsoft.com/en-us/library/cc938205.aspx –  Mark Henderson Jul 18 '11 at 1:10
Hmm, I made this change on both my local machine and our gateway and I'm not seeing any difference yet, but I'm not sure if you need to restart the machines or not. I can't restart anything until tonight. –  Mark Henderson Jul 18 '11 at 1:46
@far I believe this is only relevant to the CDN's servers? –  Jeff Atwood Jul 18 '11 at 17:50
@Jeff - oh, right. I think I mis-understood the issue then ;) I'll wait and see what happens then. –  Mark Henderson Jul 18 '11 at 20:21
Yes, at this moment, I cannot load any static resources from the CDN servers; all Stack Exchange sites (that I've tried) load without stylesheets or other static objects. –  delfuego Jul 28 '11 at 15:09
Still happens to me, six months later, all the time. Browser freezes with message "Read cdn.sstatic.net" at the bottom (Firefox, Ubuntu 12.10, not proxy/firewall, etc). –  Pointy Feb 3 '13 at 16:54

By the way, I wrote a small script

to test with some confidence whether a site has the same problem (on linux caused by enabling the tcp_tw_recycle flag). It only tests first-level effects (does not parse the site and recursively test dependent connections).

Running it over the top 1000 web sites (wget http://www.google.com/adplanner/static/top1000/, and grep out host addresses) shows that > 3% of the web sites have the same problem. I'd expect second-level effects to be higher.

Sounds like timestamp rewriting in NATs would be a great idea.

-
are you seeing improvement now that the CDN adjusted the relevant settings? –  Jeff Atwood Jul 25 '11 at 0:10

I've got the same problem; disturbingly enough, also located in Sydney, Australia. Not using Internode, though, I'm using Optus 3G. Traceroute appears to be unhappy on this connection (ICMP blocking, as far as I can tell), so nothing useful there, but I regularly get "hung pages", and cdn.sstatic.net is always the culprit in my status bar. If I wait long enough, I get a page that's lacking in CSS and images, but otherwise appears to have all the content (indicating that the attempted load timed out). Hitting the IP for cdn.sstatic.net indicates I'm getting to the Los Angeles DC.

-
is this still an issue? –  Jeff Atwood Jan 24 '12 at 18:59

FYI, this is still a major issue; from my workplace (major cabinet-level federal agency), the static CDN is still pretty much inaccessible 90+% of the time.

-
Have you sent the requested details to the team? –  Arjan Aug 12 '11 at 14:45
Yes, twice at least. –  delfuego Aug 22 '11 at 18:42
And it is still a major issue. –  delfuego Aug 22 '11 at 18:53
If things are bad, then (as a workaround) beware that simply doing a regular reload will make most browsers request the resources they already cached, with a If-Modified-Since header. Most often the CDN will respond with a 304 Not Modified, but the browser still needs to await that. (This is true for both regular keyboard shortcuts such as F5 or Command-R, and for regular clicks on some reload button, without holding down any modifier key.) To avoid that: use links to reload a page, instead of using the browser's reload option. Like: click a question title, or click the SO logo. –  Arjan Aug 22 '11 at 19:45
Overall, it's not a critical issue -- because at work (major US Government agency network), I have trained myself to not click on any StackOverflow links at all. They're useless. It's sort of a shame, because there's a lot of great stuff we (all of us on this network) are missing... but until the CDN gets fixed, it's mostly inaccessible to us in any meaningful way. –  delfuego Aug 23 '11 at 15:57
That I don't get. Even if the CDN is not responding quickly, it will still eventually download the static content. The HTTP headers should then tell your browser to cache it. So, when just clicking without reloading, all should be fine. If clicking is useless too, then something else might be wrong? –  Arjan Aug 23 '11 at 16:36
That's just not what's happening -- I can say that in my testing over the past 72 hours, the static CDN never responds, so the content is never cached. –  delfuego Aug 25 '11 at 16:04
Okay, then I guess your work is blocking things... –  Arjan Aug 25 '11 at 16:49
No, that's also not happening; we've verified it all the way out to the edge routers and content filters. That was just in that 72-hour window that nothing loaded, but about 12 hours after that comment, I did see static content load for around an hour or two... but we're now back to not loading. This is fairly clearly an issue with the CDN. –  delfuego Aug 31 '11 at 15:53
is this still an issue? –  Jeff Atwood Jan 24 '12 at 18:59
I have to say I haven't noticed it for a few weeks -- I was away from work for a month for paternity leave, and since returning it hasn't reared its head. Was there a change made at your CDN? –  delfuego Jan 25 '12 at 20:18
Same for me, on a Navy installation. Browser hangs waiting for cdn.sstatic.net. I can get the site to come up (basically plain-text) by installing adblock-plus and blocking the cdn.sstatic.net domain. –  Nicholas Smith Feb 15 '12 at 4:14
@JeffAtwood I see it, and I'm not in Australia. I have exactly the behavior described. –  Pointy Feb 3 '13 at 16:53

if you are using chrome, then you can check this repo: StackoverflowCDN, it is a chrome extension to abort long time downloading the css/js files of sstatic cdn, instead load local one.

-