39

I'm located in Sydney, Australia and on my first hit to any Stack Exchange site after I haven't been active for a period of time (say, an hour or so), I can see FireFox waiting on cdn.sstatic.net for a period between 3 and 5 seconds, but as high as 12 seconds.

enter image description here enter image description here

Tracert to cdn.sstatic.net:

C:\Users\mark.henderson>tracert cdn.sstatic.net

Tracing route to sstatic.stackexchange.netdna-cdn.com [69.174.57.102]
over a maximum of 30 hops:

  1    <1 ms     *       <1 ms  tmg.enets.local [192.168.161.17]
  2    19 ms    18 ms    19 ms  lns20.syd6.internode.on.net [150.101.199.159]
  3    19 ms    20 ms    18 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]

  4   200 ms   200 ms   200 ms  gi6-0-0-112.bdr1.syd7.internode.on.net [150.101.
120.229]
  5   201 ms   200 ms   199 ms  pos5-0.bdr1.sjc2.internode.on.net [203.16.213.16
2]
  6   179 ms   191 ms   179 ms  xe2-0.cr01.sjc01.mzima.net [206.223.116.62]
  7   178 ms   176 ms   179 ms  te0-1.cr1.lax1.us.packetexchange.net [69.174.120
.85]
  8   209 ms   178 ms   177 ms  69.174.36.2
  9   177 ms   178 ms   179 ms  69.174.57.102

Trace complete.

9 hops - that's gotta be one of the shortest trips to the US I've ever seen.

And

You are hitting the NetDNA Los Angeles Datacenter


Sorry guys, I don't wanna be rude, just wondering if there's been any updates on this issue? Is anyone else seeing the same behaviour? It's starting to get quite irritating, but of course if I'm the only one it's affecting then I'll start digging into our own network here.


This issue seems to be gone this week. I'll update this if it comes back.


I spoke too soon. The issue isn't gone, but it's now a lot less frequent and doesn't last quite as long. It now happens once or twice a day, and holds up for about 5 seconds.


Did you change something today (2011-07-13)? All of a sudden the site is back to its pre-CDN speed!

Nope, spoke to soon. Back to crappy waiting again.


Sorry to bring this up again, but it's recently got worse again. I was finally able to catch one in Firebug:

enter image description here

that was after waiting almost 30 seconds. Those requests that are still in progress never actually finish. I suspect maybe Firebug is not showing them correctly, because the site functions fine if you sit through the 16 second load time.

You are hitting the NetDNA Los Angeles Datacenter (that page took about 20 seconds to load)

C:\Users\mark.henderson>tracert sstatic.stackexchange.netdna-cdn.com

Tracing route to sstatic.stackexchange.netdna-cdn.com [67.201.31.70]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2   390 ms   354 ms   342 ms  lns20.syd6.internode.on.net [150.101.199.159]
  3   374 ms   271 ms   198 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]
  4   382 ms   368 ms   506 ms  gi3-1-0.bdr1.syd7.internode.on.net [150.101.120.225]
  5   534 ms   543 ms   517 ms  pos2-0.bdr1.sjc2.internode.on.net [203.16.213.50]
  6   624 ms   610 ms   575 ms  xe2-0.cr01.sjc01.mzima.net [206.223.116.62]
  7   574 ms   538 ms   505 ms  te0-1.cr1.lax1.us.packetexchange.net [69.174.120.85]
  8   582 ms   521 ms   501 ms  te2-0.cr2.lax1.us.packetexchange.net [69.174.120.142]
  9   550 ms   529 ms   502 ms  69.174.36.2
 10   448 ms   421 ms   424 ms  67.201.31.70

Trace complete.

C:\Users\mark.henderson>tracert sstatic.net

Tracing route to sstatic.net [64.34.119.12]
over a maximum of 30 hops:

  1     *        *        *     Request timed out.
  2   333 ms   344 ms   386 ms  lns20.syd6.internode.on.net [150.101.199.159]
  3   397 ms   350 ms   344 ms  te3-3.cor2.syd6.internode.on.net [150.101.195.9]
  4   617 ms   605 ms   572 ms  gi3-1-0.bdr1.syd7.internode.on.net [150.101.120.225]
  5   592 ms   533 ms   545 ms  pos2-0.bdr1.sjc2.internode.on.net [203.16.213.50]
  6   510 ms   461 ms   483 ms  equinix-sjo.peer1.net [206.223.116.30]
  7   477 ms   506 ms   536 ms  10ge.ten1-1.sj-mkp2-dis-1.peer1.net [216.187.89.101]
  8   483 ms   536 ms   540 ms  10ge-ten1-3.la-600w-cor-1.peer1.net [216.187.88.130]
  9   505 ms   513 ms   546 ms  10ge.ten1-1.la-600w-cor-2.peer1.net [216.187.88.146]
 10   532 ms   530 ms   433 ms  10ge-ten1-2.dal-eqx-cor-1.peer1.net [216.187.124.122]
 11   499 ms   419 ms   427 ms  10ge-ten1-1.dal-eqx-cor-2.peer1.net [216.187.124.134]
 12   502 ms   512 ms   532 ms  10ge-ten2-1.atl-telx-cor-1.peer1.net [216.187.124.118]
 13   467 ms   512 ms   507 ms  10ge-ten1-1.atl-101mar-cor-1.peer1.net [216.187.120.226]
 14   506 ms   520 ms   493 ms  10ge.xe-1-0-0.wdc-eqx-dis-1.peer1.net [216.187.115.37]
 15   489 ms   501 ms   501 ms  10ge.ten1-2.wdc-sp2-cor-1.peer1.net [216.187.115.234]
 16   513 ms   506 ms   514 ms  216.187.120.254
 17   482 ms   506 ms   503 ms  10ge.xe-2-0-0.nyc-telx-dis-1.peer1.net [216.187.115.221]
 18   544 ms   521 ms   528 ms  10ge.xe-0-0-0.nyc-telx-dis-2.peer1.net [216.187.115.182]
 19   491 ms   514 ms   519 ms  oc48-po3-0.nyc-75bre-dis-1.peer1.net [216.187.115.134]
 20   549 ms   505 ms   518 ms  gwny01.stackoverflow.com [64.34.41.58]
 21   559 ms   523 ms   520 ms  stackoverflow.com [64.34.119.12]

Trace complete.
18
  • 1
    Brr, gives new meaning to 'down under'. Looks like 7 days expiration to me though, look at Cache. May 24, 2011 at 5:37
  • @Hans - it seems to happen after about 30 minutes of inactivity May 24, 2011 at 5:44
  • Can you post a tracert to cdn.sstatic.net? And what datacenter does 67.201.31.56 tell you you're hitting?
    – balpha StaffMod
    May 24, 2011 at 6:41
  • 1
    @balpha - all done May 24, 2011 at 8:38
  • So this is still happening? Seems weird with the very good tracert times. Is this happening on every request? E.g. if you request cdn.sstatic.net/js/full.js a few times, do you have those sucky values every time?
    – balpha StaffMod
    May 26, 2011 at 6:34
  • @balpha - I got that sucky value the first time, but each time after that it was OK. Given that nobody else has brought up any CDN issues I'm suspecting it might be our office proxy screwing things up. I'll try and do a diagnostic to see how it goes. May 26, 2011 at 21:46
  • 1
    I have the same issue from Adelaide but hoping the new CDN plans will fix it.
    – Alex Angas
    May 31, 2011 at 0:36
  • Got the same issue here, somewhat infrequent as noted above. Grand Rapids, Michigan.
    – Chris S
    Jul 13, 2011 at 13:33
  • Looks like this problem are back, having trouble with cdn.sstatic.net yesterday and today
    – Qwark
    Nov 22, 2011 at 8:30
  • is this still an issue? Jan 24, 2012 at 19:00
  • @JeffAtwood - yes, but it's not as bad as when I reported it above. I'm trying to do another firebug capture of it, but it looks like it's all coming out of local cache at the moment. Jan 25, 2012 at 2:51
  • Ok, cache cleared and off the work LAN and it's down to 500ms "waiting", which is acceptable enough (compared to 12 seconds!) Jan 25, 2012 at 3:02
  • @far I am not sure the asia CDN ever came online, unfortunately Jan 25, 2012 at 6:27
  • You're not the only one with CDN issues: “waiting for cdn.sstatic.net” waits too much
    – Pops
    Feb 29, 2012 at 23:57
  • 1
    This is still the case. Several times a day I have very long wait times (>8s) for cdn.sstatic.net and stackoverflow has rendered without stylesheets three times today. I know SO puts a lot of effort in being as fast as possible so it's a waste that a CDN intermittently slows down requests to all SO sites... Jul 3, 2012 at 12:22

7 Answers 7

8

First and most importantly, there is a status page at

http://status.maxcdn.com/

Does it indicate any problems with the CDN network at this time?

Beyond that, here's a list of things that can help the support people at the CDN to debug issues like this:

  1. Sample CDN file (that is having issues), e.g. http://cdn.sstatic.net/rpg/all.css?v=c9c55a2b82c5
  2. Your IP address
  3. A traceroute to the CDN, i.e. sstatic.stackexchange.netdna-cdn.com
  4. A traceroute to the Origin, that is sstatic.net
  5. Which datacenter you are hitting (go to http://debug-02.netdna-cdn.com/, it will tell you "You are hitting the NetDNA XYZ Datacenter")

So please include as much information as possible when reporting CDN problems.

If you're uncomfortable with posting some of this publicly here on meta, you can also email it to [email protected].

1
  • Is it still hosted by maxcdn? I think its changed to cloudflare now
    – bikram990
    Jul 25, 2013 at 4:36
7

It looks like this might be an issue with both tcp timestamps and socket recycling being enabled on their servers:

So disabling TCP timestamps client side might be the workaround:

Intermittent Responses from CDN

I am going to research this further in the morning and see if our CDN provider might need to change their server settings. I am going to start with the following thread if either of you are curious: http://www.mail-archive.com/[email protected]/msg02899.html

Update:
Our CDN provider has changed these settings, so would be interested to hear if anyone still has this issue.

6
  • I have done this. I'll see how it goes. For the record, to do this on Windows, the following technet message helps: technet.microsoft.com/en-us/library/cc938205.aspx Jul 18, 2011 at 1:10
  • Hmm, I made this change on both my local machine and our gateway and I'm not seeing any difference yet, but I'm not sure if you need to restart the machines or not. I can't restart anything until tonight. Jul 18, 2011 at 1:46
  • @far I believe this is only relevant to the CDN's servers? Jul 18, 2011 at 17:50
  • @Jeff - oh, right. I think I mis-understood the issue then ;) I'll wait and see what happens then. Jul 18, 2011 at 20:21
  • Yes, at this moment, I cannot load any static resources from the CDN servers; all Stack Exchange sites (that I've tried) load without stylesheets or other static objects.
    – delfuego
    Jul 28, 2011 at 15:09
  • Still happens to me, six months later, all the time. Browser freezes with message "Read cdn.sstatic.net" at the bottom (Firefox, Ubuntu 12.10, not proxy/firewall, etc).
    – Pointy
    Feb 3, 2013 at 16:54
4

By the way, I wrote a small script

http://codepad.org/cDbTXbBp

to test with some confidence whether a site has the same problem (on linux caused by enabling the tcp_tw_recycle flag). It only tests first-level effects (does not parse the site and recursively test dependent connections).

Running it over the top 1000 web sites (wget http://www.google.com/adplanner/static/top1000/, and grep out host addresses) shows that > 3% of the web sites have the same problem. I'd expect second-level effects to be higher.

Sounds like timestamp rewriting in NATs would be a great idea.

1
  • are you seeing improvement now that the CDN adjusted the relevant settings? Jul 25, 2011 at 0:10
2

I've got the same problem; disturbingly enough, also located in Sydney, Australia. Not using Internode, though, I'm using Optus 3G. Traceroute appears to be unhappy on this connection (ICMP blocking, as far as I can tell), so nothing useful there, but I regularly get "hung pages", and cdn.sstatic.net is always the culprit in my status bar. If I wait long enough, I get a page that's lacking in CSS and images, but otherwise appears to have all the content (indicating that the attempted load timed out). Hitting the IP for cdn.sstatic.net indicates I'm getting to the Los Angeles DC.

1
  • is this still an issue? Jan 24, 2012 at 18:59
2

FYI, this is still a major issue; from my workplace (major cabinet-level federal agency), the static CDN is still pretty much inaccessible 90+% of the time.

13
  • 1
    Have you sent the requested details to the team?
    – Arjan
    Aug 12, 2011 at 14:45
  • Yes, twice at least.
    – delfuego
    Aug 22, 2011 at 18:42
  • And it is still a major issue.
    – delfuego
    Aug 22, 2011 at 18:53
  • If things are bad, then (as a workaround) beware that simply doing a regular reload will make most browsers request the resources they already cached, with a If-Modified-Since header. Most often the CDN will respond with a 304 Not Modified, but the browser still needs to await that. (This is true for both regular keyboard shortcuts such as F5 or Command-R, and for regular clicks on some reload button, without holding down any modifier key.) To avoid that: use links to reload a page, instead of using the browser's reload option. Like: click a question title, or click the SO logo.
    – Arjan
    Aug 22, 2011 at 19:45
  • 1
    Overall, it's not a critical issue -- because at work (major US Government agency network), I have trained myself to not click on any StackOverflow links at all. They're useless. It's sort of a shame, because there's a lot of great stuff we (all of us on this network) are missing... but until the CDN gets fixed, it's mostly inaccessible to us in any meaningful way.
    – delfuego
    Aug 23, 2011 at 15:57
  • That I don't get. Even if the CDN is not responding quickly, it will still eventually download the static content. The HTTP headers should then tell your browser to cache it. So, when just clicking without reloading, all should be fine. If clicking is useless too, then something else might be wrong?
    – Arjan
    Aug 23, 2011 at 16:36
  • 1
    That's just not what's happening -- I can say that in my testing over the past 72 hours, the static CDN never responds, so the content is never cached.
    – delfuego
    Aug 25, 2011 at 16:04
  • Okay, then I guess your work is blocking things...
    – Arjan
    Aug 25, 2011 at 16:49
  • No, that's also not happening; we've verified it all the way out to the edge routers and content filters. That was just in that 72-hour window that nothing loaded, but about 12 hours after that comment, I did see static content load for around an hour or two... but we're now back to not loading. This is fairly clearly an issue with the CDN.
    – delfuego
    Aug 31, 2011 at 15:53
  • is this still an issue? Jan 24, 2012 at 18:59
  • I have to say I haven't noticed it for a few weeks -- I was away from work for a month for paternity leave, and since returning it hasn't reared its head. Was there a change made at your CDN?
    – delfuego
    Jan 25, 2012 at 20:18
  • Same for me, on a Navy installation. Browser hangs waiting for cdn.sstatic.net. I can get the site to come up (basically plain-text) by installing adblock-plus and blocking the cdn.sstatic.net domain. Feb 15, 2012 at 4:14
  • @JeffAtwood I see it, and I'm not in Australia. I have exactly the behavior described.
    – Pointy
    Feb 3, 2013 at 16:53
1

if you are using chrome, then you can check this repo: StackoverflowCDN, it is a chrome extension to abort long time downloading the css/js files of sstatic cdn, instead load local one.

0

I had this problem.

After digging into my firewall, I discovered that the IPv4 addresses for cdn.sstatic.net were on my bogons list. My firewall's bogon-blocking list had become out-of-date.

So, update your blocked-bogons list, or turn off bogon filtering in your firewall, and cdn.sstatic.net becomes reachable.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .