20

There seems to be some kind of trouble with both the main, SO and SE meta chats - All three servers are

pathping seems to indicate something's fishy on the NY side of things. Could y'all take a look please?

Tracing route to chat.stackexchange.com [198.252.206.29]
over a maximum of 30 hops:

  1     4 ms     6 ms     4 ms  HECKATE_ROUTER [192.168.2.1]
  2     4 ms     4 ms     4 ms  192.168.1.254
  3     6 ms     8 ms    40 ms  REDACTED
  4    10 ms     6 ms     6 ms  202.166.123.134
  5     6 ms     6 ms     6 ms  202.166.123.133
  6     6 ms     6 ms     6 ms  ae8-0.tp-cr03.singnet.com.sg [202.166.122.50]
  7     6 ms     6 ms     6 ms  ae4-0.tp-er03.singnet.com.sg [202.166.123.70]
  8     6 ms     6 ms     6 ms  203.208.191.197
  9   187 ms   184 ms   183 ms  203.208.158.126
 10   183 ms   182 ms   183 ms  30gigabitethernet2-1.core1.pao1.he.net [198.32.176.20]
 11   192 ms   192 ms   191 ms  10ge7-5.core1.sjc2.he.net [72.52.92.70]
 12   258 ms   255 ms   255 ms  100ge10-2.core1.nyc4.he.net [184.105.81.217]
 13   245 ms   247 ms   248 ms  100ge11-1.core1.nyc5.he.net [184.105.213.218]
 14   316 ms   310 ms   310 ms  lightower-fiber-networks.10gigabitethernet3-2.core1.nyc5.he.net [216.66.50.106]
 15     *        *        *     Request timed out.
 16     *        *        *     Request timed out.
 17     *        *        *     Request timed out.
 18  ^C
C:\Users\faile>tracert chat.meta.stackexchange.com

Tracing route to chat.stackexchange.com [198.252.206.29]
over a maximum of 30 hops:

  1     3 ms     4 ms     4 ms  heckate_router [192.168.2.1]
  2     5 ms     4 ms     5 ms  _gateway [192.168.1.254]
  3     6 ms     6 ms     5 ms  REDACTED
  4     9 ms     6 ms    10 ms  202.166.123.134
  5     6 ms     6 ms     6 ms  202.166.123.133
  6     7 ms     6 ms     6 ms  ae8-0.tp-cr03.singnet.com.sg [202.166.122.50]
  7    10 ms     6 ms     6 ms  ae4-0.tp-er03.singnet.com.sg [202.166.123.70]
  8     6 ms     6 ms     6 ms  203.208.191.197
  9   183 ms   183 ms   184 ms  203.208.158.126
 10   183 ms     *      183 ms  30gigabitethernet2-1.core1.pao1.he.net [198.32.176.20]
 11   192 ms   191 ms   192 ms  10ge7-5.core1.sjc2.he.net [72.52.92.70]
 12   264 ms   255 ms   255 ms  100ge10-2.core1.nyc4.he.net [184.105.81.217]
 13   245 ms   245 ms   245 ms  100ge11-1.core1.nyc5.he.net [184.105.213.218]
 14   314 ms   316 ms   314 ms  lightower-fiber-networks.10gigabitethernet3-2.core1.nyc5.he.net [216.66.50.106]
 15     *        *        *     Request timed out.
  • For me it was down (an hour ago) for a while on WiFi, but at that time not on 4G. Right now it's working. – Glorfindel May 14 at 8:20
  • No repro. It's working for me. – Sonic the Bracketed Hedgehog May 14 at 8:22
  • I lost connection to these sites some 20 minutes ago. I'm on WiFi, with no option to switch. FWIW chat.se and chat.so both resolve to 198.252.206.29 for me. – tripleee May 14 at 8:22
  • Same here, chat doesn't work. Moscow, Russia. – Suvitruf May 14 at 8:24
  • 1
    down for me as well, from Germany. – Mad Scientist May 14 at 8:26
  • 1
    It's not just you! – αλεχολυτ May 14 at 8:28
  • Turns to Ask Ubuntu with flags – iBug May 14 at 8:29
  • I've given it a shot on 3 different ISPs here. Interestingly I can do a traceroute off my server in france... so looks like the SREs are going to have doubleplus fun fixing this ._. – Journeyman Geek May 14 at 8:30
  • Looks more like DNS issue than server issue, since it's working in some parts of the world (USA) and not in others. – Shadow The Curly Braced Wizard May 14 at 8:31
  • They're resolving to the same place so DNS is likely to be fine. I posted the traceroute because its the best bet for the SREs to work out at precisely which step things are borked. – Journeyman Geek May 14 at 8:33
  • 4
    We're aware of an issue in some parts of the world and are investigating now. – Dean Ward May 14 at 8:33
  • Up for me, from Germany @MadScientist. – MEE was the missing bracket May 14 at 8:35
  • Doesn't even connect for my two servers in United States (DigitalOcean, US West), (AWS, US East 2, Ohio). Additionally, 3 of my proxy servers can't connect as well (US East, Japan and Hong Kong). – iBug May 14 at 8:53
  • 3
    Seems back up for me – Journeyman Geek May 14 at 8:57
5

One of our ISP links to the NY data center was flapping which meant any traffic in or out of the data center to certain parts of the world was not being routed correctly. For now we've disabled that link and all traffic is being routed over our other ISPs

  • 1
    Does this also explain why we have lost hot meta posts on SO? – DavidG May 14 at 9:28

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .