What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

The Verizon techs have the link to this post, so please add any information you can here. all additional info is helpful at this point

Chat seems to be down (requests time out). Kyle, you broke stuff again!!

Unless my link to the tubes is malfunctioning... Either way, I'm having withdrawals and it's not pretty.

And really, I was going to leave it at that, but apparently that doesn't "meet our quality standards." What does that even mean?


This is affecting all services run out of our OR DC. If you are on Verizon FiOS on the East coast you'll have issues getting to:

  • chat
  • data.se
  • blog.so

-Zypher


I've turned this question community wiki. Anyone having issue please do a trace route to chat.stackexchange.com and add the IP/Reverse dns of the router that it dies at below.

Known bad routers:

g10-0-8-112.hrbgpa-lcr-01.verizon-gni.net [130.81.184.148]
G0-10-1-1.PHLAPA-LCR-22.verizon-gni.net [130.81.183.18]
g11-0-5-1026.nycmny-lcr-10.verizon-gni.net [130.81.180.4]
L100.WASHDC-VFTTP-125.verizon-gni.net [173.66.170.1]
G0-13-1-5.NYCMNY-LCR-22.verizon-gni.net [130.81.182.78]
g0-2-5-5.nrflva-lcr-21.verizon-gni.net [130.81.140.228]
G0-5-3-1.PHLAPA-LCR-21.verizon-gni.net [130.81.193.192]
G14-0-1-1630.WASHDC-LCR-06.verizon-gni.net (130.81.146.120)

Traceroute output if ya got it:

  traceroute to chat.stackexchange.com (69.59.196.211), 30 hops max, 40 byte packets
1  Wireless_Broadband_Router.home (192.168.1.1)  1.735 ms  0.606 ms  0.545 ms
2  L100.WASHDC-VFTTP-130.verizon-gni.net (96.231.130.1)  7.286 ms  6.926 ms  7.064 ms
3  G14-0-1-1630.WASHDC-LCR-06.verizon-gni.net (130.81.146.120)  8.613 ms !H *  9.221 ms !H

C:\Users\gbeech>tracert chat.stackexchange.com

Tracing route to chat.stackexchange.com [69.59.196.211]
over a maximum of 30 hops:

  1     8 ms    <1 ms    <1 ms  10.10.10.1
  2     4 ms     4 ms     3 ms  l300.nycmny-vfttp-226.verizon-gni.net [173.56.120.1]
  3  g11-0-5-1026.nycmny-lcr-10.verizon-gni.net [130.81.180.4]  reports: Destination host unreachable.

Trace complete.

Tracing route to chat.stackexchange.com [69.59.196.211]
over a maximum of 30 hops:

  1     6 ms     5 ms     6 ms  Wireless_Broadband_Router.home [192.168.1.1]
  2    77 ms    53 ms    32 ms  L100.WASHDC-VFTTP-125.verizon-gni.net [173.66.17
0.1]
  3     *        *        *     Request timed out.
  4  69.59.196.211  reports: Destination net unreachable.

Trace complete.
share|improve this question

closed as too localized by Jeff Atwood Jul 1 '11 at 19:09

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

    
I'm having issues too, but the problem seems to be somewhere in the middle. Connecting from my iPhone over 3G works fine, but no luck with my home connection. Good to know it's not just me. –  Tim Stone Jul 1 '11 at 1:02
3  
I guess it's working for the people downvoting this... –  Holocryptic Jul 1 '11 at 1:07
    
Confirming that it is broken. We're looking into it. –  Michael Pryor Jul 1 '11 at 1:15
    
If you do a trace route does it die at: g11-0-5-1026.nycmny-lcr-10.verizon-gni.net [130.81.180.4] –  Zypher Jul 1 '11 at 1:17
    
Good deal guys. Keep on keepin on. –  Holocryptic Jul 1 '11 at 1:19
1  
@Zypher Mine dies at a Philly equivalent (G0-10-1-1.PHLAPA-LCR-22.verizon-gni.net [130.81.183.18]), but yeah. –  Tim Stone Jul 1 '11 at 1:20
    
I emailed noc@verzion.net . Didn't seem to bounce -- can't find contact info on IANA or whois. –  Kyle Brandt Jul 1 '11 at 1:55
    
@Zypher my tracert seems to die on different hops each time. Not sure what's up... –  Holocryptic Jul 1 '11 at 2:28
    
@holo you mind posting a couple of the endpoints it dies at? –  Zypher Jul 1 '11 at 2:35
    
@Zypher I'm guessing it dies at endpoints that have ICMP replies turned off, as I have no IPs for them. The last endpoint I have is L100.WASHDC-VFTTP-125.verizon-gni.net [173.66.170.1] and 69.59.196.211 reports: Destination net unreachable. at the end –  Holocryptic Jul 1 '11 at 2:38
    
@Zypher and that L100 endpoint is the next hop after my router... –  Holocryptic Jul 1 '11 at 2:39
3  
Yep looks like it's an outage with their routing on the East cost, anyone in Boston w/ FIOS want to join the party? –  Zypher Jul 1 '11 at 3:01
    
Virginia Fios User, chat's down... I die here g0-2-5-5.nrflva-lcr-21.verizon-gni.net (130.81.140.228) –  Jacob Jul 1 '11 at 5:07
    
@kyle Brandt @Zypher The NOC number for Verizon is 800-243-6994 –  Jacob Jul 1 '11 at 5:12
    
@thebeachException - THANKS! that's going in fu_verizon.txt –  Zypher Jul 1 '11 at 5:29

3 Answers 3

Looks like Verizon is upgrading Fiber connections on the east coast and this is a side effect. They say it should be resolved in 2-4 hours, around 5AM Eastern Time (US).

share|improve this answer
2  
Ah, nice of them to inform us that they were going to break the most important part of the internet. Appreciate the heads up, and since I snuck a nap instead of idling on chat I guess it isn't so bad. –  Tim Stone Jul 1 '11 at 6:35
    
@Tim You're only slightly wrong. This is a travesty –  Holocryptic Jul 1 '11 at 11:36
2  
@Holocryptic Indeed. It seems Verizon has a special interpretation of "2-4 hours." –  Tim Stone Jul 1 '11 at 12:21
    
Nope still having issues here. –  Jacob Jul 1 '11 at 16:26

It looks like the issue has been fixed.

Tracing route to chat.stackexchange.com [69.59.196.211]
over a maximum of 30 hops:

  1     3 ms     4 ms     3 ms  Wireless_Broadband_Router.home [192.168.1.1]
  2    20 ms    14 ms    16 ms  L100.WASHDC-VFTTP-125.verizon-gni.net [173.66.17
0.1]
  3    13 ms    25 ms    20 ms  G5-0-5-1725.WASHDC-LCR-07.verizon-gni.net [130.8
1.140.110]
  4    15 ms    13 ms    12 ms  so-12-1-0-0.LCC1-RES-BB-RTR1-RE1.verizon-gni.net
 [130.81.151.230]
  5    20 ms    19 ms    33 ms  0.so-2-3-0.XL3.IAD8.ALTER.NET [152.63.30.17]
  6   115 ms   105 ms   113 ms  0.ge-1-2-0.XT3.POR3.ALTER.NET [152.63.106.33]
  7   102 ms   104 ms   109 ms  GigabitEthernet6-0.GW7.POR3.ALTER.NET [152.63.10
6.54]
  8   102 ms   100 ms   101 ms  noanetoregon-por-gw.customer.alter.net [157.130.
179.94]
  9   101 ms   124 ms   100 ms  vl-12-PTLDORPBCR01.lsnetworks.net [216.110.192.2
1]
 10   105 ms   103 ms   108 ms  EUGNOR53CR01.lsnetworks.net [216.110.192.42]
 11   115 ms   111 ms   106 ms  206-192-226-18.lsnetworks.net [206.192.226.18]
 12   110 ms   125 ms   112 ms  ge-0-0-0-cvo-core2.peak.org [69.59.218.197]
 13   112 ms   131 ms   111 ms  vlan5-cvo-colo2.peak.org [69.59.218.226]
 14   112 ms   105 ms   112 ms  stackoverflow.com [69.59.196.211]

Trace complete.

Now I can carry on with the naked drunken debauchery in the privacy of my own home, instead of glancing furtively over my shoulder at work.

share|improve this answer

I'm having the same problem:

tracert

  1     1 ms    <1 ms    <1 ms  Wireless_Broadband_Router.home [192.168.1.1]
  2     4 ms     3 ms     4 ms  L100.NYCMNY-VFTTP-169.verizon-gni.net [71.190.135.1]
  3     *     G0-13-1-5.NYCMNY-LCR-22.verizon-gni.net [130.81.182.78]  reports: Destination net unreachable.

Note: I have Verizon (obv.) and I love me some FIOS, so while this is a sad state right now Verizon does not suck (most of the time.)

share|improve this answer

Not the answer you're looking for? Browse other questions tagged .