What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 134 Stack Exchange communities.

For a few hours now, the heading that appears on the home page of Stack Exchange sites notifying of newly active questions has not been appearing. I have had to manually refresh in order to see new activity.

I have observed this on SF, SU and right here on mSO.

Upon hitting F12 in Chrome and looking around, it appears that the WebSockets connection to sockets.ny.stackexchange.com is failing, though Chrome doesn't tell me exactly why.

sockets.ny.stackexchange.com fails

On further investigation, I also seem to not be receiving notification of reputation changes, upvote/downvote changes, new comments notifications, flags, or inbox notifications.

share|improve this question
6  
I am having the same problem. –  Antony Jun 5 '13 at 7:07
1  
What you get in this page? –  Shadow Wizard Jun 5 '13 at 7:09
    
@ShaWizDowArd websocketstest.com/result/227406 –  Michael Hampton Jun 5 '13 at 7:10
    
So likely temporary downtime on Stack Exchange side. Happens to the best! :) –  Shadow Wizard Jun 5 '13 at 7:11
    
@Yannis The AJAX thingy needs to be notified of the updates before I can use them. There is no update without the web sockets. –  Antony Jun 5 '13 at 7:12
    
@Antony Ah, I see. –  Yannis Jun 5 '13 at 7:12
    
This has been happening for me since last night. I assumed it was a temp outage and only poked @StackStatus on Twitter about it. –  Martijn Pieters Jun 5 '13 at 7:35
1  
Appears to be back now. Where's Nick with his technical explanations? :) –  Shadow Wizard Jun 5 '13 at 12:28
    
The socket is back, at least for now. –  Martijn Pieters Jun 5 '13 at 13:09

1 Answer 1

up vote 6 down vote accepted

On our load balancer, we have multiple instances of HAProxy. One of these instances is dedicated to realtime updates (websockets) and somehow failed. After taking a core dump of that instance, I killed the process and started it again to get websockets working again.

Our current next steps are:

  1. Keep investigating to try to figure out exactly what went wrong that caused websockets to stop responding
  2. Set up more extensive monitoring for websockets so we have an alert when websockets stops responding

I will update this when I discover more information.

share|improve this answer
    
Cheers, I think this bug can be status-completed though. –  Shadow Wizard Jun 5 '13 at 13:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .