Intermittently, this error keeps coming up, when browsing Meta:

enter image description here

This happened three times in Firefox and Chrome. It would take a really long time to respond, I got a 500 server error, then this.

This tweet by Nick Craver seems related to this issue. Can you look into it, please?

  • 1
    For several minutes just a short while ago, I was also getting various errors about SE being offline for maintenance or can't connect, but everything seems to be working again now, at least for me. Oct 13 '20 at 13:34
  • @JohnOmielan I'm still getting 'em, unfortunately. I did say intermittently, wait 15 minutes or so.
    – Ollie
    Oct 13 '20 at 13:35
  • 2
    Probably related: twitter.com/Nick_Craver/status/1316005021176266752
    – janw
    Oct 13 '20 at 13:38
  • @Ollie Thanks for the extra detail. I just started using SE again a few minutes ago after not using it over 6 hours, so I didn't realize about any issues earlier. Oct 13 '20 at 13:39
  • No need to look into, as it already is fixed.
    – Luuklag
    Oct 13 '20 at 14:05

TL;DR; This should be resolved now.

Longer Explanation:

This was an issue with the SQL Server clusters that took us offline. Over the last couple of weeks, we have been upgrading the storage on quite a few of our SQL Servers. Last night, we did a failover of the main servers - the servers that run Stack Overflow, the SE Network, Jobs, and Chat - plus other things. While it took a little while for the failover to finish it appeared everything worked as expected.

Fast forward about 6 hours to this morning, I did the prepwork so we could upgrade the storage on the last couple of servers. Fast forward another hour and I shut the secondary SQL Server for Stack Overflow down. As soon as that happened, we noticed that exceptions were spiking and we were alerted that the network failing.

During the investigation and trying to bring the servers back online, we noticed that when the failover took place on the Stack Overflow cluster the quorum votes were not correct. The vote stayed on the secondary that I shutdown this morning, instead of being on the primary server in the cluster. Since the vote weight was wrong, we went offline.

After moving the vote to the primary server in the cluster, we came back online everywhere. It was an exciting morning all around.

  • 9
    See, that is what you get with too much down voting ...
    – rene
    Oct 13 '20 at 15:04

It also occurred in Mathematics Stack Exchange, but it was mentioned that the site is offline for maintenance.
enter image description here

It's fixed now.

These were the links that opened.
Status blog

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .