43

This work is now complete

tl;dr

Planned service interruption that will impact all Stack Overflow & Stack Exchange sites, Jobs, and Chat. All sites may be read-only for up to 30 minutes beginning shortly after Thursday, 16 December at 01:30 UTC (Wednesday, December 15 at 8:30 PM US/EST). Enterprise cloud hosted instances will not be impacted.

Short Version:

There will be a service degradation for up to 30 minutes starting at 01:30am UTC on 16 December, 2021 (December 15 at 8:30 PM US/EST). During that time, questions and answers will still display, job listings will still work, and job ads will still display. However, the site will be "read only" — i.e. people won't be able to add/edit new job listings, apply for jobs, post, edit or vote on questions/comments/answers, reputation won't change, etc. Users will not be able to log into Stack Overflow for Teams (free/basic/business tiers) while the site is read only. This should minimize the disruption to the majority of casual readers. We will display a banner on the sites stating that they're read-only for maintenance.

Longer Version of What's Taking Place:

Background

Our primary database servers – which power Stack Overflow, Stack Exchange, Chat, and other things – run out of New York (really New Jersey) with our secondary location in Colorado. When we need to perform maintenance on the primary servers, we need to failover to another server so we can patch, upgrade, and reboot them.

When we do patching for our database servers, we generally patch secondary servers, failover the primary server to a patched node, then patch the former primary (post-failover secondary).

What we'll be doing

During the service interruption, we'll be performing failovers from the primary database servers to the secondary servers in New York for the SQL Servers that support Stack Overflow, the Stack Exchange Network, jobs, and chat. This will allow us to patch the former primary SQL Server with the latest Windows & SQL Server patches.

We expect that the site will be in a read-only state for between 15 and 30 minutes in total. Once we've done the failover, and everything seems to be up and running, we'll take the sites out of read-only.

Questions or concerns?

Please post a comment or answer below; I'll do my best to address any concerns between now and the maintenance window.

Improve this question
10
  • 10
    Good luck with the work
    – Luuklag
    Commented Dec 13, 2021 at 15:27
  • 5
    This will be a bash for the Bash.
    – Rob
    Commented Dec 13, 2021 at 15:59
  • 1
    Why does failovering place the site in a read-only state?
    – Vikki
    Commented Dec 14, 2021 at 11:18
  • 6
    @Vikki We've been working through performance issues with SQL Server during failover. Taryn referred to it in a comment on a prior failover. We're still working through this with each failover, and until we've found & confirmed a fix, we're announcing the short read-only periods each time we do a planned failover.
    – AMtwo Staff
    Commented Dec 14, 2021 at 13:59
  • 5
    Will this interfere with the graduations?
    – wizzwizz4
    Commented Dec 15, 2021 at 15:21
  • 2
    @wizzwizz4 can you give some more context? What are "graduations"?
    – Aaron Bertrand Staff
    Commented Dec 15, 2021 at 15:58
  • 3
    @AaronBertrand (At least) some sites are due to come out of beta on that day. I don't know the details.
    – wizzwizz4
    Commented Dec 15, 2021 at 16:04
  • 4
    @wizzwizz4 This maintenance window won't have any impact beyond site access for the ~30-minute window. Everything else is on schedule for today (like Winter Bash!) is still happening & not impacted.
    – AMtwo Staff
    Commented Dec 15, 2021 at 17:57
  • We (a hospital in Denver) run an AlwaysOn Availability group and are quite curious about the performance issues you're seeing during failover. I hope you and your colleagues are able to write about it so we can all learn :)
    – Mark Iannucci
    Commented Dec 16, 2021 at 2:50
  • 3
    To close voters: Please do not close this as "No longer reproducible" - We have a history of leaving maintenance announcements open.
    – Spevacus Mod
    Commented Dec 16, 2021 at 3:00

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .