On Friday, July 29th, starting at 13:36 UTC, we experienced a very large surge in traffic to our web servers, indicating a DDoS attack. This surge effectively brought down the Stack Exchange Network sites (including Stack Overflow) and Stack Overflow for Teams (Free, Basic, and Business). Stack Overflow for Teams Enterprise was unaffected. We were able to restore service by 15:48 UTC, and have since deployed new defenses to better address these attacks in the future.
We noticed an increase in traffic spikes starting earlier this year, which can sometimes cause site instability. While we’ve gotten better at reducing the overall impact to the site, these traffic spikes are increasing in frequency and scale. These bursts of traffic can cause some users to see a maintenance page or some other error page momentarily, but we continue to keep the effects to a minimum as much as possible.
To address these overall trends, we recently adopted a new web application firewall to both mitigate vulnerabilities and also to act as an intelligent rate limiter. We’re also currently testing new observability tools to help us respond faster and predict future attacks.
Another area we want to improve is our communication during and after incidents. It is difficult for people working on the technical problem to also be providing status updates. I personally apologize for the delay in responding to the various Meta questions with this post. We are working on a number of improvements: Now that we have an automated status page, we are examining how we can improve the process so that it is updated sooner. Other improvements revolve around additions to the status page itself and how information is displayed. To be clear, the status page reflects that a human is working on the issue, not whether our monitoring system has detected issues. We are working to improve internal processes related to communication, standardizing processes to be more consistent, and clarifying what events trigger communication.
As always, we would like to thank the community for your patience as we work hard on addressing these issues.