TL;DR; This should be fixed in the next 8-24 hours as we had to block a bot scraping us.
Longer version:
I noticed the flapping over the weekend and started investigating yesterday. My first guess was a SQL issue. After some initial digging, I made some server level changes to see if things would stabilize. They did for several hours, but last night I noticed yet another outage, so I grabbed a memory dump for review this morning.
After a whole lot of flapping overnight, Nick and I started reviewing the dumps this morning, well mostly Nick reviewed and I watched. We noticed some threading issues, so we look at the total connections to SEDE to figure out what was hogging all the connections, but nothing really stood out, so we turned to our Traffic Logs and we found the issue.
We initially queried the logs for the last couple of days and saw we were being crawled by Applebot. Crawled is an understatement, we were getting slammed by it.
Here are some stats of traffic from the Applebot and everyone else to give an idea on scale - the outages started on 2020-08-16 between that date to today (2020-08-18) we had:
Applebot | Non-Applebot Traffic
---------------------------------
1484480 | 290063
We've added the Applebot the robots.txt to stop the crawling. It'll take a little while for that to take hold, so we might have a little more flapping of SEDE, but we'll be monitoring it and will kick as needed.