Create a static error page to be shown when runtime errors occur

Question

Today as also some months ago, we had on some of your services a Runtime Error:

Server Error in '/' Application.
Runtime Error
Description: An exception occurred while processing your request. Additionally, another exception occurred while executing the custom error page for the first exception. The request has been terminated.

So the aspx error page could not be executed, because of an exception in itself.

I want to suggest you to create the static files app_offline.htm and app_error.htm in your app root. I do so by having a _app_offline.htm file, that I rename to app_offline.htm to shutdown and disable my app. Also a app_error.htm, that is presenting a nice static message, if there is an unplanned exception.

These files are static, to not involve ASP.NET and therefore not to have an exception when building an exception message.

To be more precise, this static app error page is not to replace the dynamic app error page. It is only returned, if the dynamic page fails. The app offline page is to disable ASP.NET processing permanently, not to only shutdown the current application domain instance.

It's just to be nicer than the default ASP.NET fallback message.

Answer 1

It's...not that simple for us, we're a bit more complicated than an app on a web server.

To get an overall idea: we have 18 web servers running two dozen applications against 2 SQL clusters with 3 nodes each across 2 datacenters, supported by 4 central service boxes (soon 6) with 5 distinct VPN endpoints filtered by 2 sets of redundant load balancers. In Oregon where we are running form now that's 2 SQL servers, 7 web servers, 2 service boxes, 2 load balancers, 2 routers and a VPN to NY in play.

This morning two related but separate outages occurred, here's some detail:

The first outage was ultimately the result of Steve and I patching up the NY infrastructure in preparation for moving all the servers next week. While rebooting 2 of the SQL servers, 1 in each cluster, the windows clustering resources went offline in Oregon (which shouldn't have happened...MS is looking into that). The lost connection to both NY nodes (which had no quorum votes, so it shouldn't have mattered) caused Oregon to restart as a singleton instance, recycling the databases hosted in that node's availability groups. We see what happened, what we don't know is why, and the Microsoft clustering team is helping is out with that now.

The second outage was conntrack hitting a wall on our primary router out in Oregon, causing a failover and back to restore service (from the redundant secondary router and load balancer). This cause the same connection interrupt between Oregon and NY SQL nodes, again causing an outage for a few minutes.

This was just an extremely unlucky morning, as this stuff has been running fine for 2 months. The problem that needs fixing here (relating to the question) is figuring out why we're throwing a YSOD so early in the pipeline and our traditional error page isn't rendering. We'll try and get to that this week, the priority though is trying to make sure the root cause doesn't happen again. In other words: while we can make the error page prettier, we'd rather you just not get an error.