What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 131 Stack Exchange communities.

I think there are multiple people aware of this, but I want to start a question for it to open a discussion and for admins to post relevant details.

I've noticed (and from reading comments here and elsewhere, others have too) that there is an intermittent failure which results in a database error page, or the standard StackOverflow error page. I'll try to screen cap it next time it happens.

I'd assume admins likely already know about this from error logs and server messages. Just wondering what's going on here.

EDIT: So, looking at both the screenshots, it looks like there's really only one error in the back-end. The other screenshot just says that it failed to properly display the error message.

share|improve this question

closed as too localized by balpha Jan 23 '13 at 9:20

This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.If this question can be reworded to fit the rules in the help center, please edit the question.

    
I have a screen capture of both error pages if you want me to post them. –  Devon Bernard Jan 14 '13 at 2:26
    
Sure! Comment them here and I'll add them to the post. –  Emrakul Jan 14 '13 at 2:27
    
Dont bother except for Jon Skeet, we've all seen a YSOD... –  Jeremy Thompson Jan 14 '13 at 3:11
3  
They should post a question on SO to see if anyone knows a solution ;-) –  PeterJ Jan 14 '13 at 8:15

3 Answers 3

up vote 16 down vote accepted

Windows 2008 R2 server clustering on top of which SQL 2012 Always on Availability groups are built is buggy as hell. We're tracking down the YSOD cause now and attempting to fix before doing the next patch round.

To add to the fun, we've had memory and/or a processor memory controller fail in a server in New York. Apologies for the errors, but expect more fun...that's why we're doing this on a Sunday night.

Update: it wasn't the processor, which we replaced at 3 am, dammit.

Annnnnnd, after powering down New York, (but this time on a 15 minute delay, so we were well away from the data center, touché windows) clustering went offline again. But hey! at least we found out a) our NY office building is locked tight after 3am, and b) our mobile hotspots work.

share|improve this answer
    
Just curious. Is all the traffic pouring in from Reddit and HN right now complicating this? Or is that still just a tiny portion of the entire site's traffic for this time of the day/week? –  Mysticial Jan 14 '13 at 2:41
1  
@Mysticial - nah we haven't even noticed that traffic...we're doing some patching and shutdown before shutting NY down for the move. We've had issues with the clustering before, so we opted for a Sunday night operation. –  Nick Craver Jan 14 '13 at 2:48
3  
"we haven't even noticed that traffic" - wow, says something about how much traffic the network gets... –  Mysticial Jan 14 '13 at 3:00
    
Good luck, thx for the update, and thanks for staying bleeding edge with a great site! –  Jeroen Jan 14 '13 at 8:23
    
Does this impact OCD attempts to get the 100-day every-day login badge? I wanna get that badge... –  Lizz Jan 14 '13 at 8:38
    
Sunday? But it's Monday morning in Finland (Europe), workday started hours ago. Yes, I surf SO to get my work done :) –  JOM Jan 14 '13 at 8:43
    
@Mysticial According to stackexchange.com/sites#traffic, SO gets 4.4M visits per day. Assuming all visits (43k) to your question came in one day, that would still only be around 1% of the traffic SO gets on average –  Yi Jiang Jan 14 '13 at 12:02

Here are the two related error pages I took a screen shot of when the site went down in the past hour.

enter image description here enter image description here

share|improve this answer
    
Thanks but it only helps if CustomErrors was off or if we were viewing the site from the server - otherwise we cant see the actual error message. –  Jeremy Thompson Jan 14 '13 at 3:08
    
@JeremyThompson you sort of can, view source for the error page & you'll see the actual error as a HTML comment –  Sathya Jan 14 '13 at 4:14
2  
@Sathya That's not particularly doable from a screenshot –  Michael Mrozek Jan 14 '13 at 8:27

Here's another one giving me an error:

immediately redirects to the error page

which displays

Server Error in '/' Application.
Runtime Error
Description: An exception occurred while processing your request. Additionally, another exception occurred while executing the custom error page for the first exception. The request has been terminated.

when logged in. In a different browser session (logged out), it seems to hang.


It seems chat is also down: http://chat.stackexchange.com/error?aspxerrorpath=/

Back up as of 2013-01-14T08:21Z.

share|improve this answer
    
See the updated answer above for the outage you observed, still - reports appreciated! –  Nick Craver Jan 14 '13 at 14:35

Not the answer you're looking for? Browse other questions tagged .