What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

Over the past couple of days, I have noticed that Stack Overflow is faster than ever. This is severely affecting my thumb twiddling time. When am I supposed to practice my thumb twiddling if I don't have to wait for the pages to load?? Please think of the thumb twiddlers when considering future performance enhancements!

Eeeek what have you done‽‽‽

share|improve this question

closed as not constructive by status-declined, Rosinante, ChrisF, badp, Jon Seigel Mar 28 '11 at 2:44

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.If this question can be reworded to fit the rules in the help center, please edit the question.

1  
Um... since you're endorsing [always-friday-in-iceland], would it be possible to remove the synonym from [fun]? pretty please? I know how to get around it with chain synonyms, but that only works once, and then upon editing, you have to manually retag it every time. –  waiwai933 Mar 25 '11 at 1:49
35  
You could buy and use a 56kbit modem, that will bring back the slow performance. –  HoLyVieR Mar 25 '11 at 2:14
6  
You forgot that eeeek tag. :P –  Mehrdad Mar 25 '11 at 2:27
3  
Wait... wait. Since when did the always-friday-in-iceland synonym get deleted? –  uɐɯsO uɐɥʇɐN Mar 25 '11 at 3:14
1  
@George It didn't. However, before that synonym was created, a few synonyms to [afii] were created, such as [always-hydrogen-on-friday]. The synonym system doesn't chain retag, so if you use that tag, you get [afii]. However, when the post is edited again... the synonym system sees [afii] and retags it to [fun]. –  waiwai933 Mar 25 '11 at 3:23
3  
@Mysel: Don't make fun of us poor, poor souls stuck on dial-up. :( –  Ullallulloo Mar 25 '11 at 3:29
    
@waiwai933: Cool... I hadn't realized that. –  uɐɯsO uɐɥʇɐN Mar 25 '11 at 16:12
    
I keep expecting to see the word "Paid Advertisement" like you do at the bottom of the page in some magazines. –  Powerlord Mar 25 '11 at 19:47
    
Hey is this the equivalent of "rep fest" in forums? :D –  Shadow Wizard Mar 27 '11 at 13:39
    
@ShadowWizard, oy. Had just been joking around about all the Eeeek negative posts, I thought we needed something positive! (: –  Rebecca Chernoff Mar 27 '11 at 14:54
    
@Rebecca yeah, that's a good idea! Good to see such things here, and the unicorns here and there. BTW what's this character you're using ? Where did you find it? –  Shadow Wizard Mar 27 '11 at 15:15
1  
This is great info, but it probably should have been posted on the blog for everyone to see. –  Jon Seigel Mar 28 '11 at 2:44
add comment

3 Answers

up vote 70 down vote accepted

We looked closely at performance over the last few weeks, particularly in the last 4 or 5 business days.

  • we reduced the total number of queries happening on a lot of pages (question show, question lists, user page, etc.)

  • we deployed a page that shows us our worst query offenders in CPU time, total duration, reads, writes, etc. and optimized the stuff at the top of the lists

  • we improved the way we call LINQ to SQL and moved to pre-compiled LINQ queries in certain hot code paths. This seems to have gotten much faster in .NET 4.0, we saw 8x gains over plain vanilla LINQ and 2x over SQL-queries-as-LINQ-returns.

  • we reduced the number of columns we return in some hot code paths, where we had effectively select * from users and posts.

  • we were getting sloppy about the way we handled search engines, which was causing a lot of undue load -- Google's crawler is now indexing us at, and I am not making this up, 10 requests per second which is the maximum. This is confirmed in Google Webmaster Tools. Google doesn't need all the same work done per page that, say, a user does -- and when Google hits thousands of pages in a few minutes, that can kick off a lot of background work, such as rebuilding related questions. Not expensive by itself, but when multiplied by a hundred at once.. can be quite painful.

  • we optimized the way we retrieve site settings and configuration, which occurs sometimes hundreds of times per page.

  • we had some shared lock blocking and other general inefficiencies in our Redis C# library which was causing Redis (network level cache) to be slower and more blocking that it should be. This has been rewritten to be faster and block far less.

  • all our database calls were not quite going through the same central hub (almost, not quite) causing too many database connections in some circumstances. This has been correctly centralized now.

  • lots of general profiling with .NET profiling tools to identify places where we had gotten sloppy and were doing something inefficient.

  • we are restructuring and reorganizing our JavaScript to make the delineation between "only anonymous users need this" and "registered users need this" clear, so as to reduce the footprint of our site to an absolute minimum for anonymous users. This also significantly improves page load time (perceived and actual) for registered users as well.

Some Google details:

enter image description here enter image description here

share|improve this answer
15  
You guys rock. Not only do you make stuff faster, you also give us the details :) –  Benjol Mar 25 '11 at 20:53
27  
To me, disclosing the technical details about how you run and optimize your sites is one of the greatest services to the community that you guys are doing. Please never ever give it up, the totality of all these contributions already is a treasure trove for everyone interested in running a high-traffic site. (That said, this should probably go on the serverfault blog, no? ... or webmasters?) –  Pëkka Mar 28 '11 at 22:30
    
May I suggest one more improvement? Maybe shrinking whitespace via each html page would help with page load times and network performance. View the Source on your html page. Compressing and removing ALL white space outside the actual content of the html would help increase network performance. –  Scott Mar 29 '11 at 15:57
    
First <strike>page</strike> place hacker news! –  Trufa Mar 29 '11 at 16:08
    
@Scott: I don't expect that would improve performance significantly, and since SE started as (AFAIK) a programmer help site, it makes sense to have the HTML be programmer friendly. –  Mooing Duck Mar 9 '12 at 19:37
    
Some congratulations are mandatory here, I always know when my connection goes off or is being sloppy: I just type "stackoverflow.com" and see how quick the answer is :) –  Marco A. May 1 at 14:20
add comment

Move to New Zealand, you'll have plenty of network-induced delays! Especially if you happen to be behind a corporate firewall, that always helps.

share|improve this answer
10  
Better yet, I'll bet China's great firewall will help. –  Moshe Mar 25 '11 at 4:26
    
yours is caused by bits pouring out of broken cables (bitstream liquifaction - awesome duuuuude!!). For the rest of us it is just crap peering/interconnections by TelstraClear. –  slugster Mar 25 '11 at 7:03
    
People in New Zealand don't need the internet because they've got all that outdoors to play in. If you're in the country that invented bungie jumping and zorbing, what do you need with a computer? :) –  Spudley Mar 29 '11 at 12:39
add comment

enter image description here

But really read this. And then just realize our devs are crazy.

share|improve this answer
    
Okay, so it is not about the latest blog post. Then, er, what is this answer about...? –  Arjan Mar 25 '11 at 19:28
    
Crazy doesn't have a closed definition @arj –  random Mar 25 '11 at 19:40
    
@arj ... read through the blog ... the whole thing is about how we are constantly doing our part as sysadmins to make things faster, better, stronger –  Zypher Mar 25 '11 at 19:51
add comment

Not the answer you're looking for? Browse other questions tagged .