69

Over the past couple of days, I have noticed that Stack Overflow is faster than ever. This is severely affecting my thumb twiddling time. When am I supposed to practice my thumb twiddling if I don't have to wait for the pages to load?? Please think of the thumb twiddlers when considering future performance enhancements!

Eeeek what have you done‽‽‽

Improve this question
11
  • 1
    Um... since you're endorsing [always-friday-in-iceland], would it be possible to remove the synonym from [fun]? pretty please? I know how to get around it with chain synonyms, but that only works once, and then upon editing, you have to manually retag it every time.
    – waiwai933
    Mar 25, 2011 at 1:49
  • 37
    You could buy and use a 56kbit modem, that will bring back the slow performance.
    – HoLyVieR
    Mar 25, 2011 at 2:14
  • 6
    You forgot that eeeek tag. :P
    – user541686
    Mar 25, 2011 at 2:27
  • 3
    Wait... wait. Since when did the always-friday-in-iceland synonym get deleted?
    – uɐɯsO uɐɥʇɐN
    Mar 25, 2011 at 3:14
  • 1
    @George It didn't. However, before that synonym was created, a few synonyms to [afii] were created, such as [always-hydrogen-on-friday]. The synonym system doesn't chain retag, so if you use that tag, you get [afii]. However, when the post is edited again... the synonym system sees [afii] and retags it to [fun].
    – waiwai933
    Mar 25, 2011 at 3:23
  • 3
    @Mysel: Don't make fun of us poor, poor souls stuck on dial-up. :(
    – Ullallulloo
    Mar 25, 2011 at 3:29
  • I keep expecting to see the word "Paid Advertisement" like you do at the bottom of the page in some magazines.
    – Powerlord
    Mar 25, 2011 at 19:47
  • Hey is this the equivalent of "rep fest" in forums? :D
    – Shadow Wizard Is Sad And Angry
    Mar 27, 2011 at 13:39
  • @ShadowWizard, oy. Had just been joking around about all the Eeeek negative posts, I thought we needed something positive! (:
    – Rebecca Chernoff
    Mar 27, 2011 at 14:54
  • @Rebecca yeah, that's a good idea! Good to see such things here, and the unicorns here and there. BTW what's this character you're using ‽? Where did you find it?
    – Shadow Wizard Is Sad And Angry
    Mar 27, 2011 at 15:15
  • 1
    This is great info, but it probably should have been posted on the blog for everyone to see.
    – Jon Seigel
    Mar 28, 2011 at 2:44

3 Answers 3

Reset to default
78

We looked closely at performance over the last few weeks, particularly in the last 4 or 5 business days.

  • we reduced the total number of queries happening on a lot of pages (question show, question lists, user page, etc.)

  • we deployed a page that shows us our worst query offenders in CPU time, total duration, reads, writes, etc. and optimized the stuff at the top of the lists

  • we improved the way we call LINQ to SQL and moved to pre-compiled LINQ queries in certain hot code paths. This seems to have gotten much faster in .NET 4.0, we saw 8x gains over plain vanilla LINQ and 2x over SQL-queries-as-LINQ-returns.

  • we reduced the number of columns we return in some hot code paths, where we had effectively select * from users and posts.

  • we were getting sloppy about the way we handled search engines, which was causing a lot of undue load -- Google's crawler is now indexing us at, and I am not making this up, 10 requests per second which is the maximum. This is confirmed in Google Webmaster Tools. Google doesn't need all the same work done per page that, say, a user does -- and when Google hits thousands of pages in a few minutes, that can kick off a lot of background work, such as rebuilding related questions. Not expensive by itself, but when multiplied by a hundred at once.. can be quite painful.

  • we optimized the way we retrieve site settings and configuration, which occurs sometimes hundreds of times per page.

  • we had some shared lock blocking and other general inefficiencies in our Redis C# library which was causing Redis (network level cache) to be slower and more blocking that it should be. This has been rewritten to be faster and block far less.

  • all our database calls were not quite going through the same central hub (almost, not quite) causing too many database connections in some circumstances. This has been correctly centralized now.

  • lots of general profiling with .NET profiling tools to identify places where we had gotten sloppy and were doing something inefficient.

  • we are restructuring and reorganizing our JavaScript to make the delineation between "only anonymous users need this" and "registered users need this" clear, so as to reduce the footprint of our site to an absolute minimum for anonymous users. This also significantly improves page load time (perceived and actual) for registered users as well.

Some Google details:

enter image description here enter image description here

Improve this answer
6
  • 17
    You guys rock. Not only do you make stuff faster, you also give us the details :)
    – Benjol
    Mar 25, 2011 at 20:53
  • 29
    To me, disclosing the technical details about how you run and optimize your sites is one of the greatest services to the community that you guys are doing. Please never ever give it up, the totality of all these contributions already is a treasure trove for everyone interested in running a high-traffic site. (That said, this should probably go on the serverfault blog, no? ... or webmasters?)
    – Pekka
    Mar 28, 2011 at 22:30
  • May I suggest one more improvement? Maybe shrinking whitespace via each html page would help with page load times and network performance. View the Source on your html page. Compressing and removing ALL white space outside the actual content of the html would help increase network performance.
    – Scott
    Mar 29, 2011 at 15:57
  • First <strike>page</strike> place hacker news!
    – Trufa
    Mar 29, 2011 at 16:08
  • @Scott: I don't expect that would improve performance significantly, and since SE started as (AFAIK) a programmer help site, it makes sense to have the HTML be programmer friendly.
    – Mooing Duck
    Mar 9, 2012 at 19:37
  • Some congratulations are mandatory here, I always know when my connection goes off or is being sloppy: I just type "stackoverflow.com" and see how quick the answer is :)
    – Marco A.
    May 1, 2014 at 14:20
19

Move to New Zealand, you'll have plenty of network-induced delays! Especially if you happen to be behind a corporate firewall, that always helps.

Improve this answer
3
  • 11
    Better yet, I'll bet China's great firewall will help.
    – Moshe
    Mar 25, 2011 at 4:26
  • yours is caused by bits pouring out of broken cables (bitstream liquifaction - awesome duuuuude!!). For the rest of us it is just crap peering/interconnections by TelstraClear.
    – slugster
    Mar 25, 2011 at 7:03
  • 1
    People in New Zealand don't need the internet because they've got all that outdoors to play in. If you're in the country that invented bungie jumping and zorbing, what do you need with a computer? :)
    – Spudley
    Mar 29, 2011 at 12:39
17

enter image description here

But really read this. And then just realize our devs are crazy.

Improve this answer
3
  • Okay, so it is not about the latest blog post. Then, er, what is this answer about...?
    – Arjan
    Mar 25, 2011 at 19:28
  • Crazy doesn't have a closed definition @arj
    – random
    Mar 25, 2011 at 19:40
  • @arj ... read through the blog ... the whole thing is about how we are constantly doing our part as sysadmins to make things faster, better, stronger
    – Zypher
    Mar 25, 2011 at 19:51

Not the answer you're looking for? Browse other questions tagged .