5

I knew this question might be more technical, but amazing to know at least some background information about hosting model and how a dynamic site with millions of users can be navigated very fast...

Also, with slow Internet Line, I can always navigate SO without any problem :)

Can you please explain how you made that?

9
  • blog.stackoverflow.com/2008/09/… Commented Jul 23, 2013 at 12:08
  • Thanks @MartijnPieters: but we also knew about them, what is amazing here how these system integrated that makes the site very fast :)
    – user210003
    Commented Jul 23, 2013 at 12:11
  • 5
    Caching...the answers is always caching.
    – Bart
    Commented Jul 23, 2013 at 12:12
  • 1
    (lots of server power) + (efficient coding) + (minified resources) + (tons of caching) = what you see. :) Commented Jul 23, 2013 at 12:13
  • 1
    Google's indexing has nothing to do with SO itself. It's just that the site is popular and frequently "updated", so Google pays attention.
    – Bart
    Commented Jul 23, 2013 at 12:16
  • I thought about caching, but each page and post has variables such as votes, comments, status update! then I think its a power of juqery somehow and ajax
    – user210003
    Commented Jul 23, 2013 at 12:16
  • @phwd: why you deleted the image?
    – user210003
    Commented Jul 23, 2013 at 12:26
  • @Bart: I will be wrong, but fast indexing by google indicating a connection between SO and Google :) I love to see became partner
    – user210003
    Commented Jul 23, 2013 at 12:29
  • 1
    There is no connection between Google and Stack Exchange. Google's algorithm just loves sites that are frequently updated with lots of new content. That's what we do. Commented Jul 23, 2013 at 12:37

1 Answer 1

8

We are fanatical about performance - everything boils down to that. Every day we look for ways to improve.

  • Can we not run a query here?
  • Can we consolidate queries?
  • Can we extend this index to cover 20% more cases?
  • Can we denormalize this data to eliminate a join?
  • Can we stick this in redis?
  • Should we stick this in redis?
  • Should we aggregate this into a central DB?
  • Can we eliminate this API call?
  • Is JSON right here?
  • How about Protobuf?
  • Can we not do an allocation here?
  • Can we completely change how this is cached and accessed?
  • Is this hit often enough to rewrite it in IL?
  • How about we just replace the ORM?

...these are daily questions around here. It's a culture thing, and we love it.

Edit: I am preparing an architecture overview for our upcoming Velocity talk in October, we'll be sure to share it when done - for those that are curious.

3
  • That what I expected, there is no linear system for fast performance, the hardware and integration plays a part in performance, but optimization is also great when the site viewed by millions of user :)
    – user210003
    Commented Jul 23, 2013 at 12:31
  • 3
    We control all the hardware and software - so how we setup the network bonds, tweak the TCP buffers, layout security so low-latency queries need not pass a router, etc...it all plays a part, it just adds up. I should write a post on everything we do in the name of performance (it's a long, long list), just have to find the time. Commented Jul 23, 2013 at 12:33
  • 2
    Don't forget Unicorn dust. It really speeds things up.
    – Oded
    Commented Jul 23, 2013 at 12:35