Does Stack Exchange use caching and if so, how?

Question

Recently, I read this Stack Overflow blog post about how they have some of their infrastructure laid out.

I was curious to know, does Stack Exchange use caching (both at the application layer and/or database layer) and if so, how?

Answer 1

Oh boy, have I been waiting for somebody to ask something like this.

We really aggressively cache... basically everything.

Virtually all pages accessed by (and subsequently served to) anonymous users are cached, whole cloth, via Output Caching. This isn't terribly interesting, but it is terribly fast, though we've recently removed Output Caching from a small number of routes.

Perhaps more interesting, we use Redis as a caching layer for the entire network. Kyle recently mentioned the specs of the machine in passing. Prior to the NY move our Redis instance was in a virtual machine on a (lightly loaded) web tier machine.

In our (admittedly limited) experience, Redis is so fast that the slowest part of a cache lookup is the time spent reading and writing bytes to the network. This is not surprising, really, if you think about it.

We compensate for this in two ways:

1. We keep a "L1" cache (basically, HttpRuntime.Cache) of recently set/read values on a server
   • Naturally, the fastest # of bytes to send across a network is 0.
2. We compress values* before sending them to Redis
   • Most of our cached values are strings, so even at low CPU usage we get great compression ratios
   • We also have plenty of CPU time to spare on the web tier, especially in the NY datacenter.
   • We've gotten a little smarter and only compress things that actually get smaller compressed (which requires some calisthenics when reading keys), and .NET 4.5's GZip implementation is a lot better

Conceptually, each site has 3 distinct caches:

1. A "local cache," which can only be accessed from 1 server/site pair
   • Contains things like user sessions, and pending view count updates
   • This resides purely in memory, no network or DB access
2. A "site cache," which can be accessed by any instance (on any server) of a single site
   • Most cached values go here, things like hot question id lists and user acceptance rates are good examples
   • This resides in Redis (in a distinct DB, purely for easier debugging)
3. A "global cache," which is shared amongst all sites and servers
   • Inboxes, API usage quotas, and a few other truly global things live here
   • This resides in Redis (in DB 0, likewise for easier debugging)

We do our best to avoid cache invalidation, thus most things in the cache merely expire and are never explicitly removed. However, for those rare cases where removal is necessary we use Redis messaging to publish removal notices** (only to those sites that care, for scaling's sake).

Some quick stats:

• At any one point time we have around 1,300,000 keys in our Redis cache
  • Most of these expire on the order of a few minutes
  • This number has been growing as we've become more confident in Redis
• Around peak time we're getting a few hundred read/writes a second to Redis
  • This is well below the benchmarks out there
  • CPU usage on our (way overkill, it turns out) dedicated Redis machine is essentially 0%, all the time
    • Like, really, all the time
    • After a few months of growth, we're seeing peaks of 0.5% cpu usage
    • After a lot longer, we're seeing about 1% cpu usage
  • Memory usage stays south of 1/2 a GB has grown to just shy of 8 GB is now around 6 GB
    • Not surprising, as we're mostly storing sets of integers and compressed strings.
    • We've also started storing much bigger things in Redis, like inboxes.
    • Newer Redis releases have saved some space, and we've slimmed down some common keys

_{*Still, for some common use cases the largest part of a Redis command is the key name. One of these days I'm going to find some time to experiment with compressing the entire command stream, and see what kind of performance gains can be made.}

_{**These removals are necessary to invalidate the "L1 Cache," Redis naturally keeps itself in a coherent state in the face of removals.}