Recently, I read this Stack Overflow blog post about how they have some of their infrastructure laid out.

I was curious to know, does Stack Exchange use caching (both at the application layer and/or database layer) and if so, how?

  • 1
    I don't think the question belongs on meta, but I think it is too broad: there are, at different levels, a thousand things susceptible of being cached in the implementation of a large website such as SO. Many undoubtedly are. Most cache layers are transparent (it's more or less part of the spec) so the programmer doesn't have to worry, think about them, or even know they exist. Commented Nov 1, 2010 at 20:56
  • Of course—it would otherwise be prohibitively expensive (read: "impossible") for the system to scale to the level of activity Stack Overflow has these days. It's the biggest reason Stack Exchange runs so efficiently today.
    – bwDraco
    Commented Jul 14, 2015 at 18:28

1 Answer 1


Oh boy, have I been waiting for somebody to ask something like this.

We really aggressively cache... basically everything.

Virtually all pages accessed by (and subsequently served to) anonymous users are cached, whole cloth, via Output Caching. This isn't terribly interesting, but it is terribly fast, though we've recently removed Output Caching from a small number of routes.

Perhaps more interesting, we use Redis as a caching layer for the entire network. Kyle recently mentioned the specs of the machine in passing. Prior to the NY move our Redis instance was in a virtual machine on a (lightly loaded) web tier machine.

In our (admittedly limited) experience, Redis is so fast that the slowest part of a cache lookup is the time spent reading and writing bytes to the network. This is not surprising, really, if you think about it.

We compensate for this in two ways:

  1. We keep a "L1" cache (basically, HttpRuntime.Cache) of recently set/read values on a server
    • Naturally, the fastest # of bytes to send across a network is 0.
  2. We compress values* before sending them to Redis
    • Most of our cached values are strings, so even at low CPU usage we get great compression ratios
    • We also have plenty of CPU time to spare on the web tier, especially in the NY datacenter.
    • We've gotten a little smarter and only compress things that actually get smaller compressed (which requires some calisthenics when reading keys), and .NET 4.5's GZip implementation is a lot better

Conceptually, each site has 3 distinct caches:

  1. A "local cache," which can only be accessed from 1 server/site pair
    • Contains things like user sessions, and pending view count updates
    • This resides purely in memory, no network or DB access
  2. A "site cache," which can be accessed by any instance (on any server) of a single site
    • Most cached values go here, things like hot question id lists and user acceptance rates are good examples
    • This resides in Redis (in a distinct DB, purely for easier debugging)
  3. A "global cache," which is shared amongst all sites and servers
    • Inboxes, API usage quotas, and a few other truly global things live here
    • This resides in Redis (in DB 0, likewise for easier debugging)

We do our best to avoid cache invalidation, thus most things in the cache merely expire and are never explicitly removed. However, for those rare cases where removal is necessary we use Redis messaging to publish removal notices** (only to those sites that care, for scaling's sake).

Some quick stats:

  • At any one point time we have around 1,300,000 keys in our Redis cache
    • Most of these expire on the order of a few minutes
    • This number has been growing as we've become more confident in Redis
  • Around peak time we're getting a few hundred read/writes a second to Redis
    • This is well below the benchmarks out there
    • CPU usage on our (way overkill, it turns out) dedicated Redis machine is essentially 0%, all the time
      • Like, really, all the time
      • After a few months of growth, we're seeing peaks of 0.5% cpu usage
      • After a lot longer, we're seeing about 1% cpu usage
    • Memory usage stays south of 1/2 a GB has grown to just shy of 8 GB is now around 6 GB
      • Not surprising, as we're mostly storing sets of integers and compressed strings.
      • We've also started storing much bigger things in Redis, like inboxes.
      • Newer Redis releases have saved some space, and we've slimmed down some common keys

*Still, for some common use cases the largest part of a Redis command is the key name. One of these days I'm going to find some time to experiment with compressing the entire command stream, and see what kind of performance gains can be made.

**These removals are necessary to invalidate the "L1 Cache," Redis naturally keeps itself in a coherent state in the face of removals.

  • 8
    Are your webservers sticky? Are ip's routed to the same web server to which they hit first? if yes, does haproxy manage this?
    – Cherian
    Commented Mar 6, 2011 at 4:32
  • 1
    Hi , Are you using Redis on windows?? , if yes how stable it is ? Please reply as we are going start high traffic local classified site, and need to use Redis on windows.
    – Ajax2020
    Commented Oct 20, 2011 at 15:54
  • 6
    @Ajax2020 - we host Redis on linux (Ubuntu Server specifically), we've never had any stability issues. Commented Oct 20, 2011 at 16:02
  • 31
    Can you add details of the interaction between your SQL server and the cache, like how do you populate the cache? Is everything lazy loaded (implying it won't be in the cache if it hasn't been requested)? If so, are there certain pieces/classes of data you explicitly keep seeded? Do you use SQL notifications to keep the cache updated, or do you update the cache using the DAL? Commented Mar 14, 2012 at 21:02
  • Do you cache pages on client? if you do, How do you invalidate cache for an anonymous user when he becames logged user? If I access a page A as anonymous and then I access page A as logged user, how do you handle that? Commented May 17, 2012 at 14:59
  • @KevinMontrose : Is there a specific way through which you are currently doing caching for user specific data? . There are so many user accounts, and the data varies from one user to another. I was wondering if Redis provides us with anything to address such scenarios. Commented Aug 23, 2012 at 11:53
  • 2
    @boddhisattva - We cheat a little in that case, our routing is sticky by IP so user sessions can get cached locally for speed. Commented Aug 23, 2012 at 16:33
  • What's the compression algo used for sending stuff to redis? zlib? just curious
    – Mahn
    Commented Sep 3, 2012 at 16:03
  • @Mahn - We use GZip, which in .NET 4.5 is backed by zlib. Commented Sep 3, 2012 at 16:38
  • Could you tell please what client library are you using to access Redis from .net environment?
    – wallice
    Commented Sep 10, 2012 at 8:09
  • @wallice - Booksleeve was developed in house for our Redis use. Commented Sep 10, 2012 at 19:43
  • I know the question is kinda old, but I'm curious about one thing: You said you avoid cache invalidation and basically let the Redis cache expire by itself. What is the TTL you use? I was familiar with a different approach - long TTLs, but a complex invalidation system, handling invalidation as soon as the actual data changes. What is the better approach?
    – Przemek
    Commented May 29, 2013 at 15:42
  • 1
    Does anyone know if a custom OutputCache Provider is used to store the output cache in the Redis layer? Commented Nov 27, 2013 at 8:08
  • 1
    @tobi: SQL only caches the underlying data, not the result set. So, SQL queries still run against RAM. Such queries may introduce contention (read/write locking), especially since they may hit multiple tables.
    – Brian
    Commented Sep 20, 2016 at 15:18
  • 1
    Third parties have tried to offer a SQL query cache, but none of them have taken off. I know very little about mysql, but will note that it has more than one database engine, so the approach to caching might vary based on configuration. Maybe ask about that on stackoverflow?
    – Brian
    Commented Sep 22, 2016 at 13:01

You must log in to answer this question.