Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 153 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

Recently, I read this Stack Overflow blog post about how they have some of their infrastructure laid out.

I was curious to know, does Stack Exchange use caching (both at the application layer and/or database layer) and if so, how?

share|improve this question

migrated from Nov 1 '10 at 21:01

This question came from our site for professional and enthusiast programmers.

I don't think the question belongs on meta, but I think it is too broad: there are, at different levels, a thousand things susceptible of being cached in the implementation of a large website such as SO. Many undoubtedly are. Most cache layers are transparent (it's more or less part of the spec) so the programmer doesn't have to worry, think about them, or even know they exist. – Complicated see bio Nov 1 '10 at 20:56
Of course—it would otherwise be prohibitively expensive (read: "impossible") for the system to scale to the level of activity Stack Overflow has these days. It's the biggest reason Stack Exchange runs so efficiently today. – bwDraco Jul 14 '15 at 18:28

Oh boy, have I been waiting for somebody to ask something like this.

We really aggressively cache... basically everything.

Virtually all pages accessed by (and subsequently served to) anonymous users are cached, whole cloth, via Output Caching. This isn't terribly interesting, but it is terribly fast.

Perhaps more interesting, we use Redis as a caching layer for the entire network. Kyle recently mentioned the specs of the machine in passing. Prior to the NY move our Redis instance was in a virtual machine on a (lightly loaded) web tier machine.

In our (admittedly limited) experience, Redis is so fast that the slowest part of a cache lookup is the time spent reading and writing bytes to the network. This is not surprising, really, if you think about it.

We compensate for this in two ways:

  1. We keep a "L1" cache (basically, HttpRuntime.Cache) of recently set/read values on a server
    • Naturally, the fastest # of bytes to send across a network is 0.
  2. We compress values* before sending them to Redis
    • Most of our cached values are strings, so even at low CPU usage we get great compression ratios
    • We also have plenty of CPU time to spare on the web tier, especially in the NY datacenter.
    • We've gotten a little smarter and only compress things that actually get smaller compressed (which requires some calisthenics when reading keys), and .NET 4.5's GZip implementation is a lot better

Conceptually, each site has 3 distinct caches:

  1. A "local cache," which can only be accessed from 1 server/site pair
    • Contains things like user sessions, and pending view count updates
    • This resides purely in memory, no network or DB access
  2. A "site cache," which can be accessed by any instance (on any server) of a single site
    • Most cached values go here, things like hot question id lists and user acceptance rates are good examples
    • This resides in Redis (in a distinct DB, purely for easier debugging)
  3. A "global cache," which is shared amongst all sites and servers
    • Inboxes, API usage quotas, and a few other truly global things live here
    • This resides in Redis (in DB 0, likewise for easier debugging)

We do our best to avoid cache invalidation, thus most things in the cache merely expire and are never explicitly removed. However, for those rare cases where removal is necessary we use Redis messaging to publish removal notices** (only to those sites that care, for scaling's sake).

Some quick stats:

  • At any one point time we have around 1,300,000 keys in our Redis cache
    • Most of these expire on the order of a few minutes
    • This number has been growing as we've become more confident in Redis
  • Around peak time we're getting a few hundred read/writes a second to Redis
    • This is well below the benchmarks out there
    • CPU usage on our (way overkill, it turns out) dedicated Redis machine is essentially 0%, all the time
      • Like, really, all the time
      • After a few months of growth, we're seeing peaks of 0.5% cpu usage
      • After a lot longer, we're seeing about 1% cpu usage
    • Memory usage stays south of 1/2 a GB has grown to just shy of 8 GB is now around 6 GB
      • Not surprising, as we're mostly storing sets of integers and compressed strings.
      • We've also started storing much bigger things in Redis, like inboxes.
      • Newer Redis releases have saved some space, and we've slimmed down some common keys

*Still, for some common use cases the largest part of a Redis command is the key name. One of these days I'm going to find some time to experiment with compressing the entire command stream, and see what kind of performance gains can be made.

**These removals are necessary to invalidate the "L1 Cache," Redis naturally keeps itself in a coherent state in the face of removals.

share|improve this answer
Are your webservers sticky? Are ip's routed to the same web server to which they hit first? if yes, does haproxy manage this? – Cherian Mar 6 '11 at 4:32
Hi , Are you using Redis on windows?? , if yes how stable it is ? Please reply as we are going start high traffic local classified site, and need to use Redis on windows. – Ajax2020 Oct 20 '11 at 15:54
@Ajax2020 - we host Redis on linux (Ubuntu Server specifically), we've never had any stability issues. – Kevin Montrose Oct 20 '11 at 16:02
Can you add details of the interaction between your SQL server and the cache, like how do you populate the cache? Is everything lazy loaded (implying it won't be in the cache if it hasn't been requested)? If so, are there certain pieces/classes of data you explicitly keep seeded? Do you use SQL notifications to keep the cache updated, or do you update the cache using the DAL? – Daniel Schaffer Mar 14 '12 at 21:02
Do you cache pages on client? if you do, How do you invalidate cache for an anonymous user when he becames logged user? If I access a page A as anonymous and then I access page A as logged user, how do you handle that? – StackOverflower May 17 '12 at 14:59
@KevinMontrose : Is there a specific way through which you are currently doing caching for user specific data? . There are so many user accounts, and the data varies from one user to another. I was wondering if Redis provides us with anything to address such scenarios. – boddhisattva Aug 23 '12 at 11:53
@boddhisattva - We cheat a little in that case, our routing is sticky by IP so user sessions can get cached locally for speed. – Kevin Montrose Aug 23 '12 at 16:33
What's the compression algo used for sending stuff to redis? zlib? just curious – Mahn Sep 3 '12 at 16:03
@Mahn - We use GZip, which in .NET 4.5 is backed by zlib. – Kevin Montrose Sep 3 '12 at 16:38
Could you tell please what client library are you using to access Redis from .net environment? – wallice Sep 10 '12 at 8:09
@wallice - Booksleeve was developed in house for our Redis use. – Kevin Montrose Sep 10 '12 at 19:43
I know the question is kinda old, but I'm curious about one thing: You said you avoid cache invalidation and basically let the Redis cache expire by itself. What is the TTL you use? I was familiar with a different approach - long TTLs, but a complex invalidation system, handling invalidation as soon as the actual data changes. What is the better approach? – Przemek May 29 '13 at 15:42
Does anyone know if a custom OutputCache Provider is used to store the output cache in the Redis layer? – Rahul Patel Nov 27 '13 at 8:08
Can you give more color on the L1 cache? Specifically do you restrict the size of the cache explicitly, or implicitly by physical server memory? How do you ensure the right balance of page output caching and data caching takes place? And most importantly, do you have separate app pools for every single site (and subsequently separate L1 cache for each)? – John Zumbrum Sep 11 '15 at 23:30

You must log in to answer this question.