I recently saw this image posted in an answer on Meta by @Nick Craver ♦:
I'm interested to know how much bandwidth (in total GB/TB) gets used on average each month for such a large site, and on which services? (IIS/SQL split + any other servers)
Some quick stats on the past 30 days, the New York data center only (chat runs from Oregon, and we sync databases there...so there's additional bandwidth and such for that).
HTTP Traffic only:
Now that's the outward facing traffic, which I assume is what you're after here. Also note that it excludes a significant figure, the content hosted by our CDN. Here's a few figures from our CDN:
Now none of these includes other bandwidth uses, such as the aforementioned database syncs to other data centers and other VPN traffic. We maintain a VPN mesh between all DCs and offices at all times, but SQL replication, redis slaving and offsite backup copies account for the majority of the bandwidth there. For reference, here's total bandwidth data (only the external interface) from the Cisco 3945 on our primary uplink:
So you can see there we have about 2.2TB of send overhead just to run things, plus another 6TB just for requests.
For those curious what performance looks like these days, here's our database tier:
What they do:
And here's our web tier:
What they do:
The graphs are wacky on the last servers because we're taking them out and upgrading the entire tier to Server 2012 as I write this, and we whipped the monitoring back into shape for this post.
One last item for those still paying attention: Most of the bandwidth is still internal. Over the same 30 days our core switches went through 586,157,817,892,376 bytes of traffic.
If there are other things you're curious about, just ask. We're pretty open about how we run things.