2

I'm curious if SO uses any compression algorithm to reduce storage space of posts?

Or is this an unnecessary overhead since storage is cheap?

Improve this question

3 Answers 3

Reset to default
9

No. Compressing the data in storage would cause a HUGE performance penalty. Hard drive space is way too cheap to save the few pennies you would gain. But the data is gzipped (compressed on the fly) before it is transmitted to increase the speed the data is received.

Improve this answer
5

This answer is a little trickier than Robert implies in his answer.

SQL Server 2008 introduced data compression as a feature in Enterprise Edition. Pages are compressed before they're written to disk automatically, but only for the objects you choose - not just at the table level, but at the index level. I might have a list of users that I keep compressed (because they have huge about-me profiles that compress very well with low CPU overhead) and a list of indexes that I keep uncompressed (because I hit those constantly).

At first glance, it would seem like compression would slow things down, but that's not usually the case with database compression. It turns out that in most cases, we're waiting on disk anyway, and if we can write less to disk by doing some quick CPU calculations, the overall query time will decrease. CPU load goes up, but query time goes down, and SQL Server spends less time switching between tasks waiting for stored data. SQL Server's compression works very well for OLAP style data - huge volumes of data that aren't changed very frequently.

However, one of the areas where it doesn't perform as well is OLTP-style data where pages are constantly changing, like StackOverflow's setup.

If we partitioned off the older data (questions that are stale), we could get a performance boost, but partitioning has its own drawbacks, and I've written about that:

http://www.brentozar.com/archive/2008/06/sql-server-partitioning-not-the-answer-to-everything/

Improve this answer
0

Storage is cheap. Bandwidth isn't. So theirs probably compression of the stream of data somewhere.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .