What is meta? ×
Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 127 Stack Exchange communities.

The following script will tell you that you could be reducing the size of your pages by 10-13% if you chose to get rid of superfluous white space and newlines.

import urllib2
SITES = ["stackoverflow.com",
    # put all your sites domains here...

stats = []
for site in SITES:
    stream = urllib2.urlopen("http://"+site).read()
    before = len(stream)/1024.0
    after = len(" ".join(stream.strip().split()))/1024.0
    savings = before-after
    stats.append((site, before, after, savings, savings/before*100))
    print "{0:<30} {1:>5.1f} {2:>5.1f} {3:>5.1f} {4:>4.1f}%".format(*stats[-1])

The results:

stackoverflow.com              194.2 169.6  24.6 12.7%
meta.stackoverflow.com         124.6 109.1  15.4 12.4%
superuser.com                  120.9 105.9  15.0 12.4%
serverfault.com                121.0 106.0  15.0 12.4%
stackapps.com                  122.4 102.5  19.9 16.3%
careers.stackoverflow.com       15.1  12.6   2.4 16.2%
cooking.stackexchange.com      119.9 104.6  15.2 12.7%
area51.stackexchange.com        31.0  24.6   6.4 20.6%
webapps.stackexchange.com      118.2 103.0  15.2 12.9%
gaming.stackexchange.com       117.7 102.5  15.2 12.9%
ubuntu.stackexchange.com       125.5 108.2  17.3 13.8%
webmasters.stackexchange.com   117.9 102.7  15.2 12.9%
gamedev.stackexchange.com      117.8 103.0  14.8 12.6%
math.stackexchange.com         118.9 103.5  15.4 13.0%
photography.stackexchange.com  132.8 115.5  17.3 13.1%
stats.stackexchange.com        130.0 112.6  17.4 13.4%
tex.stackexchange.com          120.4 105.1  15.4 12.8%
english.stackexchange.com      119.6 104.1  15.4 12.9%
cstheory.stackexchange.com     121.8 106.9  14.9 12.2%
programmers.stackexchange.com  117.9 103.2  14.8 12.5%
unix.stackexchange.com         116.7 101.4  15.3 13.1%
apple.stackexchange.com        119.5 104.1  15.4 12.9%
wordpress.stackexchange.com    126.3 109.0  17.4 13.7%
physics.stackexchange.com      120.7 105.4  15.3 12.7%
diy.stackexchange.com          119.4 104.2  15.2 12.8%
gis.stackexchange.com          127.4 110.1  17.3 13.6%
electronics.stackexchange.com  124.8 107.4  17.4 14.0%
android.stackexchange.com      125.7 108.5  17.3 13.7%
security.stackexchange.com     127.8 110.4  17.4 13.6%

Considering that you must get tens of millions of hits a month, probably this can translate to real bandwidth savings or improved response times. Now, I am aware that the time it takes to get rid of the newlines and whitespace may be longer that actually rendering the page, but probably you can improve your templates so that they are served with less superfluous stuff.

Does this make sense or is it akin to polishing a cannon ball?


Thanks for the answers. Conclusion: it is like polishing a cannon ball.

share|improve this question
IIRC, I don't think they are even close to saturating their pipe, so right now there's no need. –  TheLQ Jul 31 '11 at 6:35
Not really sure if this covers what you're suggesting, but this might be relevant meta.stackexchange.com/questions/15394/… –  Brandon Jul 31 '11 at 6:35
Your numbers are meaningless when you measure post-decompression, and only one page. –  balpha Jul 31 '11 at 6:53
add comment

2 Answers

up vote 16 down vote accepted

Per some tests I did a long time ago:


Realistically, whitespace and linefeed removal is doing work that the compression would be doing for us. We're just adding a dab of human-assisted efficiency:

                 Raw              Compressed
Unoptimized CSS  2,299 bytes      671 bytes
Optimized CSS    1,758 bytes      615 bytes

It's only about a 10 percent savings once you factor in HTTP compression.

Personally I find that ...

  • destroying the readability of view source
  • the potential performance penalty of running this real time on every page output
  • the risk of getting the whitespace/linefeed removal wrong and causing invalid HTML

... is not worth a measly 10% reduction in bytes over the wire.

It really feels like a micro-optimization since gzip gets you 90% of the way there.

share|improve this answer
Fun fact: for bug hunting, I once tried to get a compressed JavaScript file to be an exact multiple of equal-sized TCP/IP packets. Removing a few random parts of the JavaScript then often actually increased the total gzip'd size. (Just saying that the compression is really smart in finding duplicated content, which might even be more true for generated HTML than for CSS.) –  Arjan Jul 31 '11 at 12:45
For your second point, you don't have to run this in real time. You could remove the spaces from the templates and static files as an extra build step. That said, I agree that it's probably not worth the loss of readability. –  hammar Jul 31 '11 at 16:57
@hammar True, but most pages aren't going to be static –  Michael Mrozek Jul 31 '11 at 18:44
@Michael: No, but they are presumably generated from various templates, and most of the extra spaces are coming from them. –  hammar Jul 31 '11 at 18:59
add comment

You're neglecting gzip in your scenario it appears. Once you factor it in, you're only going to see savings of 10% or so over already gzip'd content (the savings of gzip vs. ungzip'd is something like 70%, for comparison). This is based on just looking at the home page, I suspect the savings are just as lean (if not leaner) on the rest of the sites' pages.

Weigh this against the implementation of stripping out white space.

You've basically got to re-parse the HTML (either as you're writing it out, or after the whole page has been rendered) to figure out what white space is truly safe to strip. You've got make sure this parsing is fast enough to not affect page load times (which we're very serious about), and robust enough to deal with anything a person might manually enter into a post.

It's an expensive proposition, for not a lot of gain, basically.

share|improve this answer
add comment

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .