Meta Stack Exchange is where users like you discuss bugs, features, and support issues that affect the software powering all 158 Stack Exchange communities.

What is meta?
Here's how it works:
  1. Any Stack Exchange user can ask a question
  2. The community provides support, votes on ideas, and reports bugs
  3. Your voice helps shape the way Stack Exchange operates

Say I send a HTTP request with the following headers*:

GET /api/useranswers.json?userid=XXXXX&page=1&pagesize=50&sort=recent HTTP/1.1  
Accept-Encoding: gzip,deflate  

I get a response with the following headers*:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: application/json; charset=utf-8
Server: Microsoft-IIS/7.0
Date: Fri, 25 Dec 2009 03:56:13 GMT

Note that there's no Content-Encoding in the response headers. The same is true for non "/api/" URLs, like flair. I've confirmed that this behavior is replicated in browsers, so I don't think my method of retrieval is gumming anything up.

All HTML traffic seems to be compressed. JSON is understandably denser than the equivalent HTML, but I'd still expect to see ~25% compression ratios consistently.

So, why is this?

*Omitting some personally identifying information & headers

share|improve this question
Does this answer your question?… – Ladybug Killer Dec 25 '09 at 10:41
No. I'm seeing most/all traffic being GZIP, but just a few omissions (JSON requests). The omissions are curious. I've tried a number of ways of making the request, so I'm 99% sure its not my code screwing up. – Kevin Montrose Dec 26 '09 at 1:58
And by "most/all traffic" I mean "most/all HTML traffic" – Kevin Montrose Dec 27 '09 at 4:01
up vote 4 down vote accepted

Historically, the problem is that when you use IIS 7's built in caching via .NET, it ignores all compression.

This means the first request (returned by the application code) will be gzipped, and all subsequent requests (returned by the built in default ASP.NET / IIS 7 caching) will be pulled from the cache and returned uncompressed.

It's very annoying. We use our own caching to get around it, but there are places where we still use the built-in IIS 7 / ASP.NET caching functionality, and it will behave the way I described.

Edit: OK, we fixed this.

share|improve this answer
Is flair using the IIS caching or not? When GETting one's flair, it seems one does get gzip'd responses from the cache, unless during that cache-period someone requests a non-gzip'd response? After that, the cache seems to return non-gzip'd responses even when enabling gzip again. At least, that's what subsequent clicking of Submit at… tells me? (You might want to replace the user id with something else, to ensure other people are not messing up with your tests.) – Arjan Dec 28 '09 at 13:05

This surely is based on the headers sent by your application, but maybe also uses some server-side caching? If server-side caching is used, then maybe even OTHERS who are requesting the same resource have some effect on the result, if they are not specifying "Accept-Encoding: gzip" for their request? This might especially be true for GET requests, which may be cached. (A server will probably not cache POST results, unless it knows exaclty what the web application is doing.)

On Web-Sniffer, when enabling "Accept-Encoding: gzip" for a "flair" request, I only always seem to get gzip'd results when using POST. But, when using GET the results seem more random, maybe due to caching?

Disabling "Accept-Encoding: gzip" when using GET surely never gets me gzip, but enabling it does not yield gzip until the values for "Cache-Control:" and "Expires:" have expired. When switching from non-gzip to gzip, the response

Cache-Control: public, max-age=3   
Content-Type: text/javascript; charset=utf-8    
Expires: Sun, 27 Dec 2009 14:12:57 GMT  
Last-Modified: Sun, 27 Dec 2009 14:09:57 GMT

...eventually becomes:

Cache-Control: public, max-age=180 
Content-Type: text/javascript; charset=utf-8    
Content-Encoding: gzip  
Expires: Sun, 27 Dec 2009 14:16:05 GMT  
Last-Modified: Sun, 27 Dec 2009 14:13:05 GMT

(Earlier, switching User Agent seemed to yield gzip results, but those were not consistent after all. It seems that switching from POST to GET always gets me gzip for the next request? Sorry, I thought I found consistent behaviour in earlier revisions of this answer.)

share|improve this answer
I definitely assumed this was a problem with my code at first, but after trying Chrome, IE8, my own code, and some optimization sites I'm quite certain something is going on. Never did try POST though, good catch. Regardless, I can't think of a good reason to ever not gzip (ignoring the missing Accept-Encoding case). – Kevin Montrose Dec 27 '09 at 13:49
Results for GET requests can be cached by the browser, and it seems reasonable a web server is doing the same when it's passing requests to a web application. POST should not be cached by a browser, but if the server somehow knows nothing changes then caching would not harm... Of course, when your first GET does specify Accept-Encoding: gzip then there's little reason to expect the cached results to not be compressed... But if caching is in place then one needs to be very sure that all requests specify that header. (Maybe including requests of OTHERS for the same resource?) – Arjan Dec 27 '09 at 13:55
The divergence in behavior based on Content-Type is noteworthy. I've yet to see non-json data returned uncompressed. Its worth noting that S[O,U,F] serve RSS/XML data as html as far as I can tell. – Kevin Montrose Dec 27 '09 at 14:05
While testing, once a non-gzip'd response has been created it is cached for sure, and during the cache-period setting Accept-Encoding: gzip has no effect. So, I guess the question is: why does the server cache a non-gzip'd response for you? – Arjan Dec 27 '09 at 14:19
Also note that the HTML responses have different caching settings in their headers. If I use Web-Sniffer to get a non-gzipped version of this very question… then that does not have an affect on what my browser gets. Even the home page is not really cached, it seems. (At least, not when logged in.) – Arjan Dec 27 '09 at 14:31

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .