Summary
When data is requested with curl, gzip'ed data is returned, I would expect JSON to be returned. You probably want your API to work with curl
without too much effort.
Repro
curl "http://api.stackexchange.com/2.2/questions/3853722/answers?order=desc&sort=activity&site=stackoverflow" | hd
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 644 100 644 0 0 973 0 --:--:-- --:--:-- --:--:-- 974
00000000 1f 8b 08 00 00 00 00 00 04 00 ad 94 5d 6f d3 30 |............]o.0|
00000010 14 86 ff ca 64 b4 5e 85 c6 9f b1 5d 54 21 ae b8 |....d.^....]T!..|
00000020 40 48 dc 20 24 10 8a dc c4 49 4d 9b 8f d9 ce c2 |@H. $....IM.....|
00000030 36 ed 86 ff c6 ff c2 4d 83 d6 6d 4d e9 24 ee 12 |6......M..mM.$..|
00000040 9f e3 d7 e7 9c c7 af ef 80 f1 ba 72 60 f1 ed 0e |...........r`...|
00000050 34 7d ad 2d 58 dc 01 ab db ce 2b 6f 9a 1a 2c 30 |4}.-X.....+o..,0|
00000060 62 11 e8 9c b6 a9 c9 c1 82 52 89 08 81 e3 8a bf |b........R......|
00000070 69 35 58 84 f4 d2 38 af ad ce 41 04 54 96 e9 d6 |i5X...8...A.T...|
00000080 a7 56 f9 10 4a 42 66 6b 9b c2 6c 75 6a 2a 55 ee |.V..JBfk..luj*U.|
00000090 b2 d7 de b7 6e 11 c7 66 ee bc ca 36 73 53 95 9d |....n..f...6sS..|
000000a0 9d 67 4d 15 7f 66 a6 63 f3 1f 6d f9 d6 2d 11 16 |.gM..f.c..m..-..|
000000b0 b3 72 89 82 60 6e 5c bb 55 37 69 ad aa dd f6 2f |.r..`n\.U7i..../|
0
Technical details
This seems to be caused because curl
does not send an Accept-Encoding
heading, and assumes, therefore, that that the response will not be encoding. You send back gzip encoded data together with a Content-Encoding
heading. (Use --verbose
flag to see the headers)
Now, it would appear that it is curl
that is in the wrong; according to section 14.3 of https://www.ietf.org/rfc/rfc2616.txt, if Accept-Encoding
is missing you can do whatever you want. However, I imagine curl
(and old versions of curl) are sufficiently common that you aren't allowed to ignore its bugs.
Interestingly (or perhaps pedantically), you also have the same (and in this case standard-violating) behaviour for an empty Accept-Encoding
heading curl -H 'Accept-Encoding;'
, and an accept encoding that doesn't allow gzip curl -H 'Accept-Encoding: identity'
. Again this behaviour is not technically wrong, but you are ignoring recommendations.
Work around
This behaviour can be worked around by using the --compressed
option.
Possible actions
In an ideal world you might return uncompressed data in these cases, but one appreciates that you might be deliberately forcing clients to support compression.
Another option would be to return 406
errors, as this might be easier to understand. I don't know if this has implications for buggy clients that aren't include Accept-Encodings
.
Alternatively, you could do nothing and refer to this post as a workaround.