<p dir="auto">I've been trying out the DigitalOcean API when I encountered the following "problem" with requests (2.6.1):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="token = &quot;xxxxxxxxxxxxxx&quot;
api = &quot;/account/keys&quot;
headers = {&quot;Content-Type&quot;: &quot;application/json&quot;, &quot;Authorization&quot;: &quot;Bearer &quot;+token}
r = requests.get(&quot;https://api.digitalocean.com/v2/&quot;+api, headers=headers, data=data)"><pre class="notranslate"><code class="notranslate">token = "xxxxxxxxxxxxxx"
api = "/account/keys"
headers = {"Content-Type": "application/json", "Authorization": "Bearer "+token}
r = requests.get("https://api.digitalocean.com/v2/"+api, headers=headers, data=data)
</code></pre></div>
<p dir="auto">... results in output for r.content and r.text like...</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03mR\xd9\xae\xa2@\x14\xfc\x95\x9b~\xd5\x19\x96\x06\x81Nn2(^\x14\x95\xc5\x05\xc5\xc9d\xd2t\xb7\n\xb2\xc9\xa6h\xfc\xf7a\xe6y\xce\xc3IUN\xaa\xea\xa1\xce\x0bT\xd5\xe5\xf7\x95u\x15@?_ \xa2\x00)P\x82&lt;\x1c\x82S\x94\x9dYY\x94QV\x03\x04h\x88\xa0\x8c DPA#\x01\x11\x19i&lt;\x12\t\xc2\x1aR%D)\x12\x05D\x05\xa4\xa9(&lt;!\x85\x81!(\x9a0\x89\xc8_\xef^\xde\xa7|++\xfc\xa1\xf73\x86\xf6\x13O\x84\x8e\x88\xd3\x7ft~\xef\xb77\xd5\x85\x8b\x95^\x07\x83\xbd(\xd3\xb9f\xc0\xe7(\xd0\x06\xd0\x89oYP\xe4\xb9\xc2\xf9\xfb\xfc\x10\xedR\xe8u#\xb8.EM.}&gt;\xc6\xce$\x9c\x8d\x0e\xc1\xed:\xad5\xee\xacz!\x9d\xc8d\x99\x19\xb9{\xbc1ek\x8a\xc6\x05\xe3\xb83w\xd3\x91_\xeb{\xb1c\xa4\x11f&gt;]\x96\xf1ym\xca\xde\xf3\xde\x92\xb0\xb1\xa5\xa7\x15\xef\x14\x87\x97\xdb\x96O\x9d\x9d/rT\xa2\xf1\xa2\x13\x85\xbcQ7\xb6\xd9|\xe9\x87|\x99\xa8\xae\xa4\xb1\x95\xd4\x9et%\x0e\xd2\xc3\x02{\xa2\xdb\xf8\xb2\xe9\x06\xe3k\x97\x10\xb9*\x83\x88\x8ef\x92\x15\x84\xba\x91Y\xedT\xf8:[c\xc1\xa5\x0bC\x80\xcb\xbd2\xd8\x13\xc3\xa8!\x966\xdc\xa5\xd8\x1232\xb9\xc5\xca\xa0\xeb\xc6\x9e\x901~\xda\xa9\xee5\xbc\xbcN\xac\xbbAg~\xd6\xb9\xf3\xa2VI\xf2\x9c\x04\x85\xe2\xac\x8e\xcc\x91\xac\xf95\xc8}u\x93\\v\xb6Wp\x0f\x9a?,\x0f\x87\xcb4\x0e\xdc\xaea\x99\xfe\xb5=\xcd\x99S\xb5j\xa9\xdc;\xeex\xff\xfc\xfc y\xd6\xb2G\xc5\xea\x1f\xe7\x14G\xc9w\x92\xa7}?\x19NY\xdf\xcc\xff\x8e\xef_C\x90D\xd9\xb5\x7f\x8a\xd7{\x08RV\xe3\x1e\x81:\xafq\x02\x90\xf0~\xff\x01T\xdb#\x867\x02\x00\x00'"><pre class="notranslate"><code class="notranslate">'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03mR\xd9\xae\xa2@\x14\xfc\x95\x9b~\xd5\x19\x96\x06\x81Nn2(^\x14\x95\xc5\x05\xc5\xc9d\xd2t\xb7\n\xb2\xc9\xa6h\xfc\xf7a\xe6y\xce\xc3IUN\xaa\xea\xa1\xce\x0bT\xd5\xe5\xf7\x95u\x15@?_ \xa2\x00)P\x82&lt;\x1c\x82S\x94\x9dYY\x94QV\x03\x04h\x88\xa0\x8c DPA#\x01\x11\x19i&lt;\x12\t\xc2\x1aR%D)\x12\x05D\x05\xa4\xa9(&lt;!\x85\x81!(\x9a0\x89\xc8_\xef^\xde\xa7|++\xfc\xa1\xf73\x86\xf6\x13O\x84\x8e\x88\xd3\x7ft~\xef\xb77\xd5\x85\x8b\x95^\x07\x83\xbd(\xd3\xb9f\xc0\xe7(\xd0\x06\xd0\x89oYP\xe4\xb9\xc2\xf9\xfb\xfc\x10\xedR\xe8u#\xb8.EM.}&gt;\xc6\xce$\x9c\x8d\x0e\xc1\xed:\xad5\xee\xacz!\x9d\xc8d\x99\x19\xb9{\xbc1ek\x8a\xc6\x05\xe3\xb83w\xd3\x91_\xeb{\xb1c\xa4\x11f&gt;]\x96\xf1ym\xca\xde\xf3\xde\x92\xb0\xb1\xa5\xa7\x15\xef\x14\x87\x97\xdb\x96O\x9d\x9d/rT\xa2\xf1\xa2\x13\x85\xbcQ7\xb6\xd9|\xe9\x87|\x99\xa8\xae\xa4\xb1\x95\xd4\x9et%\x0e\xd2\xc3\x02{\xa2\xdb\xf8\xb2\xe9\x06\xe3k\x97\x10\xb9*\x83\x88\x8ef\x92\x15\x84\xba\x91Y\xedT\xf8:[c\xc1\xa5\x0bC\x80\xcb\xbd2\xd8\x13\xc3\xa8!\x966\xdc\xa5\xd8\x1232\xb9\xc5\xca\xa0\xeb\xc6\x9e\x901~\xda\xa9\xee5\xbc\xbcN\xac\xbbAg~\xd6\xb9\xf3\xa2VI\xf2\x9c\x04\x85\xe2\xac\x8e\xcc\x91\xac\xf95\xc8}u\x93\\v\xb6Wp\x0f\x9a?,\x0f\x87\xcb4\x0e\xdc\xaea\x99\xfe\xb5=\xcd\x99S\xb5j\xa9\xdc;\xeex\xff\xfc\xfc y\xd6\xb2G\xc5\xea\x1f\xe7\x14G\xc9w\x92\xa7}?\x19NY\xdf\xcc\xff\x8e\xef_C\x90D\xd9\xb5\x7f\x8a\xd7{\x08RV\xe3\x1e\x81:\xafq\x02\x90\xf0~\xff\x01T\xdb#\x867\x02\x00\x00'
</code></pre></div>
<p dir="auto">... which has to be ameliorated with...</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import zlib
data = zlib.decompress(r.content, 16+zlib.MAX_WBITS)"><pre class="notranslate"><code class="notranslate">import zlib
data = zlib.decompress(r.content, 16+zlib.MAX_WBITS)
</code></pre></div>
<p dir="auto">... because <code class="notranslate">r.headers['content-encoding']</code> happens to be <code class="notranslate">'gzip'</code>.</p>
<p dir="auto">It may be good to detect gzip and other compressive encodings and proactively decompress it. Or have a default option to decompress (because in some applications, premature decompression is undesirable).</p>
<p dir="auto">But all this seems a bit unusual because the documentation says that "<em>Requests automatically decompresses gzip-encoded responses, and does its best to decode response content to unicode when possible.</em>"</p>
<p dir="auto">Same thing happens on CentOS Linux release 7.0.1406 (Core) (Python Python 2.7.5 via yum; requests 2.6.1) and OS X Yosemite 10.10.2 (Python 2.7.9 via Brew; requests 2.6.1).</p>
<p dir="auto">When using requests on python 2.7.8 ubuntu 14.10 from pypi, the current version 2.6.1 exhibits different behaviour from 2.6.0. Response.content is a gzipped binary string, instead of a decoded unicode string in 2.6.0. Is this change in behaviour intended? Running this script in both 2.6.0 and 2.6.1 shows the difference.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import requests
import gzip
import json
from StringIO import StringIO

resp = requests.get('https://liquidstate.atlassian.net/rest/api/2/serverInfo')
try:
    print 'Reading as JSON'
    print resp.json()
except Exception:
    print 'Failed to load JSON, reading as Gzip'
    print 'Reading as Gzip'
    content = StringIO()
    content.write(resp.content)
    content.seek(0)
    gz = gzip.GzipFile(fileobj=content)
    print json.loads(gz.read())"><pre class="notranslate"><code class="notranslate">import requests
import gzip
import json
from StringIO import StringIO

resp = requests.get('https://liquidstate.atlassian.net/rest/api/2/serverInfo')
try:
    print 'Reading as JSON'
    print resp.json()
except Exception:
    print 'Failed to load JSON, reading as Gzip'
    print 'Reading as Gzip'
    content = StringIO()
    content.write(resp.content)
    content.seek(0)
    gz = gzip.GzipFile(fileobj=content)
    print json.loads(gz.read())
</code></pre></div>
<p dir="auto">Under version 2.6.0:<br>
Reading as JSON<br>
{u'versionNumbers': [6, 5, 0], u'serverTitle': u'JIRA', u'buildDate': u'2015-04-08T00:00:00.000+1000',        u'scmInfo': u'0dc1796b325ae7e85b1dca228885a3417f39097a', u'baseUrl': u'<a href="https://liquidstate.atlassian.net" rel="nofollow">https://liquidstate.atlassian.net</a>', u'buildNumber': 65000, u'version': u'6.5-OD-01-120'}</p>
<p dir="auto">And under version 2.6.1:<br>
Reading as JSON<br>
Failed to load JSON, reading as Gzip<br>
Reading as Gzip<br>
{u'versionNumbers': [6, 5, 0], u'serverTitle': u'JIRA', u'buildDate': u'2015-04-08T00:00:00.000+1000', u'scmInfo': u'0dc1796b325ae7e85b1dca228885a3417f39097a', u'baseUrl': u'<a href="https://liquidstate.atlassian.net" rel="nofollow">https://liquidstate.atlassian.net</a>', u'buildNumber': 65000, u'version': u'6.5-OD-01-120'}</p>