title: how to use pycurl if requested data is sometimes gzipped, sometimes not?
body: <p>I'm doing this to fetch some data:</p>
  
  <pre><code>c = pycurl.Curl()
  c.setopt(pycurl.ENCODING, 'gzip') 
  c.setopt(pycurl.URL, url)
  c.setopt(pycurl.TIMEOUT, 10)   
  c.setopt(pycurl.FOLLOWLOCATION, True)
  
  xml = StringIO()
  
  c.setopt(pycurl.WRITEFUNCTION, xml.write )
  
  c.perform()
  c.close()
  </code></pre>
  
  <p>My urls are typically of this sort:</p>
  
  <pre><code>http://host/path/to/resource-foo.xml
  </code></pre>
  
  <p>Usually I get back 302 pointing to:</p>
  
  <pre><code>http://archive-host/path/to/resource-foo.xml.gz
  </code></pre>
  
  <p>Given that I have set FOLLOWLOCATION, and ENCODING gzip, everything works great.</p>
  
  <p>The problem is, sometimes I have a URL which does not result in a redirect to a gzipped resource.  When this happens, <code>c.perform()</code> throws this error:</p>
  
  <pre><code>pycurl.error: (61, 'Error while processing content unencoding: invalid block type')
  </code></pre>
  
  <p>Which suggests to me that pycurl is trying to gunzip a resource that is not gzipped.</p>
  
  <p>Is there some way I can instruct pycurl to figure out the response encoding, and gunzip or not as appropriate?  I have played around with using different values for <code>ENCODING</code>, but so far no beans.</p>
  
  <p>The pycurl docs seems to be a little lacking.  :/</p>
  
  <p>thx!</p>