title: Python POST request encoding
body: <p>here's the situation, i'm sending POST requests and trying to fetch the response with Python
  problem is that it distorts non latin letters, which doesn't happen when i fetch the same page with direct link (with no search results), but POST requests wont generate a link</p>
  
  <p>here's what i do:</p>
  
  <pre><code>import urllib
  import urllib2
  url = 'http://donelaitis.vdu.lt/main_helper.php?id=4&amp;nr=1_2_11'
  data = 'q=bus&amp;ieskoti=true&amp;lang1=en&amp;lang2=en+-%3E+lt+%28+71813+lygiagre%C4%8Di%C5%B3+sakini%C5%B3+%29&amp;lentele=vertikalus&amp;reg=false&amp;rodyti=dalis&amp;rusiuoti=freq' 
  req = urllib2.Request(url, data)
  response = urllib2.urlopen(req)
  the_page = response.read()
  file = open("pagesource.txt", "w")
  file.write(the_page)
  file.close()
  </code></pre>
  
  <p>whenever i try</p>
  
  <pre><code>thepage = the_page.encode('utf-8')
  </code></pre>
  
  <p>i get this error:</p>
  
  <pre><code>UnicodeDecodeError: 'ascii' codec can't decode byte 0xc5 in position 1008: ordinal not in range(128)
  </code></pre>
  
  <p>whenever i try do change response header Content-Type:text/html;charset=utf-8, i do</p>
  
  <pre><code>response['Content-Type'] = 'text/html;charset=utf-8'
  </code></pre>
  
  <p>i get this error:</p>
  
  <pre><code>AttributeError: addinfourl instance has no attribute '__setitem__'
  </code></pre>
  
  <p>My question: is it possible to edit or remove response or request headers?
               if not, is there another way to solve this problem other that copying source to notepad++ and fixing encoding manually?</p>
  
  <p>i'm new to python and data mining, really hope you'd let me know if i;m doing something wrong</p>
  
  <p>thanks</p>