title: How to correctly encode responses from an AD search?
body: <p>I search in an MS Active Directory with Python and I stumbled upon an issue with the encoding of an answer I get.</p>
  
  <p>I use the code below to recursively go through the AD</p>
  
  <pre><code>import ldap
  import pickle
  
  class SearchAD():
      def __init__(self):
          self.l = ldap.initialize("ldap://ldap.example.com")
          self.l.protocol_version = ldap.VERSION3
          self.l.set_option(ldap.OPT_REFERRALS, 0)
  
          bind = self.l.simple_bind_s("user", "password")
          self.base = "DC=example,DC=com"
          self.all = list()
  
      def searchmgr(self, criteria, m):
          print criteria
          m += 1
          attributes = ['dn', 'title']
          result = self.l.search_s(self.base, ldap.SCOPE_SUBTREE, u'manager='+criteria, attributes)
          for u in result:
              cn, t = u
              if cn is not None and "Disabled Users" not in cn and t.get('title'):
                  self.all.append({'dn': cn, 'title': t['title'][0], 'm': m})
                  self.searchmgr(cn, m)
  
  
  s = SearchAD()
  s.searchmgr("CN=EXAMPLE Top,DC=example,DC=com", 0)
  with open("dir.pickle", "wb") as f:
      pickle.dump(s.all, f)
  </code></pre>
  
  <p>and get </p>
  
  <blockquote>
    <p>UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position
    16: ordinal not in range(128)</p>
  </blockquote>
  
  <p>when calling</p>
  
  <pre><code>result = self.l.search_s(self.base, ldap.SCOPE_SUBTREE, 'manager='+criteria, attributes)
  </code></pre>
  
  <p>The search works fine (I get the expected output via the <code>print</code>) until the string <code>'CN=DOE Marie-h\xc3\xa9l\xc3\xa8ne,OU=User Accounts,...'</code> which translates to <code>'CN=DOE Marie-hélène,OU=User Accounts,...'</code>.</p>
  
  <p>This is expected as <a href="http://technet.microsoft.com/en-us/library/cc961766.aspx" rel="nofollow">AD returns its results in UTF-8</a></p>
  
  <p>I therefore tried to <code>.encode('utf-8')</code> the string and use</p>
  
  <pre><code>result = self.l.search_s(self.base, ldap.SCOPE_SUBTREE, u'manager='+criteria.encode('utf-8'), attributes)
  </code></pre>
  
  <p>but I get the same error.</p>
  
  <p>How should I handle the result returned by AD?</p>