title: Improper charset encoding using pyodbc / SQL Server
body: <p>I'm accessing a SQL Server 2014 database from python using pyodbc. The problem is that multibyte unicode characters are transferred to the database as single byte characters, and then returned as two unicode characters by the same driver. I'm following up with a concrete example:</p>
  
  <pre><code>def print_hex(s):
      print ":".join("{:02x}".format(ord(c)) for c in s)
  def test_insert_unicode():
      u = User(username=u'\u00fc')
      print_hex(u.username)
      u.save()
      print_hex(User.objects.get(id=u.id).username)
  </code></pre>
  
  <p>and it produces the following output</p>
  
  <pre><code>&lt;type 'unicode'&gt;  fc
  &lt;type 'unicode'&gt;  c3:bc
  </code></pre>
  
  <p>Since c3 and bc are the bytes corresponding to the utf-8 encoding of u'\u00fc', I'm assuming that what happens is that the string is encoded into utf-8 when sent to the database (instead of cp1252, which I'm assuming to be the database's encoding - collation is latin1), then, when returned, the two bytes are each interpreted as a character (since cp1252 is a single character encoding). A select on the inserted value in the database returns indeed the two characters corresponding to codepoints c3 and bc in cp1252.</p>
  
  <p>Database access is done over pyodbc-3.0.7 on python 2.7. Database is SQL Server 2014 running on Windows 8.1, and the odbc connection is set up using SQL Native client 11.0. The same thing happens both when the 'Translate character data' option of the datasource is set and when it isn't.
  Thanks</p>
  
  <p>PS. My settings for the database are as follows, but I can't tell for sure what connection string Django generates from this:</p>
  
  <pre><code>'rx3_mssql': {
      'ENGINE': 'django_pyodbc', 
      'NAME': 'rx3',    # Or path to database file if using sqlite3.
      'USER': 'rx3',    # Not used with sqlite3.
      'PASSWORD': 'rx3',      # Not used with sqlite3.
      'OPTIONS' : {
          'encoding':'utf-8',
          'dsn' : 'mssql-rx3',
          'autocommit' : True,
          'MARS_Connection' : True
          }
  }
  </code></pre>