title: Unicode to UTF8 for CSV Files - Python via xlrd
body: <p>I'm trying to translate an Excel spreadsheet to CSV using the Python xlrd and csv modules, but am getting hung up on encoding issues. Xlrd produces output from Excel in Unicode, and the CSV module requires UTF-8.</p>
  
  <p>I imaging that this has nothing to do with the xlrd module: everything works fine outputing to stdout or other outputs that don't require a specific encoding.</p>
  
  <p>The worksheet is encoded as UTF-16-LE, according to <code>book.encoding</code></p>
  
  <p>The simplified version of what I'm doing is:</p>
  
  <pre><code>from xlrd import *
  import csv
  b = open_workbook('file.xls')
  s = b.sheet_by_name('Export')
  bc = open('file.csv','w')
  bcw = csv.writer(bc,csv.excel,b.encoding)
  for row in range(s.nrows):
      this_row = []
      for col in range(s.ncols):
          this_row.append(s.cell_value(row,col))
      bcw.writerow(this_row)
  </code></pre>
  
  <p>This produces the following error, about 740 lines in:</p>
  
  <pre><code>UnicodeEncodeError: 'ascii' codec can't encode character u'\xed' in position 5: ordinal not in range(128)
  </code></pre>
  
  <p>The value is seems to be getting hung up on is "516-777316" -- the text in the original Excel sheet is "516-7773167" (with a 7 on the end)</p>
  
  <p>I'll be the first to admit that I have only a vague sense of how character encoding works, so most of what I've tried so far are various fumbling permutations of <code>.encode</code> and <code>.decode</code> on the <code>s.cell_value(row,col)</code></p>
  
  <p>If someone could suggest a solution I would appreciate it -- even better if you could provide an explanation of what's not working and why, so that I can more easily debug these problems myself in the future.</p>
  
  <p>Thanks in advance!</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>Thanks for the comments so far.</p>
  
  <p>When I user <code>this_row.append(s.cell(row,col))</code> (e.g. s.cell instead of s.cell_value) the entire document writes without errors. </p>
  
  <p>The output isn't particularly desirable (<code>text:u'516-7773167'</code>), but it avoids the error even though the offending characters are still in the output. </p>
  
  <p>This makes me think that the challenge might be in xlrd after all. </p>
  
  <p>Thoughts?</p>