title: Python UTF-16 CSV reader
body: <p>I have a UTF-16 CSV file which I have to read. Python csv module does not seem to support UTF-16.</p>
  
  <p>I am using python 2.7.2. CSV files I need to parse are huge size running into several GBs of data.</p>
  
  <p>Answers for John Machin questions below</p>
  
  <pre><code>print repr(open('test.csv', 'rb').read(100))
  </code></pre>
  
  <p>Output with test.csv having just abc as content</p>
  
  <pre><code>'\xff\xfea\x00b\x00c\x00'
  </code></pre>
  
  <p>I think csv file got created on windows machine in USA. I am using Mac OSX Lion.</p>
  
  <p>If I use code provided by phihag and test.csv containing one record.</p>
  
  <p>sample test.csv content used. Below is print repr(open('test.csv', 'rb').read(1000)) output</p>
  
  <pre><code>'\xff\xfe1\x00,\x002\x00,\x00G\x00,\x00S\x00,\x00H\x00 \x00f\x00\xfc\x00r\x00 \x00e\x00 \x00\x96\x00 \x00m\x00 \x00\x85\x00,\x00,\x00I\x00\r\x00\n\x00'
  </code></pre>
  
  <p>Code by phihag</p>
  
  <pre><code>import codecs
  import csv
  with open('test.csv','rb') as f:
        sr = codecs.StreamRecoder(f,codecs.getencoder('utf-8'),codecs.getdecoder('utf-8'),codecs.getreader('utf-16'),codecs.getwriter('utf-16'))      
        for row in csv.reader(sr):
           print row
  </code></pre>
  
  <p>Output of the above code</p>
  
  <pre><code>['1', '2', 'G', 'S', 'H f\xc3\xbcr e \xc2\x96 m \xc2\x85']
  ['', '', 'I']
  </code></pre>
  
  <p>expected output is</p>
  
  <pre><code>['1', '2', 'G', 'S', 'H f\xc3\xbcr e \xc2\x96 m \xc2\x85','','I']
  </code></pre>