title: Reading Unicode file data with BOM chars in Python
body: <p>I'm reading a series of source code files using Python and running into a unicode BOM error. Here's my code:</p>
  
  <pre><code>bytes = min(32, os.path.getsize(filename))
  raw = open(filename, 'rb').read(bytes)
  result = chardet.detect(raw)
  encoding = result['encoding']
  
  infile = open(filename, mode, encoding=encoding)
  data = infile.read()
  infile.close()
  
  print(data)
  </code></pre>
  
  <p>As you can see, I'm detecting the encoding using <code>chardet</code>, then reading the file in memory and attempting to print it. The print statement fails on Unicode files containing a BOM with the error: </p>
  
  <blockquote>
    <p>UnicodeEncodeError: 'charmap' codec can't encode characters in position 0-2:<br>
        character maps to &lt;undefined></p>
  </blockquote>
  
  <p>I'm guessing it's trying to decode the BOM using the default character set and it's failing. How do I remove the BOM from the string to prevent this?</p>