title: UnicodeEncodeError when reading a file
body: <p>I am trying to read from rockyou wordlist and write all words that are >= 8 chars to a new file.</p>
  
  <p>Here is the code - </p>
  
  <pre><code>def main():
      with open("rockyou.txt", encoding="utf8") as in_file, open('rockout.txt', 'w') as out_file:
          for line in in_file:
              if len(line.rstrip()) &lt; 8:
                  continue
              print(line, file = out_file, end = '')
          print("done")
  
  if __name__ == '__main__':
      main()
  </code></pre>
  
  <p>Some words are not utf-8.</p>
  
  <p><code>Traceback (most recent call last):
    File "wpa_rock.py", line 10, in &lt;module&gt;
      main()
    File "wpa_rock.py", line 6, in main
      print(line, file = out_file, end = '')
    File "C:\Python\lib\encodings\cp1252.py", line 19, in encode
      return codecs.charmap_encode(input,self.errors,encoding_table)[0]
  UnicodeEncodeError: 'charmap' codec can't encode character '\u0e45' in position
  0: character maps to &lt;undefined&gt;</code></p>
  
  <p><strong>Update</strong></p>
  
  <pre><code>def main():
  with open("rockyou.txt", encoding="utf8") as in_file, open('rockout.txt', 'w', encoding="utf8") as out_file:
      for line in in_file:
          if len(line.rstrip()) &lt; 8:
              continue
          out_file.write(line)
      print("done")
  
  if __name__ == '__main__':
      main()```
  </code></pre>
  
  <p><code>Traceback (most recent call last):
    File "wpa_rock.py", line 10, in &lt;module&gt;
      main()
    File "wpa_rock.py", line 3, in main
      for line in in_file:
    File "C:\Python\lib\codecs.py", line 321, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xf1 in position 933: invali
  d continuation byte</code></p>