title: Opening a text file, and receiving a encoding error, tried multiple methods no hope
body: <p>I'm trying to open up a password database file (consists of a bunch of common passwords) and I'm getting the following error:</p>
  
  <p>Attempts so far..
  Code:</p>
  
  <pre><code>f = open("crackstation-human-only.txt", 'r')
  
  for i in f:
      print(i)
  </code></pre>
  
  <p>Error Code:</p>
  
  <pre><code>Traceback (most recent call last):
    File "C:\Users\David\eclipse-workspace\Kaplin\password_cracker.py", line 3, in &lt;module&gt;
      for i in f:
    File "C:\Users\David\AppData\Local\Programs\Python\Python37\lib\encodings\cp1252.py", line 23, in decode
      return codecs.charmap_decode(input,self.errors,decoding_table)[0]
  UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 753: character maps to &lt;undefined&gt;
  </code></pre>
  
  <p>After doing some research I was told to attempt <code>encoding = 'utf-8'</code> which I later discovered was basically guessing and hoping that the file would show all the outputs</p>
  
  <p>Code:</p>
  
  <pre><code>f = open("crackstation-human-only.txt", 'r', encoding = 'utf-8')
  
  for i in f:
      print(i)
  </code></pre>
  
  <p>Error:</p>
  
  <pre><code>Traceback (most recent call last):
    File "C:\Users\David\eclipse-workspace\Kaplin\password_cracker.py", line 3, in &lt;module&gt;
      for i in f:
    File "C:\Users\David\AppData\Local\Programs\Python\Python37\lib\codecs.py", line 322, in decode
      (result, consumed) = self._buffer_decode(data, self.errors, final)
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 5884: invalid continuation byte
  </code></pre>
  
  <p>After receiving this error message, I was recommended to attempt to download a text editor like 'Sublime Text 3', and to open the console end enter the command 'Encoding()', but unfortunately it wasn't able to detect the encoding.</p>
  
  <p>My professor was able to use bash to 'grep cat' the lines in the file (I honestly know very little about bash so if anyone else knows those terms i'm not sure if that will help them out)</p>
  
  <p>If anyone has any suggestions on what I can do in order to get this to work out I would greatly appreciate it.</p>
  
  <p>I will post the link to the text document if anyone is interested in seeing what types of characters are within the file.</p>
  
  <p><a href="http://context-aware.sandbox.csun.edu/~akaplan/425/crackstation-human-only.txt" rel="nofollow noreferrer">Link to the file, it's a .txt from my school/professors domain</a></p>
  
  <p>UPDATE:</p>
  
  <p>I have a fellow classmate that is running elementary OS, and he was using the terminal to write his python program which would iterate through the file, and he was using the encoding 'latin-1', he was able to output more characters than me, I'm on Windows 10, using Eclipse-atom for all my scripts.</p>
  
  <p>So there seems to be something that's causing me possibly not to get the correct outputs based on these factors, i'm guessing because it just seems that way based on the results, </p>
  
  <p>I will be installing elementary-os and attempting all the solutions there, to see if I can get this file to work out. I'll add another update soon!</p>