title: Unexpected results from Path.read_text (of pathlib) when reading utf-8 encoded file
body: <p>Today I learned that for <code>open(filename).read()</code> we cannot expect that the resources bound to the hidden file object are given back immediately, although I observed this on my system. (See <a href="https://stackoverflow.com/a/7409814/2932052">the accepted answer</a> of the question <a href="https://stackoverflow.com/q/7409780/2932052">Does reading an entire file leave the file handle open?</a>).</p>
  <p><a href="https://stackoverflow.com/a/39591569/2932052">The second answer</a> made me resist to roll my own helper function, it told me that <code>pathlib</code> already offers exactly this function.</p>
  <p>But actually, this seems not to be the case. With the following script (<code>test.py</code>), I get different results:</p>
  <pre><code># The German accent characters are Ä,Ö,Ü,ä,ö,ü, and ß.
  from pathlib import Path;
  
  def pathlib_read_text(filename, encoding=None):
      return Path(filename, encoding=encoding).read_text()
  
  def mylocal_read_text(filename, encoding=None):
      with open(filename, encoding=encoding) as f:
          return f.read()
  
  def test(fun):
      print(fun+'_read_text:')
      print(eval(fun+'_read_text')(__file__, 'utf-8'))
  
  test('pathlib')
  test('mylocal')
  </code></pre>
  <p>The output to the Windows console (<code>python test.py</code>) contains <code>Ã&quot;,Ã-,Ão,Ã¤,Ã¶,Ã¼, and ÃY.</code> in the first block, when I redirect the output into a file, I get the second block wrong (In Notepad++ it's displayed <code>xC4,xD6,xDC,xE4,xF6,FC, and xDF</code> in White on Black) if the file is treated as utf-8.</p>
  <p>Is there anything I overlooked?</p>
  <p>I tried to examine the 3.6.3 code, but found no bug so far ...</p>
  <ul>
  <li><a href="https://github.com/python/cpython/blob/2c5fed86e0cbba5a4e34792b0083128ce659909d/Lib/pathlib.py#L1170" rel="nofollow noreferrer">read_text (1170)</a></li>
  <li><a href="https://github.com/python/cpython/blob/2c5fed86e0cbba5a4e34792b0083128ce659909d/Lib/pathlib.py#L1152" rel="nofollow noreferrer">open (1152)</a></li>
  </ul>
  <h2>Edit</h2>
  <p>The following version reinforces my feeling that it's a bug in <code>pathlib</code> or in one of the underlying libraries/functions. Maybe it's only a Windows issue, where the default encoding is mostly different from <code>utf-8</code>. Now it's sufficient to run the test in a console window.</p>
  <pre><code>accents = '''
  Ä,Ö,Ü,ä,ö,ü,ß
  '''
  from pathlib import Path;
  import codecs
  
  def pathlib_read_text(filename, encoding=None, errors=None):
      return Path(filename, encoding=encoding, errors=errors).read_text()
  
  def mylocal_read_text(filename, encoding=None, errors=None):
      with open(filename, encoding=encoding, errors=errors) as f:
          return f.read()
  
  def space_it(error):
      return ' ';
  codecs.register_error('space_it', space_it)
  
  def test(fun):
      s = eval(fun+'_read_text')(__file__, 'utf-8', errors='space_it')
      print(fun+'_read_text:', s.split(&quot;\n&quot;)[1] == accents.strip())
  
  test('pathlib')
  test('mylocal')
  </code></pre>
  <p>It produces the following output:</p>
  <pre class="lang-none prettyprint-override"><code>pathlib_read_text: False
  mylocal_read_text: True
  </code></pre>