title: Are there any gotchas with open(my $f, '<:encoding(UTF-8)', $n)
body: <p>I am having a problem that I am unable to reproduce in a manner suitable for Stackoverflow although it's reproducable in my production environment.</p>
  
  <p>The problem occors in a Perl script that, among others, iterates over a file that looks like so:</p>
  
  <pre><code>abc-4-9|free text, possibly containing non-ascii characters|
  cde-3-8|hällo wörld|
  # comment
  
  xyz-9-1|and so on|
  qrs-2-8|and so forth|
  </code></pre>
  
  <p>I can verify the correctness of the file with this Perl script:</p>
  
  <pre><code>use warnings;
  use strict;
  
  open (my $f, '&lt;:encoding(UTF-8)', 'c:\path\to\file') or die "$!";
  
  while (my $s = &lt;$f&gt;) {
    chomp($s);
    next unless $s;
    next if $s =~ m/^#/;
    $s =~ m!(\w+)-(\d+)-(\d+)\|([^|]*)\|! or die "\n&gt;$s&lt;\n didn't match on line $.";
  }
  
  print "Ok\n";
  close $f;
  </code></pre>
  
  <p>When I run this script, it won't die on line 10 and consequently print <code>Ok</code>.</p>
  
  <p>Now, I use essentially the same construct in a huge Perl script (hence irreproducable for Stackoverflow) and it will die on line 2199 of the input file.</p>
  
  <p>If I change the first line (which is completely unrelated to line 2199) from something like</p>
  
  <pre><code> www-1-1|A line with some words|
  </code></pre>
  
  <p>to</p>
  
  <pre><code> www-1-1|x|
  </code></pre>
  
  <p>the script will process line 2199 (but fail later).</p>
  
  <p>Interestingly, this behaviour was introduced when I changed </p>
  
  <pre><code>open (my $f, '&lt;', 'c:\path\to\file') or die "$!";
  </code></pre>
  
  <p>to</p>
  
  <pre><code>open (my $f, '&lt;:encoding(UTF-8)', 'c:\path\to\file') or die "$!";
  </code></pre>
  
  <p>Without the <code>:encoding(UTF-8)</code> directive, the script does not fail. Of course, I need the encoding directive since the file contains non-ascii characters.</p>
  
  <p>BTW, the same script runs without problems on Linux.</p>
  
  <p>On Windows, where it fails, I use Strawberry Perl 5.24</p>