title: How the open :utf8 pragma works in perl - in relation to CPAN modules?
body: <p>I'm already read thru the next:</p>
  
  <ul>
  <li><a href="http://www.perl.com/pub/2012/04/perlunicook-standard-preamble.html" rel="nofollow noreferrer">Perl Unicode Cookbook</a></li>
  <li><a href="https://stackoverflow.com/questions/14566460/how-differs-the-open-pragma-with-different-utf8">How differs the open pragma with different utf8?</a></li>
  <li><a href="https://stackoverflow.com/questions/16484066/how-to-use-utf8-encode-with-open-pragma">How to use utf8 encode with open pragma</a> </li>
  <li><a href="https://stackoverflow.com/questions/4959384/does-the-autodie-pragma-have-influence-on-the-encoding">Does the autodie-pragma have influence on the encoding?</a></li>
  <li>many others... (like: perlunicode perluniintro, perlunifaq, perlunitut)... ;(</li>
  <li>and of course tchrist's geniue answer <a href="https://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default/6163129#6163129">Why does modern Perl avoid UTF-8 by default?</a></li>
  </ul>
  
  <p>but probably missed some BASIC points.</p>
  
  <p>Using the</p>
  
  <pre><code>use open(:utf8);
  </code></pre>
  
  <p>Affects cpan modules too? E.g. when some CPAN module opens any file, it will be opened with <code>:utf8</code>?  Is this statement TRUE? (or the open pragma is only lexically scoped?) AFAIK - it affects modules too, but in "inconsistent" way.. (probably it is a problem of the modules).</p>
  
  <p>Have the <code>open pragma</code> effect to <code>opendir</code>? - what i already tried - <strong>no</strong> - i still need extra <code>decode</code> on all filenames coming from <code>readdir</code> (in addition to NFC). So, IO::Dir is something different - what open pragma doesn't covers?</p>
  
  <p>Affect the open pragma <code>sockets</code>, pipes too? (e.g. anything what is a sort of IO::Handle ?)</p>
  
  <p>All (or most) CPAN modules knows when doing i/o how they need to do it (utf8 or lattin1 or raw?) (probably not, because a simple <code>autodie</code> doesn't works with the <code>open</code> pragma... :()</p>
  
  <p>In many places I can read a similar rule: <em>Remember the canonical rule of Unicode: always encode/decode at the edges of your application.</em> This is nice rule - but the application edge mean: my own source code. CPAN modules are (usually) <em>behind the edge</em> too - not only the "outer world", like system or network...</p>
  
  <p>From my experiance, 3/4 of the content my short scripts (what heavily uses CPAN)  contains: top declarations, and dozens of encode/decode/NFC for nearly everything...</p>
  
  <p>E.g.:
  Even logging utilities, need explicit encoding:</p>
  
  <pre><code>use Log::Any qw($log);
  use Log::Any::Adapter ('File', 'file.log');
  $log-&gt;error( encode('utf-8', "tökös"));
  </code></pre>
  
  <p>Even, when want add <code>tie</code> to my code, need replace every <code>$key</code> <code>$value</code> with encoded versions.</p>
  
  <p><strong>Is this true, or i missed some really basic point in the all above doccu?</strong></p>
  
  <p>Some CPAN module handling utf8 (inside) like, JSON::XS, YAML::XS, File::Slurp.. (altough never succeeded get correct "things" from YAML::XS, pure YAML and JSON::XS works without any problems...</p>
  
  <p>For some modules exists "hacks" - like <code>DBIx::Class::ForceUTF8</code>, <code>Template::Stash::ForceUTF8</code>, <code>HTML::FillInForm::ForceUTF8</code> - and so, - what doesn't allow write correct application for "both" utf and non-utf world... ;(</p>
  
  <p>Many CPAN modules doesn't calls internally the above 'hacked variants' -  (e.g. <code>HTML::FillInForm::ForceUTF8</code>) but only the simple-one, so it is impossible to use them correctly with utf8...  Others, silently fail.. ;(</p>
  
  <p>Plack application doesn't handles utf8 <em>logging messages</em> without the annoying "Wide character...." ;( /modern perl :(/ and can continue ;(</p>
  
  <p>From the above I "deducted" (probably wrongly) - than i <strong>MUST</strong> know and remember for every CPAN module how it is handling utf8 encoded strings and because nowhere is some "registry" - is is mostly trial/error based.</p>
  
  <p><strong>So the main question is:</strong></p>
  
  <p>While i remembering: <em>Here is no magic bullet</em>, <strong>but is here some good way <em>how detect</em> and know "<em>utf8 ready CPAN modules</em>" what doesn't need special encode/decode before using them?</strong> </p>
  
  <p>If someone need to know, i'm using the next in my every script:</p>
  
  <pre><code>use 5.014;
  use warnings;
  use utf8;
  use feature qw(unicode_strings);
  use charnames qw(:full);
  use open(:utf8); #this sometimes is bad, so using only open qw(:std :utf8);
  use Encode qw(encode decode);
  use Unicode::Normalize qw(NFD NFC);
  </code></pre>
  
  <p>Hm.. just "discovered" the <code>utf8:all</code> perl module what replace the <code>readdir</code> with version doing decode.</p>