title: Howto correct set the regex to substitute multiline variable placeholder with perl
body: <p>I'm writing complex configuration files for a data transformation tool with Perl 5.20.</p>
  <p>The configuration file has placeholders for several aspects at load time and runtime encapsulating some <a href="https://metacpan.org/pod/Data::DPath" rel="nofollow noreferrer">path code</a> like</p>
  <pre><code># Load time placeholder example
  CONFIG: NAME ${/path/pos/*/test/*[123] == 'ABC' }
  
  # Runtime placeholder example
  COLUMN: CSV_NAME STRING DEFAULT :{./CSV_FIRST} 
  </code></pre>
  <p>For some reason it should work also on multi line expressions.</p>
  <p>I've written scanner using <code>Text::ParseWord</code>, the standard separator <code>\s+</code> and want to escape the placeholder expressions before splitting the data line(s) into single words by <code>base64</code> encoded expressions not containing <code>\s+</code>. The expressions are also the keys for the subsequent data substitution.</p>
  <p>The escaping is driven by a pattern match defined by (...see code below):</p>
  <pre class="lang-perl prettyprint-override"><code> my @pat = $line =~ /([^\\]\Q$pfx\E\{[^\Q$pfx\E\{\}]+\})/gs;
  </code></pre>
  <p>Which IMO defines a multi line pattern <code>${...}</code> when I use <code>$pfx = '$'</code> for example, but masks (escapes) <code>\${...}</code> expressions.</p>
  <h3>Question</h3>
  <p>I'm struggling a while with the inner part of the pattern <code>...</code> and get <code>[^\Q$pfx\E\{\}]+</code> to work, but have the feeling that is not the correct one, because</p>
  <ol>
  <li>it includes only the symbols set not to use,</li>
  <li>but not the sequence of the outer part,</li>
  <li>to prevent nested expressions for example.</li>
  </ol>
  <p>What is the right expression to do this?</p>
  <h2>Test routine</h2>
  <pre class="lang-perl prettyprint-override"><code>#!/usr/bin/env perl
  use strict;
  use warnings;
  use MIME::Base64;
  
  use feature qw(signatures);
  no warnings 'once';
  no warnings 'experimental';
  no warnings 'experimental::signatures';
  
  my $line =
  '# test data
   This are:
   1. ${/multiline/used/*[3]
        = &quot;12345&quot;}
   2. ${/single/line/compile/time/pattern/*[3]}
   3. ${/single/line/runtime/pattern/x == 1234}
   4. ${/multi/line/runtime/pattern \
        defer/1 \
        defer/2 \
        defer/3
       }
   5. ${//PG.GRM/*[
          key eq &quot;TEST.VAR&quot;
         ]}
  ';
  
  sub testSpacedPlaceHolder($pfx, $line) {
      my %match;
      
      my @pat = $line =~ /([^\\]\Q$pfx\E\{[^\Q$pfx\E\{\}]+\})/gs;
      my %seen = ();
      my @uniq = grep { ! $seen{$_} ++ } @pat;    
      for my $key (@uniq) {
          my $hkey=$pfx.encode_base64($key);
          $hkey =~ s/\n//g;
          my $var = substr($key, 3, -1);
          $match{$hkey}= [ $var, $key ];
          $line =~ s/\Q$key\E/$hkey/g;
      }
      # Test the output ------------------------------
      my $cnt = 0;
      print &quot;\nRESULT:\n&quot;;
      for my $key (sort keys %match) {
          $cnt++;
          my ($var, $orig) = @ { $match{$key} };
          print &quot;---- $cnt ----\n&quot;;
          print &quot;ORG: $orig\n&quot;;
          print &quot;VAR: $var\n&quot;;
          print &quot;ESC: $key\n&quot;;
  
      }
      print &quot;\nLINE:\n$line\n&quot;;
      return ($line, \%match);
  }
  
  testSpacedPlaceHolder('$', $line);
  </code></pre>
  <h2>Result</h2>
  <pre><code>/usr/bin/env perl &quot;test-strings.pl&quot;
  
  RESULT:
  ---- 1 ----
  ORG:  ${/multi/line/runtime/pattern \
        defer/1 \
        defer/2 \
        defer/3
       }
  VAR: /multi/line/runtime/pattern \
        defer/1 \
        defer/2 \
        defer/3
       
  ESC: $ICR7L211bHRpL2xpbmUvcnVudGltZS9wYXR0ZXJuIFwKICAgICAgZGVmZXIvMSBcCiAgICAgIGRlZmVyLzIgXAogICAgICBkZWZlci8zCiAgICAgfQ==
  ---- 2 ----
  ORG:  ${/multiline/used/*[3]
        = &quot;12345&quot;}
  VAR: /multiline/used/*[3]
        = &quot;12345&quot;
  ESC: $ICR7L211bHRpbGluZS91c2VkLypbM10KICAgICAgPSAiMTIzNDUifQ==
  ---- 3 ----
  ORG:  ${/single/line/compile/time/pattern/*[3]}
  VAR: /single/line/compile/time/pattern/*[3]
  ESC: $ICR7L3NpbmdsZS9saW5lL2NvbXBpbGUvdGltZS9wYXR0ZXJuLypbM119
  ---- 4 ----
  ORG:  ${/single/line/runtime/pattern/x == 1234}
  VAR: /single/line/runtime/pattern/x == 1234
  ESC: $ICR7L3NpbmdsZS9saW5lL3J1bnRpbWUvcGF0dGVybi94ID09IDEyMzR9
  ---- 5 ----
  ORG:  ${//PG.GRM/*[
          key eq &quot;TEST.VAR&quot;
         ]}
  VAR: //PG.GRM/*[
          key eq &quot;TEST.VAR&quot;
         ]
  ESC: $ICR7Ly9QRy5HUk0vKlsKICAgICAgICBrZXkgZXEgIlRFU1QuVkFSIgogICAgICAgXX0=
  
  LINE:
  # test data
   This are:
   1.$ICR7L211bHRpbGluZS91c2VkLypbM10KICAgICAgPSAiMTIzNDUifQ==
   2.$ICR7L3NpbmdsZS9saW5lL2NvbXBpbGUvdGltZS9wYXR0ZXJuLypbM119
   3.$ICR7L3NpbmdsZS9saW5lL3J1bnRpbWUvcGF0dGVybi94ID09IDEyMzR9
   4.$ICR7L211bHRpL2xpbmUvcnVudGltZS9wYXR0ZXJuIFwKICAgICAgZGVmZXIvMSBcCiAgICAgIGRlZmVyLzIgXAogICAgICBkZWZlci8zCiAgICAgfQ==
   5.$ICR7Ly9QRy5HUk0vKlsKICAgICAgICBrZXkgZXEgIlRFU1QuVkFSIgogICAgICAgXX0=
  </code></pre>
  <h2>Edit</h2>
  <p>Assuming I have a script that defines some kind of configuration:</p>
  <pre><code>MAGIC: MAGIC.TYPE
  
  CONTAINER: NAME BEGIN
  
  DEFINE: VAR1 'USER.NAME'
  DEFINE: VAR2 '65789'
  
  INTERNAL.CONTAINER: INTERNAL.NAME BEGIN
  
      TAG1: 'ABCDEF'
      TAG2: ${/NAME/VAR1}
  
      # Unwanted nested variant 
      TAG3: ${/NAME/VAR1 ${/NAME/VAR2} }
  
      # Valid runtime interpolation variant
      TAG4: &quot;${/NAME/VAR1}/:{NAME.KEY}&quot;
      
      # Valid runtime path variant but ignored
      TAG5: ${/NAME/VAR1/*/:{TEST{KEY}}
  
  END.INTERNAL.NAME
   
  END.NAME 
  </code></pre>
  <p>I want to avoid the nested line</p>
  <pre class="lang-perl prettyprint-override"><code>    # Nested variant 
      TAG3: ${/NAME/VAR1 ${/NAME/VAR2}}
  </code></pre>
  <p>for variable resolve reasons, but keep</p>
  <pre class="lang-perl prettyprint-override"><code>    # Valid runtime path variant but ignored
      TAG5: ${/NAME/VAR2/*/:{TEST{KEY}}
  </code></pre>
  <p>because they are runtime driven.</p>
  <p>My variant blocks TAG5 due to the simple sequence <code>[\$\{\}]+</code>.</p>