title: Do some regex on a 50GB file in perl
body: <p>How can I stream a 50GB large file in perl to execute some regex for each line (or chunk)? I have tried with plain vanilla </p>
  
  <pre><code>for $line (&lt;FH&gt;) {
     # do regex
  }
  </code></pre>
  
  <p>I have also tried Tie::File and File::Stream, but perl always tries to load the whole file into memory which is simply not possible.</p>
  
  <pre><code>#!/usr/bin/perl
  
  use IO::Handle;
  use Tie::File;
  use File::Stream;
  #tie @array, 'Tie::File', $ARGV[0] or die "could not open file";
  
  STDOUT-&gt;autoflush(1);
  
  $file=$ARGV[0];
  open(INFO, "&lt; $file") or die("Could not open  file.");
  
  print "opening ... \n";
  my $stream = File::Stream-&gt;new(&lt;INFO&gt;);
  
  #$out = $ARGV[1];
  #open(my $OH, '&gt;', $out) or die "Could not open file '$out' $!";
  print "starting ... \n";
  while (&lt;$stream&gt;)  {
      $line = $_;
      $line =~ s/\n/\[!BR!\]/g;
      $line =~ s/&lt;page&gt;/\n&lt;page&gt;/g;
      $line =~ s/&lt;\/page&gt;/&lt;\/page&gt;\n/g;
      print $line;
  
      #STDOUT-&gt;flush();
  }
  
  close(INFO);
  </code></pre>