title: Memory leak in Powershell 3.0 script to monitor log file
body: <p>I've written a script in Powershell 3.0 to monitor a log file for specific errors.  The script starts a background process, which monitors the file.  When anything gets written to the file, the background process simply passes it to the foreground process, if it matches the proper format (a datestamped line).  The foreground process then counts the number of errors.</p>
  
  <p>Everything works correctly with no errors.  The issue is that, as the source logfile grows in size, the memory consumed by Powershell increases dramatically.  These logs are capped at ~24M before they are rotated, which amounts to ~250K lines.  In my tests, by the time the log size reaches ~80K lines or so, the monitor process is consuming 250M RAM (foreground and background processes combined. They're consuming ~70M combined when they first start.  This type of growth is unacceptable in our environment.  What can I do to decrease this?</p>
  
  <p>Here's the script:</p>
  
  <pre><code># Constants.
  $F_IN = "C:\Temp\test.log"
  $RE = "^\d+-\d+-\d+ \d+:\d+:\d+,.+ERROR.+Foo$"
  $MAX_RESTARTS = 3  # Max restarts for failed background job.
  $SLEEP_DELAY = 60  # In seconds.
  
  # Background job.
  $SCRIPT_BLOCK = { param($f, $r)
                    Get-Content -Path $f -Tail 0 -Wait -EA SilentlyContinue `
                    | Where { $_ -match $r }
                  }
  
  function Start-FileMonitor {
      Param([parameter(Mandatory=$true,Position=0)][alias("f")]
              [String]$file,
            [parameter(Mandatory=$true,Position=1)][alias("b")]
              [ScriptBlock]$SCRIPT_BLOCK,
            [parameter(Mandatory=$true,Position=2)][alias("re","r")]
              [String]$regex)
  
      $j = Start-Job -ScriptBlock $SCRIPT_BLOCK -Arg $file,$regex
      return $j
  }
  
  
  
  function main {
      # Tail log file in the background, return any errors.
      $job = Start-FileMonitor -b $SCRIPT_BLOCK -f $F_IN -r $RE
      $restarts = 0  # Current number of restarts.
  
      # Poll background $job every $SLEEP_DELAY seconds.
      While ($true) {
          $a = (Receive-Job $job | Measure-Object)
          If ($job.JobStateInfo.State -eq "Running") {
              $restarts = 0
              If ($a.Count -gt 0) {
                  $t0 = $a.Count
                  Write-Host "Error Count: ${t0}"
              }
          }
  
          Else {
              If ($restarts -lt $MAX_RESTARTS) {
                  $job = Start-FileMonitor -b $SCRIPT_BLOCK -f $F_IN -r $RE
                  $restarts++
                  Write-Host "Background job not running.  Attempted restart ${restarts}."
              }
  
              Else {
                  Write-Host "`$MAX_RESTARTS (${MAX_RESTARTS}) exceeded. Exiting."
                  Break
              }
          }
  
          # Sleep for $SLEEP_DELAY.
          Start-Sleep -Seconds $SLEEP_DELAY
      }
  
      Write-Host "Done."
  
  }
  
  
  
  # Execute script.
  main
  </code></pre>
  
  <p>...and here's the sample data:</p>
  
  <pre><code>2015-11-19 00:00:00, WARN Foo
  2015-11-19 00:00:00, ERROR Foo
  </code></pre>
  
  <p>In order to replicate this issue:</p>
  
  <ol>
  <li>Paste the sample data lines into the file <code>C:\Temp\test.log</code>.  Save.</li>
  <li>Start the monitoring script.</li>
  <li>Paste additional sample data lines into the log and save.  Wait for the <code>Error Count:</code> line to confirm that everything is working correctly.</li>
  <li>Continue to paste additional lines and watch the memory consumption for <code>powershell.exe</code> in Task Manager.  Note how much it increases at 400 lines...800 lines...8,000 lines...80,000 lines...</li>
  </ol>