title: Powershell - get diff of two files without consuming huge amount of memory - is there any alternative c# or c++ API?
body: <p>I have two large files to compare (over 10 GB). The below command works fine for small files but seems to eat up RAM space on my machines.</p>
  <p>How can I get diff of two files without consuming huge amount of memory ?</p>
  <p>Any thoughts would be really appreciated.</p>
  <pre><code>robocopy.exe C:\Folder\ C:\Folder\ /l /nocopy /is /e /fp /ns /nc /njh /njs /tee  /log:c:\temp\FolderList.txt
  
  $path = 'C:\Folder\'
  $pattern = [regex]::Escape($path)
  $newContent = @()
  Get-Content -Path &quot;c:\temp\FolderList.txt&quot; | ForEach-Object {$newContent += $_ -replace $pattern, ''}
  Set-Content -Path &quot;c:\temp\FolderList.txt&quot; -Value $newContent
  
  (Get-Content C:\temp\FolderList.txt).Trim() -ne '' | Set-Content C:\temp\FolderList.txt
  
  robocopy.exe C:\Folder2\ C:\Folder2\ /l /nocopy /is /e /fp /ns /nc /njh /njs /tee  /log:c:\temp\FolderList2.txt
  
  $path = 'C:\Folder2\'
  $pattern = [regex]::Escape($path)
  $newContent = @()
  Get-Content -Path &quot;c:\temp\FolderList2.txt&quot; | ForEach-Object {$newContent += $_ -replace $pattern, ''}
  Set-Content -Path &quot;c:\temp\FolderList2.txt&quot; -Value $newContent
  
  (Get-Content C:\temp\FolderList2.txt).Trim() -ne '' | Set-Content C:\temp\FolderList2.txt
  
  Compare-Object -ReferenceObject (Get-Content c:\temp\FolderList.txt) -DifferenceObject (Get-Content c:\temp\FolderList2.txt)
  </code></pre>
  <hr />
  <p>LAST UPDATE</p>
  <p>Folderlist.txt</p>
  <pre><code>C:\Folder\Data2\Documents\
          C:\Folder\Data2\Documents\1.txt
          C:\Folder\Data2\Documents\2.txt
          C:\Folder\Data2\Documents\3.txt
          C:\Folder\Data2\Documents\4.txt
          C:\Folder\Data2\Documents\5.txt
  </code></pre>
  <p>CompareLog1.txt</p>
  <pre><code>Data2\Documents\
  C:\Folder\Data2\Documents\
          Data2\Documents\1.txt
          C:\Folder\Data2\Documents\1.txt
          Data2\Documents\2.txt
          C:\Folder\Data2\Documents\2.txt
          Data2\Documents\3.txt
          C:\Folder\Data2\Documents\3.txt
          Data2\Documents\4.txt
          C:\Folder\Data2\Documents\4.txt
          Data2\Documents\5.txt
          C:\Folder\Data2\Documents\5.txt
  </code></pre>
  <p>Desired output :</p>
  <pre><code>Data2\Documents\
  Data2\Documents\1.txt
  Data2\Documents\2.txt
  Data2\Documents\3.txt
  Data2\Documents\4.txt
  Data2\Documents\5.txt
  </code></pre>
  <p>UPDATE -2  :</p>
  <p>Output :</p>
  <pre><code>Data2\Documents\
  C:\Folder\Data2\Documents\
  Data2\Documents\1.txt
  C:\Folder\Data2\Documents\1.txt
  Data2\Documents\2.txt
  C:\Folder\Data2\Documents\2.txt
  Data2\Documents\3.txt
  C:\Folder\Data2\Documents\3.txt
  Data2\Documents\4.txt
  C:\Folder\Data2\Documents\4.txt
  Data2\Documents\5.txt
  C:\Folder\Data2\Documents\5.txt
  </code></pre>