title: How can I cat multiple files together into one without intermediary file?
body: <p>Here is the problem I'm facing: </p>
  
  <ul>
  <li>I am string processing a text file ~100G in size. </li>
  <li>I'm trying to improve the runtime by splitting the file into many hundreds
  of smaller files and processing them in parallel. </li>
  <li>In the end I cat the resulting files back together in order. </li>
  </ul>
  
  <p>The file read/write time itself takes hours, so I would like to find a way to improve the following:</p>
  
  <pre><code>cat file1 file2 file3 ... fileN &gt;&gt; newBigFile
  </code></pre>
  
  <ol>
  <li><p>This requires double the diskspace as <code>file1</code> ... <code>fileN</code> takes up 100G, and then <code>newBigFile</code> takes another 100Gb, and then <code>file1</code>... <code>fileN</code> gets removed.</p></li>
  <li><p>The data is already in <code>file1</code> ... <code>fileN</code>, doing the <code>cat &gt;&gt;</code> incurs read
  and write time when all I really need is for the hundreds of files to
  reappear as 1 file...</p></li>
  </ol>