title: How to merge multiple large .gz files into one in a efficient way?
body: <p>I'm trying to combine multiple (29) compressed files (.gz), one after the other, into one file. </p>
  
  <p>The compressed files are around 500MB and in their uncompressed format ~30GB. All the files start with a header that I don't want in the final file.</p>
  
  <p>I have tried to do it using <code>zcat</code>and <code>gzip</code>, but it takes a lot of time (more than 3hours):</p>
  
  <pre><code> zcat file*.gz | tail -n +2 | gzip -c &gt;&gt; all_files.txt.gz 
  </code></pre>
  
  <p>I have also tried it with <code>pigz</code>:</p>
  
  <pre><code> unpigz -c file*.gz | tail -n +2 | pigz -c &gt;&gt; all_files_pigz.txt.gz 
  </code></pre>
  
  <p>In this case, I'm working in a cluster where they don't have this command and I can't install anything.</p>
  
  <p>The last thing I have tried is to merge all with <code>cat</code>:</p>
  
  <pre><code> cat file*.gz &gt; all_files_cat.txt.gz
  </code></pre>
  
  <p>It doesn't take a lot of time, but when I'm going to read it, at some pint appears the following message:</p>
  
  <pre><code> gzip: unexpected end of file
  </code></pre>
  
  <p>How could I deal with this?</p>