title: Why is bzip2's maximum blocksize 900k?
body: <p><code>bzip2</code> (i.e. <a href="http://bzip.org/" rel="nofollow noreferrer">this program</a> by Julian Seward)'s lists available block-sizes between 100k and 900k:</p>
  
  <pre><code> $ bzip2 --help
   bzip2, a block-sorting file compressor.  Version 1.0.6, 6-Sept-2010.
  
   usage: bzip2 [flags and input files in any order]
  
     -1 .. -9            set block size to 100k .. 900k
  </code></pre>
  
  <p>This number corresponds to the <code>hundred_k_blocksize</code> value written into the <a href="https://en.wikipedia.org/wiki/Bzip2#File_format" rel="nofollow noreferrer">header</a> of a compressed file.</p>
  
  <p>From the <a href="http://www.bzip.org/1.0.3/html/memory-management.html" rel="nofollow noreferrer">documentation</a>, memory requirements are as follows:</p>
  
  <pre><code>Compression:   400k + ( 8 x block size )
  
  Decompression: 100k + ( 4 x block size ), or
                 100k + ( 2.5 x block size )
  </code></pre>
  
  <p>At the time the original program was written (1996), I imagine 7.6M (400k + 8 * 900k) might have been a hefty amount of memory on a computer, but for today's machines it's nothing.</p>
  
  <p>My question is two part:</p>
  
  <p>1) Would better compression be achieved with larger block sizes? (Naively I'd assume yes). Is there any reason not to use larger blocks? How does the cpu time for compression scale with the size of block? </p>
  
  <p>2) Practically, are there any forks of the bzip2 code (or alternate implementations) that allow for larger block sizes? Would this require significant revision to the source code?</p>
  
  <p>The file format seems flexible enough to handle this. For example ... since <code>hundred_k_blocksize</code> holds an 8-bit character that indicates the block-size, one could extend down the <a href="http://www.asciitable.com/" rel="nofollow noreferrer">ASCII table</a> to indicate larger block-sizes (e.g. <code>':'</code> = <code>x3A</code> => <code>1000k</code>, <code>';'</code> = <code>x3B</code> => <code>1100k</code>, <code>'&lt;'</code> = <code>x3C</code> => <code>1200k</code>, ...).</p>