title: Deflater.deflate and small output buffers
body: <p>I'm seeing a strange situation with small output buffers with Java 8u45 and the <a href="http://docs.oracle.com/javase/8/docs/api/java/util/zip/Deflater.html#deflate-byte:A-int-int-int-" rel="nofollow"><code>java.util.Deflater.deflate(byte[] b, int off, int len, int flush)</code></a> method when used with small output buffers.</p>
  
  <p>(I'm working on some low level networking code related to WebSocket's upcoming <code>permessage-deflate</code> extension, so small buffers are a reality for me)</p>
  
  <p>The example code:</p>
  
  <pre class="lang-java prettyprint-override"><code>package deflate;
  
  import java.nio.charset.StandardCharsets;
  import java.util.zip.Deflater;
  
  public class DeflaterSmallBufferBug
  {
      public static void main(String[] args)
      {
          boolean nowrap = true;
          Deflater deflater = new Deflater(Deflater.DEFAULT_COMPRESSION,nowrap);
  
          byte[] input = "Hello".getBytes(StandardCharsets.UTF_8);
  
          System.out.printf("input is %,d bytes - %s%n",input.length,getHex(input,0,input.length));
  
          deflater.setInput(input);
  
          byte[] output = new byte[input.length];
  
          // break out of infinite loop seen with bug
          int maxloops = 10;
  
          // Compress the data
          while (maxloops-- &gt; 0)
          {
              int compressed = deflater.deflate(output,0,output.length,Deflater.SYNC_FLUSH);
              System.out.printf("compressed %,d bytes - %s%n",compressed,getHex(output,0,compressed));
  
              if (compressed &lt; output.length)
              {
                  System.out.printf("Compress success");
                  return;
              }
          }
  
          System.out.printf("Exited compress (maxloops left %d)%n",maxloops);
      }
  
      private static String getHex(byte[] buf, int offset, int len)
      {
          StringBuilder hex = new StringBuilder();
          hex.append('[');
          for (int i = offset; i &lt; (offset + len); i++)
          {
              if (i &gt; offset)
              {
                  hex.append(' ');
              }
              hex.append(String.format("%02X",buf[i]));
          }
          hex.append(']');
          return hex.toString();
      }
  }
  </code></pre>
  
  <p>In the above case, I'm attempting to generate compressed bytes for the input <code>"Hello"</code> using an output buffer of 5 bytes in length.</p>
  
  <p>I would assume the following resulting bytes:</p>
  
  <pre class="lang-none prettyprint-override"><code>buffer 1 [ F2 48 CD C9 C9 ]
  buffer 2 [ 07 00 00 00 FF ]
  buffer 3 [ FF ]
  </code></pre>
  
  <p>Which translates as</p>
  
  <pre class="lang-none prettyprint-override"><code>[ F2 48 CD C9 C9 07 00 ] &lt;-- the compressed data
  [ 00 00 FF FF ]          &lt;-- the deflate tail bytes
  </code></pre>
  
  <p>However, when <code>Deflater.deflate()</code> is used with a small buffer, this normal loop continues infinitely at 5 bytes of compressed data (seems to only manifest at buffers of 5 bytes or lower).</p>
  
  <p>Resulting output of running the above demo ...</p>
  
  <pre class="lang-none prettyprint-override"><code>input is 5 bytes - [48 65 6C 6C 6F]
  compressed 5 bytes - [F2 48 CD C9 C9]
  compressed 5 bytes - [07 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  compressed 5 bytes - [FF 00 00 00 FF]
  Exited compress (maxloops left -1)
  </code></pre>
  
  <p>If you make the input/output larger than 5 bytes then the problem seems to go away.  (Just make the input string <code>"Hellox"</code> to test this for yourself)</p>
  
  <p>Results of making the buffer 6 bytes (input as <code>"Hellox"</code>)</p>
  
  <pre class="lang-none prettyprint-override"><code>input is 6 bytes - [48 65 6C 6C 6F 78]
  compressed 6 bytes - [F2 48 CD C9 C9 AF]
  compressed 6 bytes - [00 00 00 00 FF FF]
  compressed 5 bytes - [00 00 00 FF FF]
  Compress success
  </code></pre>
  
  <p>Even these results are bit quirky to me, as it seems there's 2 deflate tail-byte sequences present.</p>
  
  <p>So, I guess my ultimate question is, am I missing something about the <code>Deflater</code> usage that making thing odd for me, or is this pointing at a possible bug in the JVM <code>Deflater</code> implementation itself?</p>
  
  <p><strong>Update: Aug 7, 2015</strong></p>
  
  <p>This discovery has been accepted as <a href="http://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8133170" rel="nofollow">bugs.java.com/JDK-8133170</a> </p>