title: Using protocol buffers for binary logging
body: <p>We're thinking of using Protocol Buffers for binary logging because:</p>
  
  <ul>
  <li>It's how we're encoding our objects anyway</li>
  <li>It is relatively compact, fast to read / write etc.</li>
  </ul>
  
  <p>That said, it isn't obvious how we should go about it because the APIs tend to focus on creating whole objects, so wrapping a list of DataLogEntry as a repeated field in a DataLogFile would be what you'd do in messaging terms but what we really want is just to be able to write and then read a whole DataLogEntry out, appending it to the end of a file.</p>
  
  <p>The first issue we've hit by doing that is that doing this (in a test:</p>
  
  <pre><code>        FileInputStream fileIn = new FileInputStream(logFile);
          CodedInputStream in = CodedInputStream.newInstance(fileIn);
          while(!in.isAtEnd()) {
              DataLogEntry entry = DataLogEntry.parseFrom(in);
              // ... do stuff
          }
  </code></pre>
  
  <p>Only results in 1 DataLogEntry being read from the stream. Without the isAtEnd, it never stops.</p>
  
  <p>Thoughts?</p>
  
  <p>Edit: I've switched to using entry.writeDelimitedTo and BidLogEntry.parseDelimitedFrom and that seems to work... </p>