title: Flink RocksDB Performance issues
body: <p>I have a flink job (scala) that is basically reading from a kafka-topic (1.0), aggregating data (1 minute event time tumbling window, using a <strong>fold</strong> function, which I know is deprecated, but is easier to implement than an aggregate function), and writing the result to 2 different kafka topics.</p>
  
  <p>The question is - when I'm using a FS state backend, everything runs smoothly, checkpoints are taking 1-2 seconds, with an average state size of 200 mb - that is, until the state size is increasing (while closing a gap, for example).</p>
  
  <p>I figured I would try rocksdb (over hdfs) for checkpoints - but the throughput is SIGNIFICANTLY less than fs state backend. As I understand it, flink does not need to ser/deserialize for every state access when using fs state backend, because the state is kept in memory (heap), rocks db DOES, and I guess that is what is accounting for the slowdown (and backpressure, and checkpoints take MUCH longer, sometimes timeout after 10 minutes).</p>
  
  <p>Still, there are times that the state cannot fit in memory, and I am trying to figure out basically how to make rocksdb state backend perform "better".</p>
  
  <p>Is it because of the deprecated fold function? Do I need to fine tune some parameters that are not easily searchable in documentation? any tips?</p>