title: Apache NiFi - OutOfMemory Error: GC overhead limit exceeded on SplitText processor
body: <p>I am trying to use NiFi to process large CSV files (potentially billions of records each) using HDF 1.2. I've implemented my flow, and everything is working fine for small files.</p>
  
  <p>The problem is that if I try to push the file size to 100MB (1M records) I get a <code>java.lang.OutOfMemoryError: GC overhead limit exceeded</code> from the <strong>SplitText</strong> processor responsible of splitting the file into single records. I've searched for that, and it basically means that the garbage collector is executed for too long without obtaining much heap space. I expect this means that too many flow files are being generated too fast. </p>
  
  <p>How can I solve this? I've tried changing nifi's configuration regarding the max heap space and other memory-related properties, but nothing seems to work.</p>
  
  <p>Right now I added an intermediate <strong>SplitText</strong> with a line count of 1K and that allows me to avoid the error, but I don't see this as a solid solution for when the incoming file size will become potentially much more than that, I am afraid I will get the same behavior from the processor.</p>
  
  <p>Any suggestion is welcomed! Thank you</p>