title: How to get data into h2o fast
body: <p>What my question isnt:    </p>
  
  <ul>
  <li><a href="https://stackoverflow.com/questions/44683514/efficient-way-to-maintain-a-h2o-data-frame">Efficient way to maintain a h2o data frame</a></li>
  <li><a href="https://stackoverflow.com/questions/45783048/h2o-running-slower-than-data-table-r">H2O running slower than data.table R</a></li>
  <li><a href="https://stackoverflow.com/questions/34082792/loading-data-bigger-than-the-memory-size-in-h2o">Loading data bigger than the memory size in h2o</a></li>
  </ul>
  
  <p>Hardware/Space:    </p>
  
  <ul>
  <li>32 Xeon threads w/ ~256 GB Ram</li>
  <li>~65 GB of data to upload. (about 5.6 billion cells)</li>
  </ul>
  
  <p><strong>Problem:</strong><br>
  It is taking hours to upload my data into h2o.  This isn't any special processing, only "as.h2o(...)".  </p>
  
  <p>It takes less than a minute using "fread" to get the text into the space and then I make a few row/col transformations (diff's, lags) and try to import.  </p>
  
  <p>The total R memory is ~56GB before trying any sort of "as.h2o" so the 128 allocated shouldn't be too crazy, should it?</p>
  
  <p><strong>Question:</strong><br>
  What can I do to make this take less than an hour to load into h2o?  It should take from a minute to a few minutes, no longer.</p>
  
  <p><strong>What I have tried:</strong>    </p>
  
  <ul>
  <li>bumping ram up to 128 GB in 'h2o.init'</li>
  <li>using slam, data.table, and options( ...</li>
  <li>convert to "as.data.frame" before "as.h2o"</li>
  <li>write to csv file (r write.csv chokes and takes forever.  It is writing a lot of GB though, so I understand).</li>
  <li>write to sqlite3, too many columns for a table, which is weird.  </li>
  <li>Checked drive cache/swap to make sure there are enough GB there.  Perhaps java is using cache.  (still working)</li>
  </ul>
  
  <p><strong>Update:</strong><br>
  So it looks like my only option is to make a giant text file and then use "h2o.importFile(...)" for it.  I'm up to 15GB written.  </p>
  
  <p><strong>Update2:</strong><br>
  It is a hideous csv file, at ~22GB (~2.4Mrows, ~2300 cols).  For what it was worth, it took from 12:53pm until 2:44PM to write the csv file.  Importing it was substantially faster, after it was written.</p>