title: DataBricks: Fastest Way to Insert Data Into Delta Table?
body: <p>I have a handful of tables that are only a few MBs in filesize each that I want to capture as Delta Tables. Inserting new data into them takes an extraordinarily long time, 15+ minutes, which I am astonished at.</p>
  <p>The culprit, I am guessing, is that while the table is very small; there are over 300 columns in these tables.</p>
  <p>I have tried the following methods, with the former being faster than the latter (unsurprisingly(?)): (1) <code>INSERT INTO</code> , (2) <code>MERGE INTO</code>.</p>
  <p>Before inserting data into the Delta Tables, I apply a handful of Spark functions to clean the data and then lastly register it as a temp table (e.g., <code>INSERT INTO DELTA_TBL_OF_INTEREST (cols) SELECT * FROM tempTable</code></p>
  <p>Any recommendations on speeding this process up for trivial data?</p>