title: Waterline/Sails.js mass insertion and deletion with MariaDB
body: <p>In my Sails.js app, I have a table <code>auctions</code>, which contains the data received as of the last dump from the <a href="https://dev.battle.net/io-docs" rel="nofollow">Battle.net API</a>.</p>
  
  <p>Thus, every time a new dump for a <code>realm</code> arrives, I need to clear all rows who are associated with this <code>realm</code>, and then insert the newer ones.</p>
  
  <p>I can either do it:</p>
  
  <ul>
  <li>all-at-once (sending a single query with <code>destroy()</code>, and another one with the dump itself);</li>
  <li>or by small batches (grouping them by <code>item</code>, then <code>destroy()</code>ing all records with <code>realm</code> and `item, then inserting the new ones).</li>
  </ul>
  
  <p>I am currently doing it with <code>async.forEachOfSeries</code>, with the small batches as described above, but with the current load (45k records inserted each time I run the job) it's unbearably slow, and that does not take into account if I need to update the <code>item</code> tooltip records.
  If I tried to inserting them all at once, either Node.js starts paging and hangs, or MariaDB takes longer than Waterline allows.</p>
  
  <p>How can I approach this more scalably? Would <code>async.queue</code> or <code>async.cargo</code> work?</p>