title: Mongo bulk insert and avoid duplicate values for multiple keys
body: <p>I have two collections <code>items</code> with 120,000 entries and <code>itemHistories</code> with more than 20 million entries. I periodically update all <code>items</code> and <code>itemHistories</code> by fetching an API that lists all <code>history</code> data for an item.</p>
  
  <p>What I need to do is batch insert the history data to the collection while avoiding duplicates. Also the history API returns only <code>date</code>, <code>info</code>, <code>item_id</code> values.</p>
  
  <p>Is it possible to batch insert in Mongo so that it doesn't add duplicates for 2 values combined (date, item_id). So if there already is an entry with the same <code>date</code> and <code>item_id</code> don't add it. Basically the <code>date</code> is an unique index for the <code>item_id</code>. It's allowed to have duplicate <code>date</code> values in the collection but only if the <code>item_id</code> is different for all the duplicates.</p>
  
  <p>One item can have close to a million entries so I don't think fetching the history from the collection and comparing it to the API response is going to be optimal.</p>
  
  <p>My current idea was to add another key to the collection called <code>hash</code> that is an <code>md5(date,info,item_id)</code> and make it an unique index. Suggestions?</p>