title: create mongodb document with subdocuments atomically?
body: <p>I hope I'm having a big brainfart moment. But here's my situation in a scraping szenario;</p>
  
  <p>I'm wanting to be able to scrape over multiple machines and cores. Per site, I have different <code>Front</code> pages, I scrape (exmpl. for the site stackoverflow I'd have fronts stackoverflow.com/questions/tagged/javascript and stackoverflow.com/questions/tagged/nodejs).</p>
  
  <p>An <code>article</code> could be on every <code>Front</code> and when I discover an article I want to create an <code>Article</code> if the url is unknown, if its known I want to make an <code>Front</code> entry in <code>article.discover</code> if <code>Front</code> is unknown and otherwise insert my <code>FrontDiscovery</code> to the apropriate <code>Front</code>.</p>
  
  <p>Here are my Schemas;</p>
  
  <pre><code>FrontDiscovery = new Schema({
      _id         :{ type:ObjectId, auto:true },
      date        :{ type: Date, default:Date.now},
      dims        :{ type: Object, default:null},
      pos         :{ type: Object, default:null}
  });
  
  Front = new Schema({
      _id         :{ type:ObjectId, auto:true },
      url         :{type:String}, //front
      found       :[ FrontDiscovery ]
  });
  
  Article = new Schema({
      _id         :{ type:ObjectId, auto:true },
      url         :{ type: String , index: { unique: true } },
      site        :{ type: String },
      discover:[ Front]
  });
  </code></pre>
  
  <p>The Problem I am thinking I will eventually be running into is a race condition. When two job-runners (in parallel) find the same (before unknown) article and create a new one. Yes, I have a unique index on it and could handle it that way - quite inelegantly imho.</p>
  
  <p>But lets go further; When - for what ever reason - my 2 job-runners are scraping the same front at the same time and both notice that for <code>Front</code> there is no entry yet and create a new one adding the <code>FrontDiscovery</code>, I'd end with two entry's for the same <code>Front</code>.</p>
  
  <p>What are your strategies to circumvent such a situation? findByIdAndUpdate with the upsert:true for each document seperately? If so, how can I only push something to the embedded document collection and not overwrite everything else at the same time but still create the defaults if it hasnt been created?</p>
  
  <p>Thank you for any help in directing me in the right direction! I really hope I'm having a massive brainfart..</p>