title: Good resource for distributed computing / scale out patterns
body: <p>we're about to start working on a parallel cloud processing application and I'm looking for good resources how to set things up. Let me set the context:</p>
  
  <ul>
  <li>First we load a DB with a whole lot of data</li>
  <li>Then we have n-instances of cloud services that will generate PDFs from the data</li>
  <li>PDFs will then be merged, again should be scaleable</li>
  <li>Result stored in DB</li>
  <li>Done.</li>
  </ul>
  
  <p>I'm looking for resources to help me answer questions like:</p>
  
  <ul>
  <li>How can you measure progress for all of these instances? I suppose one controller instance that's monitoring. Should we use polling or a pub/sub system?</li>
  <li>How can you control these n-instances to start/stop/pause, whatever?</li>
  <li>Should the controller process be aware of the processors, or should he listen to broadcasts?</li>
  <li>We're thinking about a 'data'-queue, the queue from which each PDF generation instance gets the instructions to process, should we also use a 'command' queue for commands like 'start/stop'?</li>
  </ul>
  
  <p>Or - is there already something out of the box for this? I'm looking for the 'Patterns of Enterprise Application Architecture' but tailored to scaling out/parallel/cloud processing.</p>
  
  <p>Any thoughts?</p>
  
  <p><strong>EDIT</strong>
  Thanks for the -1. In case you're in doubt, I have Googled it, I have searched PluralSight and I have looked through Azure videos. I haven't ran across any patterns describing a process controller/processor setup.</p>