title: Batch messaging tracking and monitoring
body: <p>I receive a multiple files(huge nos) of 100,000 records and need to be processed with 3 hours.</p>
  
  <p>Each record needs to be processed by multiple(4) tasks and atlast all records from 1 file are merged together and processed.</p>
  
  <p>Now, I'm planning to use REDIS or RABBIT-MQ as my queuing system.</p>
  
  <p>ie:-</p>
  
  <pre><code>file1
  
  batch-1 
      record-1
      record-2
  
  batch-2
      record-3
      record-4
  </code></pre>
  
  <p>How can I track all the records of file1 at last ie; I need a correlation id for my different batches. </p>
  
  <p>1.Is there a preferred architecture to implement this?
  2.How can I create a correlation id for all my messages?</p>
  
  <p>I'm very new to queuing system, so inputs would be very useful.</p>