title: How to normalize/reduce time data in mongoDB?
body: <p>I'm storing minutely performance data in MongoDB, each collection is a type of performance report, and each document is the measurement at that point in time for the port on the array:</p>
  
  <pre><code>{
    "DateTime" : ISODate("2012-09-28T15:51:03.671Z"),
    "array_serial" : "12345",
    "Port Name" : "CL1-A",
    "metric" : 104.2
  }
  </code></pre>
  
  <p>There can be up to 128 different "Port Name" entries per "array_serial".</p>
  
  <p>As the data ages I'd like to be able to average it out over increasing time spans:</p>
  
  <ul>
  <li>Up to 1 Week : minute</li>
  <li>1 Week to 1 month : 5 minute</li>
  <li>1 - 3 months: 15 minute</li>
  </ul>
  
  <p>etc.. 
  Here's how I'm averaging the times so that they can be reduced :</p>
  
  <pre><code>var resolution = 5; // How many minutes to average over     
  var map = function(){
          var coeff = 1000 * 60 * resolution;
          var roundTime = new Date(Math.round(this.DateTime.getTime() / coeff) * coeff);
          emit(roundTime, { value : this.metric, count: 1 } );
   };
  </code></pre>
  
  <p>I'll be summing the values and counts in the reduce function, and getting the average in the finalize funciton.</p>
  
  <p>As you can see this would average the data for just the time leaving out the "Port Name" value, and I need to average the values over time for each "Port Name" on each "array_serial".</p>
  
  <p>So how can I include the port name in the above map function? Should the key for the emit be a compound "array_serial,PortName,DateTime" value that I split later? Or should I use the query function to query for each distinct serial, port and time? Am I storing this data in the database correctly?</p>
  
  <p>Also, as far as I know this data gets saved out to it's own collection, what's the standard practice for replacing the data in the collection with this averaged data? </p>
  
  <hr>
  
  <p>Is this what you mean Asya? Because it's not grouping the documents rounded to the lower 5 minute (btw, I changed 'DateTime' to 'datetime'): </p>
  
  <pre><code>    $project: {
                  "year" : { $year : "$datetime" },
                  "month" : { $month : "$datetime" },
                  "day" : { $dayOfMonth : "$datetime" },
                  "hour" : { $hour : "$datetime" },
                  "minute" : { $mod : [ {$minute : "$datetime"}, 5] },
                  array_serial: 1,
                  port_name: 1,
                  port_number: 2,
                  metric: 1
  }
  </code></pre>
  
  <p>From what I can tell the "$mod" operator will return the remainder of the minute divided by five, correct?</p>
  
  <p>This would really help me if I could get the aggregation framework to do this operation rather than mapreduce.</p>