title: MongoDB. How to carry calculated fields through an aggregation pipeline?
body: <p>I'm trying to gather some insights from some OSM data in json form. Here's an example of the documents I'm using in MongoDB/PyMongo:</p>
  
  <pre><code>{"amenity": "post_office",
   "name": "Dominion Road Postshop", 
   "created": {"uid": "10829", 
               "changeset": "607706", 
               "version": "5", 
               "user": "myfanwy",   
                "timestamp": "2007-11-24T12:41:04Z"}, 
   "pos": [-36.8801299, 174.7495053], 
   "created_by": "Potlatch 0.5d", 
   "type": "node", 
   "id": "61076379"}
  </code></pre>
  
  <p>So each document has a user and an amenity. I want to find the count of each amenity each user records, divided by the total amount of amenities that each user records. </p>
  
  <p>So to help clarify here are the code snippets I'd use to find each individual count:</p>
  
  <p>Query 1. Finding how many of each amenity each user records:</p>
  
  <pre><code>amenity_per_user = coll.aggregate([{"$match":{"amenity":{"$exists":True}}},
                                 {"$group":{"_id":{"user":"$created.user", "amenities":"$amenity"}, "count":{"$sum":1}}},
                                 {"$sort":{"count":-1}}])
  </code></pre>
  
  <p>Query 2. Find how many amenities each user records:</p>
  
  <pre><code>results = coll.aggregate([{"$match":{"amenity":{"$exists":True}}},
                        {"$group":{"_id":"$created.user", "count":{"$sum":1}}},
                        {"$sort":{"count":-1}}])
  </code></pre>
  
  <p>And the answers for both are(limited to 5 results each):</p>
  
  <pre><code>Finding how many of each amenity each user records:
  {u'_id': {u'amenities': u'parking', u'user': u'Rudy355'}, u'count': 1886}
  {u'_id': {u'amenities': u'post_box', u'user': u'Rudy355'}, u'count': 547}
  {u'_id': {u'amenities': u'telephone', u'user': u'Rudy355'}, u'count': 485}
  {u'_id': {u'amenities': u'parking', u'user': u'myfanwy'}, u'count': 451}
  {u'_id': {u'amenities': u'restaurant', u'user': u'Rudy355'}, u'count': 429}
  Find how many amenities each user records:
  {u'_id': u'Rudy355', u'count': 6321}
  {u'_id': u'myfanwy', u'count': 951}
  {u'_id': u'Robert Ancell', u'count': 599}
  {u'_id': u'lcmortensen', u'count': 366}
  {u'_id': u'Marks2000', u'count': 228}
  </code></pre>
  
  <p>Now what I want to do is divide the amount for each users top amenity (ie.Rudy355 made 1886 entries for the parking amenity) by their total amount of recordings(Query 2). - So one end result would be that Rudy355 made 0.3 of his recordings in the 'parking' amenity. - 1886/6321=0.3.</p>
  
  <p>This is what I've got to:</p>
  
  <pre><code>coll.aggregate([{"$match":{"amenity":{"$exists":True}}},
                      {"$group":{"_id":"$created.user", "user_count":{"$sum":1}}},
                      {"$group":{"_id":{"user":"$created.user", "amenities":"$amenity"}, "amenity_count":{"$sum":1}, 
                                 "ucount":{"$push":"$user_count"}}},
                      {"$unwind":"$ucount"},
                      {"$project":{"$divide":{"$ucount", "$amenity_count"}}},
                      {"$sort":{"count":-1}}])
  </code></pre>
  
  <p>Any help would be awesome!</p>
  
  <p>By the way I really don't like the idea of using $push to save the value for 'user_count'. Does anyone know of a better way of saving calculated fields like it. </p>