title: Sort list of objects by a number of attributes using a generator
body: <p>I've got a list of objects which will number somewhere between the thousands &amp; tens of thousands. These objects could be thought of as people which I'm looking to rank based on a score they have.</p>
  
  <p>So first of all they're split into groups by age, then gender etc. At each point a ranking is provided corresponding to that age/gender category. The fields on the objects are <code>age_group</code> and <code>gender</code>. So you'd first collect everybody that's got the <code>30-39</code> age group, then all the men (<code>M</code>) and all women (<code>W</code>) from that age group.</p>
  
  <p>Creating a new list at each of these points is very memory intensive so I'm attempting to use a generator &amp; <a href="/questions/tagged/itertools" class="post-tag" title="show questions tagged &#39;itertools&#39;" rel="tag">itertools</a> to group using the original list. So I've got a function to do that;</p>
  
  <pre><code>def group_standings(_standings, field):
      """ sort list of standings by a given field """
      getter = operator.attrgetter(field)
      for k, g in itertools.groupby(_standings, getter):
          yield list(g)
  
  
  def calculate_positions(standings):
      """
      sort standings by age_group then gender &amp; set position based on point value 
      """
      for age_group in group_standings(standings, 'age_group'):
  
          for gender_group in group_standings(age_group, 'gender'):
  
              set_positions(
                  standings=gender_group,
                  point_field='points',
                  position_field='position',
              )
  </code></pre>
  
  <p>For <code>set_positions</code> to function correctly it needs the whole group so that it can sort by the <code>point_field</code> value then set the <code>position_field</code> value.</p>
  
  <p>Debugging the generator, <code>groupby</code> isn't collecting all objects matching the key as I'd expected. The output is something like;</p>
  
  <pre><code>DEBUG generating k 30-39
  DEBUG generating g [&lt;Standing object at 0x7fc86fedbe10&gt;, &lt;Standing object at 0x7fc86fedbe50&gt;, &lt;Standing object at 0x7fc86fedbe90&gt;]
  
  DEBUG generating k 20-29
  DEBUG generating g [&lt;Standing object at 0x7fc86fedbed0&gt;]
  
  DEBUG generating k 30-39
  DEBUG generating g [&lt;Standing object at 0x7fc86fedbf10&gt;]
  
  DEBUG generating k 20-29
  DEBUG generating g [&lt;Standing object at 0x7fc86fedbf50&gt;, &lt;Standing object at 0x7fc86fedbf90&gt;, &lt;Standing object at 0x7fc86fedbfd0&gt;, &lt;Standing object at 0x7fc856ecc050&gt;, &lt;Standing object at 0x7fc856ecc090&gt;, &lt;Standing object at 0x7fc856ecc0d0&gt;, &lt;Standing object at 0x7fc856ecc110&gt;, &lt;Standing object at 0x7fc856ecc150&gt;, &lt;Standing object at 0x7fc856ecc190&gt;, &lt;Standing object at 0x7fc856ecc1d0&gt;]
  </code></pre>
  
  <p>To confirm, for <code>set_positions</code> to function, the list provided by the generator would need to contain all objects in the <code>20-29</code> age group, but as above, objects from that group are being found on multiple iterations of the list.</p>