title: pandas split-apply-combine with results returned to original DataFrame
body: <p>I want to document a particular case of the <a href="https://pandas.pydata.org/pandas-docs/stable/groupby.html" rel="nofollow noreferrer">'split-apply-combine'</a> method here. </p>
  
  <p>The issue: I start with a DataFrame and I have to do some processing of the data that requires a groupby split and returns some results. Let's further assume that the processing cannot be accomplished by a simple built-in operation (there are shortcuts for that), but it is complex enough that I have to loop through the groups. Furthermore, I want the results to be put back into the original DataFrame. (Yes, I know, if the result is a <em>single scalar</em> for the whole group, I will be waisting memory. However, often I do want to have those results in the original data table rather than deal with another table). </p>
  
  <p>I have found that this case is poorly documented both in the pandas documentation and here. </p>
  
  <p>Let's illustrate with an example:</p>
  
  <pre><code>import pandas as pd
  import numpy as np
  
  # Prepare DataFrame
  data = {'Group':['A']*4+['B']*4+['C']*4,'Prop1':['S1','S2','S3','S4','S5','S6','S7','S8','S9','S10',
  'S11','S12'],'Prop2':[2004,2004,3004,3004,4004,4004,5004,5004,6004,6004,7004,7004],
  'Res1':[0,1,2,3,10,11,12,13,20,21,22,23]}
  df = pd.DataFrame(data=data)
  df
  </code></pre>
  
  <p><a href="https://i.stack.imgur.com/C9Sto.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/C9Sto.png" alt="Initial DataFrane"></a></p>
  
  <p>Now we go ahead with the groupby operation and the calculation of the new results:</p>
  
  <pre><code>df['Res2'] = np.nan
  df['Res3'] = np.nan
  grouped = df.groupby('Group')
  for (key,gr) in grouped:
      # Calculate two new results based on gr values
      res2 = gr['Res1'].mean()
      gr['Res2'] = res2
      res3 = gr['Prop2'] + gr['Res1']
      gr['Res3'] = res3
      # At this point gr has two new columns Res2 and Res3
      # Now we need to copy those changes back to the original DataFrame df
      df.update(gr)
  </code></pre>
  
  <p>The tricky part here is putting the results back into <code>df</code>. Notice that I had to first create the two new columns in <code>df</code>, otherwise that would not work. The first method is to use the <code>.update</code> command. This is easy to read. Using the <code>%timeit</code> command, I timed this at (1.85ms, 1.84ms, 1.74ms). </p>
  
  <p>A second method to do the same is to note that the original index is preserved in <code>gr</code>. So we can just replace <code>df.update(gr)</code> with:</p>
  
  <pre><code>df.loc[gr.index,('Res2','Res3')] = gr[['Res2','Res3']]
  </code></pre>
  
  <p>After either of those cells, we get the desired result (along with the notorious <code>SettingWithCopy</code> warning):</p>
  
  <p><a href="https://i.stack.imgur.com/tIEKG.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/tIEKG.png" alt="final result"></a></p>
  
  <p>Interestingly enough, the second method is slightly faster, timing at (1.66ms,1.69ms,1.73ms).</p>
  
  <p>The question I have is, is there an easier way?
  And, how is this done for a DataFrame with row multi-index?</p>