title: Adding Subtotals to Pandas Groupby
body: <p>I am looking for a cleaner way to add subtotals to Pandas groupby. </p>
  
  <p>Here is my DataFrame: </p>
  
  <pre><code>df = pd.DataFrame({
  'Category':np.random.choice( ['Group A','Group B'], 50),
  'Sub-Category':np.random.choice( ['X','Y'], 50),
  'Product':np.random.choice( ['Product 1','Product 2'], 50),
  'Units_Sold':np.random.randint(1,100, size=(50)),
  'Dollars_Sold':np.random.randint(100,1000, size=50),
  'Date':np.random.choice( pd.date_range('1/1/2011','03/31/2011',  
                        freq='D'), 50, replace=False)})
  </code></pre>
  
  <p>From there, I create a new Groupby Dataframe as such:</p>
  
  <pre><code>df1 = df.groupby(['Category','Sub-Category','Product',pd.TimeGrouper(key='Date',freq='M')]).agg({'Units_Sold':'sum','Dollars_Sold':'sum'}).unstack().fillna(0)
  </code></pre>
  
  <p>I would like to provide sub-totals for both Category &amp; Sub-Category. I can do this using this code:</p>
  
  <pre><code>df2 = df1.groupby(level=[0,1]).sum()
  df2.index = pd.MultiIndex.from_arrays([df2.index.get_level_values(0),
                                     df2.index.get_level_values(1) + ' Total',
                                     len(df2) * ['']])
  df3 = df1.groupby(level=[0]).sum()
  df3.index = pd.MultiIndex.from_arrays([df3.index.get_level_values(0) + ' Total',
                                     len(df3) * [''],
                                     len(df3) * ['']])
  pd.concat([df1,df2,df3]).sort_index()
  </code></pre>
  
  <p>This gives me the DataFrame I want:
  <a href="https://i.stack.imgur.com/1O6Vk.png" rel="nofollow noreferrer">Final DataFrame</a></p>
  
  <p>My question - is there a more pythonic way to do this than to have to create a new DataFrame for each level then concat together? I have researched this, but can not find a better way. I have to do this for many different MultiIndex dataframes &amp; am seeking a better solution.</p>
  
  <p>Thanks in advance for your help!</p>
  
  <p>EDIT WITH ADDITIONAL INFORMATION:</p>
  
  <p>Thank you to both @Wen &amp; @DaFanat for their replies. I attempted to use the link @Wen provided on my data [link]:<a href="https://stackoverflow.com/questions/43238183/python-pandas-add-subtotal-on-each-lvl-of-multiindex-dataframe">Python (Pandas) Add subtotal on each lvl of multiindex dataframe</a></p>
  
  <pre><code>pd.concat([df.assign(\
      **{x: 'Total' for x in "CategorySub-CategoryProduct"[i:]}\
      ).groupby(list('abc')).sum() for i in range(1,4)])\
      .sort_index()
  </code></pre>
  
  <p>This sums the total, however it ignores the dates that make up the second level of the columns. It leaves me with this outcome.<a href="https://i.stack.imgur.com/o4jHF.png" rel="nofollow noreferrer">Resulting Image</a></p>
  
  <p>I've tried to add in a TimeGrouper with the groupby, but that returns an error. Any help would be greatly appreciated. Thanks! </p>