title: Rolling standard deviation excluding current rows in python
body: <p>I'm trying calculate the rolling standard deviation for a groupby object which contains Invoice Date, Vendor Name and Agency Name, while excluding the current row in the calculation. So the rolling std() for each Vendor-Agency pair basically. It’s my first time asking a question here, so please let me know if additional information is required.</p>
  <p>I already managed to do something similar with the rolling mean with the codes below.</p>
  <pre><code>last_3_invoices = dft_gb_comb.set_index(&quot;INVOICE_DATE&quot;).groupby([&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;])[&quot;daily_avg_amount&quot;].rolling(3,min_periods = 1, closed=&quot;left&quot;).mean().fillna(0).reset_index()
  last_30_df= dft_gb_comb.set_index(&quot;INVOICE_DATE&quot;).groupby([&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;])[&quot;daily_avg_amount&quot;].rolling(&quot;30d&quot;,closed=&quot;left&quot;).mean().fillna(0).reset_index()
  </code></pre>
  <p>The problem is, if I try to replicate this code with .std() instead of .mean() I get no values. If i omit the &quot;closed =&quot;left&quot;&quot; statement I get values, but those include the current row, so that wouldn't work for me.
  I tried several ways to work with the .shift() statement instead</p>
  <p>This could would work for some cases but it does not respect the [&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;] boundaries.</p>
  <pre><code>dft_gb_comb.set_index(&quot;INVOICE_DATE&quot;).groupby([&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;])[&quot;daily_avg_amount&quot;].rolling(3,min_periods=1).std().shift().reset_index()
  </code></pre>
  <p>If I use shift() directly after the groupby like below it only returns the Invoice date with the values while dropping [&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;] from the groupby object...</p>
  <pre><code>dft_gb_comb.set_index(&quot;INVOICE_DATE&quot;).groupby([&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;])[&quot;daily_avg_amount&quot;].shift().rolling(3,min_periods=1).std().reset_index()
  
  dft_gb_comb.set_index(&quot;INVOICE_DATE&quot;).groupby([&quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot;])[&quot;daily_avg_amount&quot;].apply(lambda x : x.shift(1).rolling(3,min_periods=1).std()).reset_index()
  </code></pre>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: left;"></th>
  <th style="text-align: center;">INVOICE_DATE</th>
  <th style="text-align: right;">daily_avg_amount</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: left;">0</td>
  <td style="text-align: center;">2018-04-19</td>
  <td style="text-align: right;">NaN</td>
  </tr>
  <tr>
  <td style="text-align: left;">1</td>
  <td style="text-align: center;">2018-12-10</td>
  <td style="text-align: right;">NaN</td>
  </tr>
  <tr>
  <td style="text-align: left;">408184</td>
  <td style="text-align: center;">2019-06-05</td>
  <td style="text-align: right;">139.02567</td>
  </tr>
  <tr>
  <td style="text-align: left;">408185</td>
  <td style="text-align: center;">2019-06-19</td>
  <td style="text-align: right;">124.92972</td>
  </tr>
  <tr>
  <td style="text-align: left;">408186</td>
  <td style="text-align: center;">2018-05-16</td>
  <td style="text-align: right;">34.92400</td>
  </tr>
  <tr>
  <td style="text-align: left;">408187</td>
  <td style="text-align: center;">2019-04-19</td>
  <td style="text-align: right;">135.41095</td>
  </tr>
  <tr>
  <td style="text-align: left;">408188</td>
  <td style="text-align: center;">2018-08-06</td>
  <td style="text-align: right;">NaN</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>instead of something like this, which would be the desired end result.</p>
  <div class="s-table-container">
  <table class="s-table">
  <thead>
  <tr>
  <th style="text-align: left;"></th>
  <th style="text-align: center;">VENDOR_NAME</th>
  <th style="text-align: center;">OCP_AGNCY_NAME</th>
  <th style="text-align: center;">INVOICE_DATE</th>
  <th style="text-align: right;">daily_avg_amount</th>
  </tr>
  </thead>
  <tbody>
  <tr>
  <td style="text-align: left;">0</td>
  <td style="text-align: center;">AED PROFESSIONALS  N. E.</td>
  <td style="text-align: center;">OKLA. A &amp; M COLLEGE</td>
  <td style="text-align: center;">2018-04-19</td>
  <td style="text-align: right;">NaN</td>
  </tr>
  <tr>
  <td style="text-align: left;">2</td>
  <td style="text-align: center;">E3 DIAGNOSTIC,INC. DBA E3 GORDON STOWE/</td>
  <td style="text-align: center;">OKLAHOMA STATE UNIVERSITY</td>
  <td style="text-align: center;">2018-01-17</td>
  <td style="text-align: right;">215.66757</td>
  </tr>
  </tbody>
  </table>
  </div>
  <p>I need to join the groupby object with the rolling values back to the main dataframe in the end, so I can't allow &quot;VENDOR_NAME&quot;,&quot;OCP_AGNCY_NAME&quot; to be dropped.</p>
  <p>current python version would be</p>
  <p>python           : 3.8.8.final.0</p>
  <p>pandas           : 1.2.4</p>
  <p>numpy            : 1.20.1</p>
  <p>I would be thankful for any kind of suggestions. I'm working with vendor data from <a href="https://data.ok.gov/dataset/state-of-oklahoma-vendor-payments-fy2022" rel="nofollow noreferrer">here</a></p>