title: Pandas rolling over irregular time series with lowest deviation to regularly spaced series
body: <p>I've got a DataFrame with a timestamp as an index, where the newest data has the highest frequency and the older the data is, the lower the frequency will be (to save memory).</p>
  
  <p>Luckily pandas has the <code>rolling</code> module, which supports a time offset with for example <code>df.rolling('15T')</code> for 15 minutes rolling windows. But the deviations to rolling window applied to a time series interpolated to a regular time index are quite big. Especially the peaks are either flattened or too big.</p>
  
  <p><strong>The code may look a little bit clunky, but most of it is just generating the sample data and plotting it.</strong> The relevant about applying the rolling mean is only 4 lines of code.<br>
  The time stamp starts with 5-minute samples for the oldest samples, goes to 2-minute samples and finally to 1 minute samples for the newest data.<br>
  <code>df</code> is the irregularly spaced time series, <code>df2</code> is resampled and interpolated to 1-minute frequency and <code>df3</code> is resampled to 1-minute without interpolating. The DataFrames to which rolling means have been applied have an <code>r</code> appended to their names.<br>
  Standard deviations and norm are printed to compare the resulting deviations.<br>
  My example code to produce all different kinds of rolling means is the following.</p>
  
  <pre><code>tr1 = pd.date_range('20180820 1400', '20180820 1600', freq='1T')
  tr2 = pd.date_range('20180820 1100', '20180820 1400', freq='2T')
  tr3 = pd.date_range('20180820 0700', '20180820 1100', freq='5T')
  tr = np.concatenate((tr3[:-1], tr2[:-1], tr1))
  a = np.random.rand(tr.shape[0])
  df = pd.DataFrame(data=a, index=tr, columns=['irreg. data'])
  df2 = df.resample('1T').interpolate().rename(columns={'irreg. data': 'reg. data'})
  df3 = df.asfreq('1T').rename(columns={'irreg. data': 'asfreq. data'})
  
  freq = '15T'  # apply rolling mean with a window length of 15 minutes
  dfr = df.rolling(freq).mean().rename(columns={'irreg. data': 'rolling irreg. data'})
  df2r = df2.rolling(freq).mean().rename(columns={'reg. data': 'rolling reg. data'})
  df3r = df3.rolling(freq).mean().rename(columns={'asfreq. data': 'rolling asfreq. data'})
  
  fig = plt.figure()  # plot the comparison
  ax1 = fig.gca()
  df.plot.line(ax=ax1, marker='1')  # irregularly shaped time series
  dfr.plot.line(ax=ax1, marker='o')  # roll. mean of irregularly shaped time series
  df2.plot.line(ax=ax1)  # regularly shaped time series
  df2r.plot.line(ax=ax1, marker='x')  # roll. mean of regularly shaped time series
  df3.plot.line(ax=ax1)  # irreg. time series, resampled without interpolation
  df3r.plot.line(ax=ax1, marker='4')  # roll. mean of irreg. time series, resampled without interpolation
  ax1.legend()
  
  dev_irreg = (dfr.resample('1T').interpolate().iloc[:, 0] - df2r.iloc[:, 0])
  dev_asfreq = (df3r.resample('1T').interpolate().iloc[:, 0] - df2r.iloc[:, 0])
  print('deviations irreg:', dev_irreg.std(axis=0), np.linalg.norm(dev_irreg))
  print('deviations asfreq:', dev_asfreq.std(axis=0), np.linalg.norm(dev_asfreq))
  </code></pre>
  
  <hr>
  
  <h2>My question:</h2>
  
  <p>Is there any pandas module which will produce a result comparable to <code>df2r</code> (the dataframe which is interpolated to 1 minute before applying the rolling mean), without having to interpolate the data? Performance and especially memory usage is relevant, thus I'd like to avoid interpolating.<br>
  Furthermore the <code>asfreq</code> method in <code>df3</code> may have a slightly better deviance than using the irregularly spaced time series, but since I have to calculate differentials out of it, I don't really like the "plateaus".</p>
  
  <p>Thanks for your help in advance!</p>