Loop through group and difference lags I have time-series data (t) that I want to loop through each t and take the difference of the previous five lags t-1 ... t-5. 

This code works, but is inefficient. I'm not sure how to increase the performance and efficiency of the code.  

Sample data

import pandas as pd
import numpy.random as nr

dat = pd.DataFrame({'t': [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 'rnum': nr.rand(10)})


Function

def diff_n(idat):
    # Get t from group
    i = idat['t'].iat[0]

    # If one, nothing to difference
    if i == 1:
        return None

    # Get value to loop through and diff
    h1 = idat.groupby('t')['rnum'].mean().values

    # Subset dat (outside fn)
    h2 = dat[(dat.t &lt; i ) &amp; (dat.t &gt;= i - 5)]

    # Combine data for return
    retdat = pd.DataFrame()

    # Loop through t-1 to last obs
    for j in range(max(h2.t), min(h2.t), -1):

        # Get value to difference from h1
        h3 = h2[h2.t == j].groupby('t')['rnum'].mean().values

        # Difference
        ndiff = (h3 - h1)

        # Build df and concat
        indat = pd.DataFrame({'t': [i], 'lag': i - j, 'diff': ndiff})
        retdat = pd.concat([retdat, indat])
    return retdat


Groupby and apply the function to each t

dat.groupby('t').apply(lambda x: diff_n(x))


Output

&gt;&gt;&gt; dat.groupby('t').apply(lambda x: diff_n(x))
      t  lag      diff
t                      
2  0   2    1  0.440644
3  0   3    1 -0.284075
   0   3    2  0.156569
4  0   4    1  0.439154
   0   4    2  0.155079
   0   4    3  0.595723
5  0   5    1 -0.095552
   0   5    2  0.343602
   0   5    3  0.059527
   0   5    4  0.500171
6  0   6    1  0.296337
   0   6    2  0.200784
   0   6    3  0.639939
   0   6    4  0.355864
   0   6    5  0.796507
7  0   7    1 -0.749913
   0   7    2 -0.453576
   0   7    3 -0.549128
   0   7    4 -0.109974
   0   7    5 -0.394049
8  0   8    1  0.506379
   0   8    2 -0.243534
   0   8    3  0.052803
   0   8    4 -0.042749
   0   8    5  0.396405
9  0   9    1 -0.021761
   0   9    2  0.484618
   0   9    3 -0.265295
   0   9    4  0.031042
   0   9    5 -0.064510
10 0  10    1  0.409118
   0  10    2  0.387357
   0  10    3  0.893736
   0  10    4  0.143823
   0  10    5  0.440160
First of all, nice code, very readable and good comments.

There are several improvements you can do:


Reindex your dataframe with t. You are writing a lot of times .t, if you will use index some of pandas index optimization will kick in and your code will be faster.
Do not do concat on only 2 items. Concatenate function will work much faster if you will just do all of the concatenation of all 5 elements at once.
Why do you need h2? Try to avoid unnecessary coping of the data frame. You can work on the original h1 or even idat since your function does not change anything implicitly.
Use type annotations. Using type annotations can make the code run faster since the interpreter does not need to figure out what is the variable type. Also, probably even more important, it helps readability.