Python: Create an incrementing variable that changes based on a condition
Question: I want to create a variable called 'inc' that increments sequentially each
time a condition is met (say, Delta>=5), holds the same otherwise, and resets
to 1 each time it encounters a new group (ID in this example). So here's an
example dataframe:

    
    
    from pandas import *
    d={'rx': [1,1,1,1,1,2,2,2,2,2],
        'vals': [1,2,3,8,10,1,10,20,21,22]}
    df=DataFrame(d)
    
    
    def diff_func(df):
        return df.diff()                                
    
    df['delta'] = df.groupby(['rx'])['vals'].apply(diff_func)  
    

Which produces this:

    
    
    rx  vals delta
    1   1    NaN
    1   2    1
    1   3    1
    1   8    5
    1   10   2
    2   1    NaN
    2   10   9
    2   20   10
    2   21   1
    2   22   1
    

So now I want to create a new column called 'Event' that I will later use to
group observations within each value of rx separated by a delta < 5:

    
    
    rx  vals    delta   Event
    1   1       NaN     1
    1   2       1       1
    1   3       1       1
    1   8       5       2
    1   10      2       2
    2   1       NaN     1
    2   10      9       2
    2   20      10      3
    2   21      1       3
    2   22      1       3
    

Note that 'event' returns to 1 on the first occurrence of rx. I am used to
doing this in vbasic or SAS, where you simply retain a value and then
increment by 1 each time the threshold trigger is met. Is there a similarly
simple solution to this in Python?


Answer: The usual approach is to do a comparison and then a cumulative sum. For
example, something like:

    
    
    >>> df["Event"] = (df["delta"] >= 5).groupby(df["rx"]).cumsum() + 1
    >>> df
       rx  vals  delta  Event
    0   1     1    NaN      1
    1   1     2      1      1
    2   1     3      1      1
    3   1     8      5      2
    4   1    10      2      2
    5   2     1    NaN      1
    6   2    10      9      2
    7   2    20     10      3
    8   2    21      1      3
    9   2    22      1      3
    

which works because (ignoring the groupby for simplicity and concentrating
only on `rx == 1`:) `False == 0` and `True == 1`:

    
    
    >>> df["delta"]
    0   NaN
    1     1
    2     1
    3     5
    4     2
    Name: delta, dtype: float64
    >>> df["delta"] >= 5
    0    False
    1    False
    2    False
    3     True
    4    False
    Name: delta, dtype: bool
    >>> (df["delta"] >= 5).cumsum()
    0    0
    1    0
    2    0
    3    1
    4    1
    Name: delta, dtype: int64
    

