Describing gaps in a time series pandas
Question: I'm trying to write a function that takes a continuous time series and returns
a data structure which describes any missing gaps in the data (e.g. a DF with
columns 'start' and 'end'). It seems like a fairly common issue for time
series, but despite messing around with groupby, diff, and the like -- and
exploring SO -- I haven't been able to come up with much better than the
below.

It's a priority for me that this use vectorized operations to remain
efficient. There has got to be a more obvious solution using vectorized
operations -- hasn't there? Thanks for any help, folks.

    
    
    import pandas as pd
    
    
    def get_gaps(series):
        """
        @param series: a continuous time series of data with the index's freq set
        @return: a series where the index is the start of gaps, and the values are
             the ends
        """
        missing = series.isnull()
        different_from_last = missing.diff()
    
        # any row not missing while the last was is a gap end        
        gap_ends = series[~missing & different_from_last].index
    
        # count the start as different from the last
        different_from_last[0] = True
    
        # any row missing while the last wasn't is a gap start
        gap_starts = series[missing & different_from_last].index        
    
        # check and remedy if series ends with missing data
        if len(gap_starts) > len(gap_ends):
             gap_ends = gap_ends.append(series.index[-1:] + series.index.freq)
    
        return pd.Series(index=gap_starts, data=gap_ends)
    

For the record, Pandas==0.13.1, Numpy==1.8.1, Python 2.7


Answer: This problem can be transformed to find the continuous numbers in a list. find
all the indices where the series is null, and if a run of (3,4,5,6) are all
null, you only need to extract the start and end (3,6)

    
    
    import numpy as np
    import pandas as pd
    from operator import itemgetter
    from itertools import groupby
    
    
    # create an example 
    data = [2, 3, 4, 5, 12, 13, 14, 15, 16, 17]
    s = pd.series( data, index=data)
    s = s.reindex(xrange(18))
    print find_gap(s)  
    
    
    def find_gap(s): 
        """ just treat it as a list
        """ 
        nullindex = np.where( s.isnull())[0]
        ranges = []
        for k, g in groupby(enumerate(nullindex), lambda (i,x):i-x):
            group = map(itemgetter(1), g)
            ranges.append((group[0], group[-1]))
        startgap, endgap = zip(* ranges) 
        return pd.series( endgap, index= startgap )
    

reference : [Identify groups of continuous numbers in a
list](http://stackoverflow.com/questions/2154249/identify-groups-of-
continuous-numbers-in-a-list)

