Python: compute average of n-th elements in list of lists with different lengths
Question: Suppose I have the following list of lists:

    
    
    a = [ 
          [1, 2, 3],
          [2, 3, 4],
          [3, 4, 5, 6] 
        ]
    

I want to have the average of each n-th element in the arrays. However, when
wanting to do this in a simple way, Python generated out-of-bounds errors
because of the different lengths. I solved this by giving each array the
length of the longest array, and filling the missing values with None.

Unfortunately, doing this made it impossible to compute an average, so I
converted the arrays into masked arrays. The code shown below works, but it
seems rather cumbersome.

    
    
    import numpy as np
    import numpy.ma as ma
    
    a = [ [1, 2, 3],
          [2, 3, 4],
          [3, 4, 5, 6] ]
    
    # Determine the length of the longest list
    lenlist = []
    for i in a:
        lenlist.append(len(i))
    max = np.amax(lenlist)
    
    # Fill each list up with None's until required length is reached
    for i in a:
        if len(i) <= max:
            for j in range(max - len(i)):
                i.append(None)
    
    # Fill temp_array up with the n-th element
    # and add it to temp_array 
    temp_list = []
    masked_arrays = []
    for j in range(max):
        for i in range(len(a)):
            temp_list.append(a[i][j])
        masked_arrays.append(ma.masked_values(temp_list, None))
        del temp_list[:]
    
    # Compute the average of each array 
    avg_array = []
    for i in masked_arrays:
        avg_array.append(np.ma.average(i))
    
    print avg_array
    

Is there a way to do this more quickly? The final list of lists will contain
600000 'rows' and up to 100 'columns', so efficiency is quite important :-).


Answer: [tertools.izip_longest](https://docs.python.org/2/library/itertools.html#itertools.izip_longest)
would do all the padding with None's for you so your code can be reduced to:

    
    
    import numpy as np
    import numpy.ma as ma
    from itertools import izip_longest
    
    a = [ [1, 2, 3],
          [2, 3, 4],
          [3, 4, 5, 6] ]
    
    
    averages = [np.ma.average(ma.masked_values(temp_list, None)) for temp_list in izip_longest(*a)]
    
    print(averages)
    [2.0, 3.0, 4.0, 6.0]
    

No idea what the fastest way in regard to the numpy logic but this is
definitely going to be a lot more efficient than your own code.

If you wanted a faster pure python solution:

    
    
    from itertools import izip_longest, imap
    
    a = [[1, 2, 3],
         [2, 3, 4],
         [3, 4, 5, 6]]
    
    
    def avg(x):
        x = filter(None, x)
        return sum(x, 0.0) / len(x)
    
    
    filt = imap(avg, izip_longest(*a))
    
    print(list(filt))
    [2.0, 3.0, 4.0, 6.0]
    

If you have 0's in the arrays that won't work as 0 will be treated as Falsey,
you will have to use a list comp to filter in that case but it will still be
faster:

    
    
    def avg(x):
        x = [i for i in x if i is not None]
        return sum(x, 0.0) / len(x)
    
    filt = imap(avg, izip_longest(*a))
    

