Help calculating Avg and std for excel files saved as CSV
Question: I have about 20 excel files saved as CSV in a single folder. Each excel file
has numbers saved in the first, second and third columns. I was trying to read
the first column for all of the files, second column for all of the file, and
third column for all of the files using CSV module in python, and calulate the
average and standard deviation for each column and save these results in a
single separate excel. please help.... this is what i have so far...how can i
access each column separately?

    
    
    import csv
    import os
    from numpy import array
    
    path="A:\\hello\\folder"
    dirList=os.listdir(path)
    for file in dirList:
        fullpath=os.path.join(path,file)
    ##    print fullpath
        with open(fullpath, 'rb') as f:
            [[val for val in line.split(',')] for line in f.readlines()]
    ##        print line
            nums = array([line])
    
            for row in nums:
                print row.mean()
    


Answer: A [**list
comprehension**](http://docs.python.org/tutorial/datastructures.html#list-
comprehensions) works kinda like a backwards for-loop that automatically
constructs a `list` for you. If you nest these with the "columns" on the
inside and the "rows" on the outside you should get a matrix thingy (nested
list structure):

    
    
    nums = [[int(val) for val in line.split(',')] for line in my_file.readlines()]
    

Or maybe if you have a csv reader object it might be like this:

    
    
    nums = [[int(val) for val in line] for line in my_csv_reader]
    

And now you've got your matrix in a variable called `nums` thanks to the above
_list comprehension_.

Then you should probably use [numpy](http://new.scipy.org/download.html) to
compute your stats. This is nice because you can access columns of a numpy
array very easily and when you do it returns the column in the form of a numpy
array. numpy arrays also happen to have built-in methods for mean and standard
deviation. You can cast your `nums` to a numpy array just by passing it into
the `array()` constructor function :

    
    
    from numpy import array
    anums = array(nums)
    

Then if you want to iterate through columns, use the array slice notation and
the `shape` variable that is a member of every numpy array:

    
    
    # The 1 index of anums.shape should tell you how many columns you have
    for c in range(anums.shape[1]):
        column = anums[:,c]
        col_mean = column.mean()
        col_std = column.std()
        # Do something with these variables here, probably
    

