Memory error at python
Question: 
    from __future__ import division
    import dataProcess
    import csv,re
    from collections import OrderedDict
    import itertools
    #######################################################################################
    #                   Pruning of N-grams depending upon the frequency of tags           #                                                       
    #######################################################################################
    
    for k in range(2,8):
        filename="Dataset/Cross/N_gram_Features_Pruned/"+str(k)+"_gram.txt"
        filewrite=open(filename,"w")
        CSV_tag_reader=csv.reader(open("Dataset/Cross/N_grams_recored/"+str(k)+"_gram.csv","r"),delimiter=',')
        header_data=CSV_tag_reader.next();    
        table = [row for row in CSV_tag_reader]
        values=[]
        result_tag=[]
        for j in range(0,len(header_data)):
            sum1=0
            avg1=0
            for i in range (0,3227):
                sum1=sum1+int(table[i][j])
        ##    print "************************************************************"
        ##    print sum1
            avg1=sum1/3227
        ##    print avg1
            if(avg1>=0.3):
                result_tag.append(header_data[j])
        print len(header_data)
        print len(result_tag)
        print "************************************************************"
        filewrite.write(str(result_tag))
    

My code is to count frequency of particular word in 3227 samples of data.i
have record about 277436 words frequency in 3227 samples.so image csv file
with 3227 rows and 60k columns.so am reading each word and sum the frequency
and find the average..bt am getting memory error when am running this code?how
can i solve?

>
>     Error:
>     Traceback (most recent call last):
>       File "N_gram_pruning.py", line 15, in <module>
>         table = [row for row in CSV_tag_reader]
>     MemoryError
>  

My csv file like thisss

    
    
    f1 f2 f3  f4.....f277436(header row)
    0  9  1    4      70
    56 2  66   8      23
    (3227 rows...)
    


Answer: The problem is that you're reading the entire file into memory. To avoid this,
you may have to restructure your algorithm. It seems that you're operating on
every column individually, meaning operations on each column are independent.
Therefore, if you transpose your csv files so they can be read line by line,
you can iterate over those lines rather than reading them all into memory.

Alternatively, you could use
[file.seek()](http://docs.python.org/2/library/stdtypes.html#file.seek),
though it'll be very slow.

