csv file compression without using existing libraries in Python
Question: I'm trying to compress a .csv file without using any 3rd party or framework
provided compression libraries.

I have tried, what I wish to think, everything. I looked at Huffman, but since
I'm not allowed to use that solution I tried to do my own.

An example:

    
    
    6NH8,F,A,0,60541567,60541567,78.78,20
    6NH8,F,A,0,60541569,60541569,78.78,25
    6AH8,F,B,0,60541765,60541765,90.52,1
    QMH8,F,B,0,60437395,60437395,950.5,1
    

I made an algorithm that counts every char and gives me amount of times
they've been used and, depending on how many time they been dedicated a
number.

    
    
    ',' --- 28  
    '5' --- 18
    '6' --- 17 
    '0' --- 15
    '7' --- 10  
    '8' --- 8 
    '4' --- 8  
    '1' --- 8
    '9' --- 6  
    '.' --- 4
    '3' --- 4 
    '\n'--- 4 
    'H' --- 4    
    'F' --- 4
    '2' --- 3 
    'A' --- 3
    'N' --- 2 
    'B' --- 2 
    'M' --- 1 
    'Q' --- 1 
    
    [(',', 0), ('5', 1), ('6', 2), ('0', 3), ('7', 4), ('8', 5), 
     ('4', 6), ('1', 7), ('9', 8), ('.', 9), ('3', 10), ('\n', 11), 
     ('H', 12), ('F', 13), ('2', 14), ('A', 15), ('N', 16), ('B', 17), 
     ('M', 18), ('Q', 19)]
    

So instead of storing for example ord('H') = 72, I give H the value 12, and so
on.

But, when I change all the chars to my values, my generated cvs(>40MB) is
still larger than original(19MB).

I even tried the alternatives to divide the list into 2. i.e. for one row make
it two rows.

    
    
    [6NH8,F,A,0,]
    [60541567,60541567,78.78,20]
    

But still larger, even larger than my "huffman" version.

**QUESTION** : Anybody have any suggestions on how to 1.Read a .csv file,
2.use something thats a lib. or 3rd party. 3.generate and write a smaller .csv
file?

For step 2 Im not asking for a full computed solution, just suggestions of how
to minimize the file, by i.e. write each value as one list ? etc.

Thank you


Answer: Try running your algorithm on the contents of each cell instead of individual
characters and then creating a new CSV file with the compressed cell values.

If the data you have provided is an example of the larger file you may want to
run the compression algorithm on each column separately. For example it may
only help to compress columns 0,4 and 5.

For reading and writing CSV files check out the
[csv](https://docs.python.org/2/library/csv.html "csv") module where you can
do things like:

    
    
    import csv
    with open('eggs.csv', 'rb') as csvfile:
        spamreader = csv.reader(csvfile, delimiter=' ', quotechar='|')
        for row in spamreader:
            print ', '.join(row)
    

