How to arrange lines into colums of csv file with python?
Question: I am a chemistry student and I am interested in performing conformational
analysis of molecules. I have performed a Potential Energy Surface Scan on the
cumaric acid in order to find the most stable conformer. With this simple
process the different spatial arrangements due to rotations of groups of atoms
about a bond are visualized. The image **pes_molecule.png** of the molecule
shows clearly the two varying dihedrals of the chain.

The program used for this purpose is called Gaussian 09 and gives the
following **pes5part.csv** output for the first five conformers:

    
    
                       1         2         3         4         5
     Eigenvalues --  -570.08934-570.08821-570.08676-570.08521-570.08384
           B1           1.38384   1.38327   1.38324   1.38348   1.38413
           B2           1.38571   1.38662   1.38692   1.38687   1.38631
           A2         119.68274 119.74315 119.80026 119.84218 119.85816
           B3           1.39004   1.38856   1.38754   1.38685   1.38683
           A3         119.90377 119.88911 119.86542 119.83707 119.82679
           D3         359.78590 359.83552 359.88306 359.93484 359.98413
           B4           1.37736   1.37902   1.38023   1.38107   1.38117
           A4         119.75636 119.73537 119.72486 119.72923 119.74312
           D4           0.71367   0.72647   0.69117   0.56509   0.38069
           B5           1.39645   1.39466   1.39330   1.39215   1.39158
           A5         121.33129 121.30763 121.28873 121.27166 121.23298
           D5           0.35956   0.44698   0.45240   0.42630   0.33448
           B6           1.47220   1.47528   1.47926   1.48347   1.48738
           A6         122.40820 121.98088 121.61637 121.36363 121.16036
           D6         180.48284 181.09688 181.65183 182.01495 181.86758
           B7           1.32697   1.32601   1.32486   1.32369   1.32268
           A7         126.15279 125.45399 124.91354 124.58356 124.35302
           D7         326.35068 316.35068 306.35068 296.35068 286.35068
           B8           1.47594   1.47706   1.47838   1.47958   1.48079
           A8         119.99708 120.12965 120.23195 120.29720 120.33716
           D8         180.53457 180.77470 180.92143 180.91869 180.76068
           B9           1.07411   1.07413   1.07416   1.07418   1.07420
           A9         118.93985 118.98599 119.01911 119.04122 119.05329
           D9         181.37285 181.38492 181.22672 180.94401 180.58221
           B10          1.34694   1.34770   1.34843   1.34907   1.34959
           A10        122.64744 122.58131 122.55418 122.55000 122.56749
           D10        180.42161 180.46502 180.42820 180.34924 180.21926
           B11          1.07626   1.07630   1.07630   1.07624   1.07612
           A11        119.03402 119.08722 119.10807 119.12392 119.13418
           D11        179.35212 179.21303 179.20177 179.31786 179.55673
           B12          1.07697   1.07704   1.07710   1.07715   1.07720
           A12        120.07413 120.05334 120.01240 119.97693 119.94390
           D12        180.48654 180.55485 180.52338 180.39366 180.25905
           B13          1.07508   1.07529   1.07540   1.07548   1.07561
           A13        119.03861 119.18885 119.28342 119.31016 119.29960
           D13        181.28569 181.16448 180.90103 180.58626 180.30590
           B14          0.94291   0.94286   0.94282   0.94279   0.94274
           A14        111.19697 111.19860 111.17512 111.14446 111.13678
           D14        359.87694 359.98739 360.03935 359.94679 360.14975
           B15          1.33041   1.33009   1.32973   1.32951   1.32933
           A15        111.93106 111.92554 111.91202 111.89198 111.87131
           D15        180.31345 180.31345 180.31345 180.31345 180.31345
           B16          1.19235   1.19199   1.19165   1.19132   1.19107
           A16        126.00937 125.96822 125.92197 125.88559 125.85792
           D16          0.53326   0.61269   0.54073   0.55376   0.45438
           B17          1.07741   1.07759   1.07781   1.07807   1.07828
           A17        116.61938 117.00542 117.31889 117.52706 117.69428
           D17        149.32579 139.91922 130.07838 119.74879 108.88744
           B18          1.07393   1.07424   1.07440   1.07445   1.07448
           A18        123.00819 122.72745 122.54598 122.45741 122.42974
           D18          0.14076   0.61929   0.95343   1.10958   0.96334
           B19          0.94770   0.94770   0.94774   0.94780   0.94787
           A19        108.07785 108.09603 108.12787 108.16255 108.20337
           D19        180.24961 180.28903 180.28314 180.25552 180.18273
    

**My goal is to create a csv file** with the following arrangement:

    
    
        Eigenvalues    D7         D15
        -570.08934  326.35068  180.31345
        -570.08821  316.35068  180.31345
        -570.08676  306.35068  180.31345
        -570.08521  296.35068  180.31345
        -570.08384  286.35068  180.31345
    

The reason I need this is to create the 3D PES graph of the energy and the two
dihedrals and afterwards to retrieve the conformer with the lowest energy. For
this purpose I have created the following script:

    
    
    #! /usr/bin/python2.7
    
    import csv
    import re
    
    ifile =open('pes5part.csv', 'rb')
    infile = csv.reader(ifile)
    
    for line in open('pes5part.csv'):
        rec = line.strip()
        if rec.startswith('Eigenvalues') or rec.startswith('D7') or rec.startswith('D15'):
            print line
    

When the script runs the following is printed into the terminal:

    
    
     Eigenvalues --  -570.08934 -570.08821 -570.08676 -570.08521 -570.08384
    
           D7         326.35068 316.35068 306.35068 296.35068 286.35068
    
           D15        180.31345 180.31345 180.31345 180.31345 180.31345
    

So in order to proceed I need your help in order to arrange the values of the
first line of the eigenvalues in the first column. Then the values of the
second line of angle D7 to the second column and finally the values of angle
D15 to the third column as depicted at **my goal csv file ** above.ccs

The complete PES scan file output from Gaussian with all 361 conformers is the
**pesFULL.csv** :  
The final complete desired PES file created by hand with all 361 conformers
after 5 hours of typing is **pes.ods**  
while the final PES graphs are depicted in the files **pes_graph1.png** and
**pes_graph2.png**

I have attached all above files inside the shared dropbox folder  
<https://www.dropbox.com/sh/5185f19tifpfr8s/AAB8cj0-niTFGbfGtEvjmfdGa?dl=0>

Thank you in advance developers for any suggestion or help.


Answer: This is a very basic example, but it should do the job. Pay attention to use
the right delimiter. You can modify the print statement to get the right
formatting.

CSV: CSV stands for Comma Separated Values, but there are at least three
possible separators in CSV files. Tools and libraries can use semicolons,
commas or the tab character as the separator. Depending on with which
separator the file was created you have to make sure to use the same separator
when reading it. The csv library in python calls the separator delimiter.
Since the input file wasn't posted I can't know which separator is being used
in it.

    
    
    import csv
    
    D = list(csv.reader(open(r"pes5part.csv"), delimiter=";"))
    for l in zip(*filter(lambda e: e[0].strip() in ["Eigenvalues", "D7", "D15"], D)):
        print "\t".join(l)
    

Of course doing it step-wise is not necessary, but this way I find it easier
to read.

Upon further study of your question and example, i think the issue is that
although the file has the csv extensions, it's not a proper CSV. So try this
instead:

    
    
    import re
    
    splitter = re.compile("\s+")
    D = [splitter.split(a) for a in open(r"pes5part.csv").readlines()]
    for l in zip(*filter(lambda e: e[0] in ["Eigenvalues", "D7", "D15"], D)):
        print "\t".join(l)
    

