python pandas dataframe from file
Question: I want to create a dataframe object from a file. The file looks something
similar to this

    
    
    Gibberish Header1
    Gibberish Header2
    Gibberish Header3
    Gibberish Header4 (etc)...
    TAG THING_I_WANT_AS_COLUMN_NAME Column1
    1.0  # I'll want this index as data 1 
    1.2  # I'll want this index as data 2 
    1.3  etc
    TAG THING_I_WANT_AS_COLUMN_NAME Column2
    1.1
    1.1
    1.7
    

I would want the DataFrame to look similar to:

    
    
           Column_1    Column_2 
    data1  1.0         1.1
    data2  1.2         1.1
    data3  1.3         1.7
    

Is there a way to do this without writing a function to turn this into a
dictionary that can be write directly into the DataFrame class? I was thinking
possibly reshape, but I'm not really sure how that would work in my case. I
know writing it to a dict is an easy thing to do, but these files can be quite
large, so it might dramatically slow me down and even not be possible with my
RAM.

Any help, even a point in the right direction would be greatly appreciated.

Thanks!


Answer: When creating a pandas dataframe from scratch I usually do... import pandas as
pd

    
    
    # create array for each column
    col1 = [float(35*x) for x in xrange(10)]
    col2 = [float(220*x) for x in xrange(10)]
    
    dict = {'col1_name': col1, 'coll2_name': col2}
    df = pd.DataFrame(dict)
    

Like the other people have said, you'll have to parse the data first. It's
hard to say exactly what without seeing the actual data. You could potentially
loop over each line in the file, ignore headers, once you hit the line with
column name and stuff set a flag. Then you can start appending values to an
array. Once you've got both sets of numbers in arrays, build a dictionary like
I did up there and you'll have a dataframe!

