Python: Pandas, dealing with spaced column names
Question: If I have multiple text files that I need to parse that look like so, but can
vary in terms of column names, and the length of the hashtags above:
![txt.file](https://s4.postimg.org/8p69ptj9p/feafdfdfdfdf.png)

How would I go about turning this into a pandas dataframe? I've tried using
`pd.read_table('file.txt', delim_whitespace = True, skiprows = 14)`, but it
has all sorts of problems. My issues are...

All the text, asterisks, and pounds at the top needs to be ignored, but I
can't just use skip rows because the size of all the junk up top can vary in
length in another file.

The columns "stat (+/-)" and "syst (+/-)" are seen as 4 columns because of the
whitespace.

The one pound sign is included in the column names, and I don't want that. I
can't just assign the column names manually because they vary from text file
to text file.

Any help is much obliged, I'm just not really sure where to go from after I
read the file using pandas.


Answer: Consider reading in raw file, cleaning it line by line while writing to a new
file using `csv` module. Regex is used to identify column headers using the
_i_ as match criteria. Below assumes more than one space separates columns:

    
    
    import os
    import csv, re
    import pandas as pd
    
    rawfile = "path/To/RawText.txt"
    tempfile = "path/To/TempText.txt"
    
    with open(tempfile, 'w', newline='') as output_file:
        writer = csv.writer(output_file)    
    
        with open(rawfile, 'r') as data_file:
            for line in data_file:            
                if re.match('^.*i', line):                     # KEEP COLUMN HEADER ROW
                    line = line.replace('\n', '')                
                    row = line.split("  ")                
                    writer.writerow(row)
    
                elif line.startswith('#') == False:            # REMOVE HASHTAG LINES
                    line = line.replace('\n', '')
                    row = line.split("  ")            
                    writer.writerow(row)
    
    df = pd.read_csv(tempfile)                                 # IMPORT TEMP  FILE
    df.columns = [c.replace('# ', '') for c in df.columns]     # REMOVE '#' IN COL NAMES     
    
    os.remove(tempfile)                                        # DELETE TEMP FILE
    

