I'm working on a web application at work that will allow clients to upload CSV files (that adhere to almost zero standards unfortunately), will parse the CSV for the appropriate data and then display the data in a matplotlib graph with a [Pandas](http://pandas.pydata.org/) [DataFrame](http://pandas.pydata.org/pandas-docs/dev/generated/pandas.DataFrame.html) object to a Django HTML template. 

Below is an example CSV file that could be uploaded:

```
Shock Name, 
Shock ID, 
Vehicle, 
Location, 
Compression Valving, 
Rebound Valving, 
Piston Valving, 
Other Valving, 
Compression Setting, 
Rebound Setting, 
Preload Setting, 
Notes, 
, 
, 
Measured_Stroke, 2.00 in
Test_Temperature, 79.58 F
Peak_Velocity, 9.98 in/sec
, 
Amplitude, 1.00 in
Test_Period, 0.01 sec
Gas_Force, 9.25 lbs
Test_Speed, 1.00 in/sec

Compression Velocity, Compression Force, Rebound Velocity, Rebound Force
in/sec, lbs, in/sec, lbs
-8.373589E-03, 6.810879, -8.373589E-03, 6.810879
-0.9864202, 140.6932, 0.9310969, -170.4664
-1.97424, 158.4015, 1.915599, -388.0251
-2.984882, 171.0502, 2.903838, -410.7928
-3.976808, 178.6395, 3.910722, -425.9714
-4.987449, 186.2288, 4.898961, -441.15
-5.941944, 191.2883, 5.905845, -451.269
-6.952637, 198.8775, 6.894975, -463.9178
-7.963353, 203.937, 7.865953, -474.0368
-8.955353, 208.9965, 8.855605, -486.6855
-9.947352, 214.056, 9.882603, -494.2748

```

The various variable-esque names at the top of the file are merely settings saved by the original program that generated the CSV. The relevant data begins with the row including `Compression Velocity, Compression Force, etc...` and then continues until the end. In fact the only really relevant data needed for the plot is the first and second column, namely: `Compression Force` and `Compression Velocity`. The row with the units below the header is needed because the graph must be in a metric format so upon construction of the plot the values in the appropriate columns will need to be converted (e.g. "in/sec" to "meters/sec", etc.), but is not a part of the graph.

My idea was to scan the CSV until an instance of `"Compression Velocity"` is found and then use that row as the header row for the DataFrame. I believe I accomplished that correctly, but couldn't get it to construct the graph correctly. Below is my attempt:

```
def graph(request):
    pd.set_option('display.mpl_style', 'default')
    plt.rcParams['figure.figsize'] = (15,10)

    # get filename from sessions id
    new_file = request.session.get('docFile')

    # Process csv file --> 
    raw_data = open(new_file, 'rb').read()  
    rows = re.split('\n', raw_data.decode())

    for index, row in enumerate(rows):
        cells = row.split(',')
        if 'Compression Velocity' in cells:     # scan for the string 'Compression Velocity' in csv file read   
            header_names = cells
            header_row = index
            break
        else:
            header_names = {''}
            header_row = 0

    fig = Figure()
    ax = fig.add_subplot(111)
    ax.set_xlabel("Velocity")
    ax.set_ylabel("Force")
    data_df = pd.read_csv(new_file, header=header_row-2)
    data_df = pd.DataFrame(data_df)
    data_df.plot(ax=ax, title="Roehrig Shock Data", style="-o")

    canvas = FigureCanvas(fig)
    response = HttpResponse( content_type = 'image/png')
    canvas.print_png(response)
    return response

```

After that didn't work I attempted to parse the CSV and store the appropriate rows into a NumPy array and then construct the DataFrame from the array, but that also didn't work. Below is my attempt at that:

```
raw_data = open(new_file, 'rb').read()  
    rows = re.split('\n', raw_data.decode())

    for index, row in enumerate(rows):
        cells = row.split(',')
        if 'Compression Velocity' in cells:     # scan for the string 'Compression Velocity' in csv file read   
            header_names = cells
            header_row = index
            break
        else:
            header_names = {''}
            header_row = 0

    useable_data = []
    csv_reader = csv.reader(open(new_file, 'r'))

    for row in islice(csv_reader, header_row, None):
        if 'Compression Velocity' in cells:
            useable_data.append(row)
        else:
            continue

    useable_data = np.array(useable_data)

    fig = Figure()
    ax = fig.add_subplot(111)
    ax.set_xlabel("Velocity")
    ax.set_ylabel("Force")
    data_df = pd.read_csv(new_file, header=header_row)
    data_df = pd.DataFrame(useable_data, columns=['Compression Velocity', 'Force'])
    data_df = pd.DataFrame.from_csv(new_file, header=header_row, index_col=True)
    data_df.plot(ax=ax, title="Roehrig Shock Data", style="-o")

    canvas = FigureCanvas(fig)
    response = HttpResponse( content_type = 'image/png')
    canvas.print_png(response)
    return response

```

I know it's difficult to ask a question on SO with no error messages, but my problem is that I don't have any error messages to give at the moment. Everything that is going wrong is based on logical errors rather than syntactical. If you can think of a better method to accomplishing this goal then by all means I would love to hear it, or if you can spot the blaring errors that are obviously existing in my above code that could produce a solution then I would love that as well. Thanks for the help.

---

**Edit:**
---------

My current Dataframe:

```
-8.373589E-03, 6.810879, -8.373589E-03, 6.810879.1
-0.9864202,140.6932,0.9310969000000001,-170.4664
-1.9742400000000002,158.4015,1.915599,-388.0251
-2.984882,171.0502,2.903838,-410.7928
-3.976808,178.6395,3.9107220000000003,-425.9714
-4.987449,186.2288,4.898961,-441.15
-5.941944,191.2883,5.905844999999999,-451.269
-6.952636999999999,198.8775,6.894975,-463.9178
-7.963353,203.937,7.865953,-474.0368
-8.955353,208.9965,8.855605,-486.6855
-9.947352,214.05599999999998,9.882603,-494.2748

```

Per the suggestion of Paul H. I added the skiprows parameter, but since I am skipping the rows with the units it also seems to be ignoring the column headers row as well. 

### Data read

```
data_df = pd.read_csv(new_file, index_col=0, skiprows=header_row+2)  # skip the row with the units
    data_df = pd.DataFrame(data_df)

```