Simple loading time series and plot in python
Question: Being a beginner with python, I am very frustrated because after hours of
research I can not find a solution for reading+plotting time serieses in
python which could be done in matlab, R or gnuplot in 1 minute.

Data file:

    
    
    # id  date                value1  quality anothervalue value2
       1  2011-05-19_16:30:19  974.3  3_1x    NODATA        10E-4
    ...
    

I tried this:

    
    
    import pandas as pd
    import numpy as np
    from pandas import Series, DataFrame, Panel
    import matplotlib as mp
    import matplotlib.pyplot as plt
    import matplotlib.dates as mdates
    a,b,c,d,e,f = np.loadtxt("dataset.dat", unpack=True,
        converters={ 1: mdates.strpdate2num('%Y-%m-%d_%H:%M:%S') })
    plt.plot_date(x=b, y=c)
    plt.show()
    

Python keeps failing because it tries to automatically convert the data into
float. So I tried to use the converters, but it fails for the other columns.
No plot at all.

  * how do I tell python to ignore lines beginning with # ?
  * do i have to convert everything by hand or can i just read in a dataset without conversion?
  * how to finally plot value1 over datetime?
  * can I tell python to interpret NODATA as a value without data? So it does not plot it?

Is python so badly documented that one could not google a solution after hours
or did I just miss something really practically useful out there?


Answer: Not sure if you can achieve desired result with numpy, as it's not well suited
for arrays of different types. But as you import pandas, you can use
`pd.read_csv`:

    
    
    >>> from StringIO import StringIO
    >>> s = """# id  date                value1  quality anothervalue value2
    ... 1  2011-05-19_16:30:19  974.3  3_1x    NODATA        10E-4"""
    >>> pd.read_csv(StringIO(s[2:]), sep='\s+', 
    ...      date_parser=mdates.strpdate2num('%Y-%m-%d_%H:%M:%S'), 
    ...      parse_dates=['date'])
       id          date  value1 quality anothervalue  value2
    0   1  734276.68772   974.3    3_1x       NODATA       0
    

